DC-DC have costs for their own needs and losses.
This part is constant and therefore becomes noticeable at low load.
If you are carrying an empty backpack, you feel its weight. When you put a 16kg dumbbell in your backpack, you won't notice the whole backpack.
You should not worry about efficiency, but about your own DC-DC consumption.
Usually, low-power sources have less self power than high-power ones.
Just choose a DC-DC close to the rated power.