I believe AC wallwarts are typically just plain old transformers, which have no closed loop feedback (or any silicon at all, for that matter), and so they do sag under load a lot more than an SMPS. So if it's a dumb transformer, the output voltage will be a function of output current (i.e., "load"), AND the mains electricity input voltage, which isn't a precisely fixed number either.
So I think your suspicion is correct; the voltage will drop when you put a load on it. How much it drops will depend how much load. You could use a big resistor (say you want to draw 1A and 15V, that's 15 ohms. But that'll burn off 15V * 1A = 15W of heat, most normal resistors can only dissipate 0.25W, so you need a
big resistor]), or you could just use the DSL box it's designed to power...