Alright, well, I did some more careful testing.
I decided to leave the scope in AC coupled mode just to put it into the same point of view of my cheap multimeters. I allowed everything to warm up over an hour and then did some careful measurements.
In order to sort of establish how well things were accurate, relative to each other, I ran a 60 Hz sine wave first. That ought to be something any reasonable meter can measure, right? And indeed, all instruments were under 2% of error, which is pretty good for my bunch. I'm using the HP scope as my reference since it actually arrived calibrated (it's since lapsed, but not that long ago).
I then did a basic sweep to see where each meter fell off 3dB (power). Most of my meters provided plenty of bandwidth for audio-range signals, but the Owon HDS272S surprised me at only 4 KHz. After about 5 KHz it's basically deaf. The Tenma 72-410A went out all the way to 236 KHz before losing 3dB, however, it was not very flat along the way, showing little bumps of a 100mV here and there. The Owon B35T went out to 600 KHz before dropping 3dB, which surprised me. This little meter seems to hold its own pretty well! Even the RadioShack (22-174B) held out to a respectable 60 KHz.
Next came the square wave test. I did this at 60 Hz and at 1 KHz. Given the bandwidth of some of these meters, there was no point going out much further. The HP400EL didn't do so well on the square wave test, despite its massive bandwidth advantage, simply because it is not an RMS meter. It is calibrated to show the RMS level of a sine wave, but it lacks a circuit to actually compute or measure RMS. So despite it's massive bandwidth (well over 10 MHz), it is simply not accurate for a square wave.
As for the other meters, they did pretty well except for the Owon HDS272S, which probably did poor simply due to it's rather pathetic 4 KHz bandwidth. This is somewhat ironic given the scope portion of this meter can read out to 70 MHz... Add to this that the scope lacks an RMS measurement readout. Go figure. It's still a nice meter though, lovely bright display and very handy in tight spots to have a scope.
The standard deviation on the sine wave test was 14.6mV, the 60 Hz square wave was 106mV, and the 1 KHz square wave was 129mV. So you can see that indeed there is far more variation as you move up in frequency and some of these meters start to fall apart. I wouldn't read too far into this though since my meters are not exactly high-quality nor are they even similar to each other.
A little story on the radioshack meter. I got it when I worked at radioshack and it went on clearance for $29.97. With my employee discount it was probably half that price. I thought it was pretty great, it had HFE, capacitance, a dB scale, a bargraph, all the sorts of things a kid in the 90s would consider pretty premium stuff. Years later I was working as a tech for a guy and I noticed that the meter had poor performance over about 10 KHz. The guy I was working for had a basement full of old vacuum tube HP equipment, so I dragged up an enormous HP meter and compared the two, and indeed, the RS meter was off a few dB at the high end. Now comes the fun part. I had a warranty on the meter because, why not, it was dirt cheap as an employee, so I drove to the local radio shack store. I walked in and proceeded to plug in an HP oscillator and an HP meter, and planned to demonstrate how poor the performance was. The poor guy behind the counter had no idea what was going on and just offered to let me pick a similar model to replace it (since they didn't have it anymore). I picked the most expensive one they had. But! When I tested it, it was worse!! So I left with my original meter. That evening, I took the meter apart and discovered several pots inside it. After a few hours of playing around in the basement of HP equipment, I had it flat to 20 KHz (which was all I needed at the time). What surprises me about all this, is that "calibration" happened in around 2003. The meter's plastic melted in my car and several sets of batteries corroded up inside it. And yet, here it is, 20 years later, showing under 2% of error. It still has holes drilled in the back over the pots with little bits of tape over the holes.
Anyway, that's my story! Results are attached.