A very interesting site to compare certain multimeters

[Lightages]

This person has put together a rather informative way of comparing multimeters. Perhaps he will add more in the the future. The one thing, it appears to be a comparison of samples of one. As interesting as this is, it cannot represent the true statistical accuracy of any certain model.

http://www.johannes-bauer.com/electronics/mmcomparison/app/index.html

[alm]

Looks cool, but the value is somewhat limited in my opinion. As you state, n=1 makes the significance of the differences almost zero. It would also be better to have several measurements over the life of the instrument. I find it interesting that there is no uncertainty in the errors, is the source that good that uncertainty is insignificant compared to the error? I kind find any mention of the source and its uncertainty.

But it's not like I've made anything better, and I'm sure it will be used for arguing which multimeter is better, despite the issues.

[EEVblog]

This is what everyone seems to complain about my review - "why don't I compare it with a reference to see how "accurate" it is?"
Yup, sample size of 1...

And when I do for whatever reason they scream "see, the cheapo meter is more or just as "accurate" as the expensive meter!"

Dave.

[Rufus]

Looks cool, but the value is somewhat limited in my opinion. As you state, n=1 makes the significance of the differences almost zero.

There is no n=1. He hasn't measured any meters. The site lets you graphically compare manufacturers specifications across the full measurement range. It shows you the combined effect of counts, full scale and 'digits' accuracy. Very useful, but, you still have to decide how much you trust specifications from the different manufacturers.

[Lightages]

There is no n=1. He hasn't measured any meters. The site lets you graphically compare manufacturers specifications across the full measurement range. It shows you the combined effect of counts, full scale and 'digits' accuracy. Very useful, but, you still have to decide how much you trust specifications from the different manufacturers.

Aha, you are right. I skimmed the description and assumed.... well you know what happens when you assume. Still an interesting way to look at the specs at least.

[Fraser]

Just to add to Dave's comments

Disregarding sample size for a moment.

I would compare expensive and cheap meters to my car....My Audi sails through it's yearly MOT with no faults or tolerance issues....yet a cheaper car (I will not pick on one) will start to show it's age after around 3 years and the MOT will reveal all manner of tolerance drifts, such as braking balance, corroded parts and deteriorating bushes etc.

What is my point you may say....well if you buy a very high quality meter with a good reputation you can be assured that it will likely stay pretty accurate and perform well for many years even if no formal calibration is carried out. Quality components with low drift specs are attached to a well designed, environmentally stable PCB substrate as part of a performance design.

Now a very cheap (and I mean very cheap) meter will be built to minimum cost, lowest cost components and PCB.....would you expect it to keep up with the higher quality meter in terms of maintaining accuracy and reliability over many years ? I wouldn't but if it does, then it would be luck rather than by design IMHO.

The grey area is in the middle ground where you have meters that are neither expensive nor cheap and are mid-range quality. These seem to perform well in tests and in my experience seem to maintain decent accuracy and reliability....BUT if I were a professional relying upon a meter, I would still have that element of doubt about continued accuracy over a long period of time or differing environmental conditions. 

I have a full range of hand and bench meters in my lab. The Flukes are simply gorgeous to use. I have the 87 III and several 8840A bench meters. The build quality of these meters is superb and all tested accurate and on spec against my offices standards and they are anything but new ! Now I also have AVO Megger meters that also proved to be accurate as per specs and are well made. Moving down to the UNI-T range of meters, of which I own around 10 different units including the UT81 hybrid, these were accurate and to spec when I bought them but I have not re-checked them recently as I lost access to a decent standard at work .... they disposed of it  :-( I actually like the UNI-T meters and believe they offer excellent value for money. I would go as far as recommending them to the hobbyist. Now we get to the cheapo end of the market that provides a meter with an 'amazing' range of features for a very low price.... well the shocker here was that four cheap Maplin and Rapid meters that I owned all proved to be within their specs when tested new 3 years ago....now the killer.....none of them is still working and they ended up being stripped for parts and the excess binned. The build quality was truly chronic so it was no surprise that they keeled over in short time. The faults ranged from obvious inaccuracy to intermittent operation and failure to provide a stable reading. I used them in my cars as emergency meters so guess they didn't like temperature swings.

Just my 2 cents worth. 

[Bored@Work]

If you want to say that this comparison ignores or doesn't cover certain aspects of a meter then you are of course right.

What this website IMHO does is comparing specs. Which is one annoying part when looking for a meter. So that is a nice service. It does, however, skip some aspects, like 24h or 1 year stability. So it isn't exactly clear if the compared values are in fact comparable.

What the site IMHO shows is that comparing meter specs is a tricky business, and that some meter specs are just too good to be true. E.g. when a Tonghui beats an Agilent in the specs I get nervous.

[qno]

I do not understand the graphs.
What is he comparing?

I would expect the error to increase at lower voltages.
Meters are calibrated and specified with the full scale as reference.

An example; if you have a range of 100 VDC and a 1% accuracy.
When you connect a very accurate 100.0 volt on the input  any indication between 99.0 and 101.0 is within spec.
If you now lower the voltage to an accurate 10 V on the input and do not change the measuring range any indication on the display between 9 and 11 Volt is within spec but the accuracy has dropped to 10% of the measured value.

[Uncle Vernon]

Just to add to Dave's comments

Disregarding sample size for a moment.

I would compare expensive and cheap meters to my car....

Good analogy, mine is similar.
I'd compare multimeters with cameras. Think of your Fluke or Agilent in the same light as a professional camera, your better Extech Uni-T etc as a consumer SLR and the rest as fixed lens happy snappers.
On a good day when the sun is right andyour subject still a consumer SLR will results almost as good as a professional camera, but when it's dark, or cold or hot and the subject isn't cooperating only a professional instrument can be trusted day in day out. Same applies for your meters and test equipment. Therearen't too many specs there for long term repeatability and reliability yet those are what differentiates the tools from the toys.

[alm]

There is no n=1. He hasn't measured any meters. The site lets you graphically compare manufacturers specifications across the full measurement range. It shows you the combined effect of counts, full scale and 'digits' accuracy. Very useful, but, you still have to decide how much you trust specifications from the different manufacturers.

You're right, I was wrong assuming they were actual measurements, my comments don't apply. It's a fun tool, although it does miss some details. Are they all 12month specs? Some meters specify 24h, 1month and 12month, and sometimes even different temperature ranges. And of course you're only comparing what the manufacturer tells you, which may be more reliable for some brands than others. Even the cheap crap will usually specify some unrealistic level of accuracy. But I guess it's a nice way to visualize the x% of value plus x% of full scale spec.

I would expect the error to increase at lower voltages.
Meters are calibrated and specified with the full scale as reference.

The error usually consists of a percentage of value plus a percentage of full scale (see Dave's video about digits and counts, it was one of his early videos). If the temperature is different from the temperature it was calibrated at, this also introduces extra error.

On a good day when the sun is right andyour subject still a consumer SLR will results almost as good as a professional camera, but when it's dark, or cold or hot and the subject isn't cooperating only a professional instrument can be trusted day in day out.

Consumer SLRs can be surprisingly sturdy and reliable, which is why some professionals will take a consumer SLR when they don't want the bulk of a pro body and don't need its speed. Think Galen Rowell with an FM2/FM10 and consumer grade lenses. Image quality of a modern consumer DSLR also tends to be very close or sometimes better (if the sensor is newer), as long as the AF can cope. Metering tends to be very good in consumer bodies. It's also quite uncommon for a consumer SLR to blow up in your face if you try to take a picture of something it can't handle.

Therearen't too many specs there for long term repeatability and reliability yet those are what differentiates the tools from the toys.

Long term repeatability should be in the accuracy specs, if a meter is within 0.03% with a 1 year calibration interval, that means that the uncertainty should be always lower than that as long as you calibrate it every year. Of course whether these specs can be trusted is a different matter. Few manufacturers specify reliability apart from warranty period. Even warranty period isn't a good indicator, since they can also increase the price and use the extra profit to invest in the extra warranty repairs.

Long term experiences of a large population of meters tends to be the best way, which is how we know that eg. a Fluke 87 or Agilent 34401A tends to be quite reliable. Calibration departments tend to have access to this kind of data, but rarely share this with the outside world. They will usually only have this data for equipment that is worth calibrating, is bought by the kind of organizations that do large-scale calibration, and is on the market for a number of years.

[Bored@Work]

Meters are calibrated and specified with the full scale as reference.

No, only analog ones are. Digital ones are specified as percentage of reading (and not scale), plus counts (that is a possible offset).

You can convert the counts back to a full scale reference, because the counts offset is independent of the reading, but why bother?

[alm]

You can convert the counts back to a full scale reference, because the counts offset is independent of the reading, but why bother?

To compare the accuracy of meters with different number of digits, like on the site that's the topic of this thread?

[Bored@Work]

For that you don't have to convert to full scale, but just to multiply with the range's resolution.

[saturation]

The website has promise, but comparing published specs from UniT vs Agilent or Fluke is problematic.  I think some users on eevblog have posted real tests of certain UniT model accuracy and its between 2-5x worse than published, while Fluke is 2x better than published.

Also from this page, I'm not sure where he gets is numbers from

http://www.johannes-bauer.com/electronics/mmcomparison/?menuid=2

But the Fluke 87V ratings I know are published as ~ 0.05% on VDC, a factor of >10x better than what he uses, and the UniT can be very variable but unlikely what they publish in their spec sheet.