Author Topic: How reliable are accredited calibration laboratories  (Read 5458 times)

0 Members and 1 Guest are viewing this topic.

Offline CrazyTigerTopic starter

  • Regular Contributor
  • *
  • Posts: 95
  • Country: my
How reliable are accredited calibration laboratories
« on: June 21, 2021, 01:23:59 am »
Hi,

I am sharing my recent experience sending two of my repaired signal generator for calibration. This lab is a well know international company. They were ANAB accredited for ISO17025 and I checked with ANAB, they were indeed accredited for the scope of my calibration. I was charged a premium price for the calibration. Upon completion of the calibration I was furnished with 18 pages of calibration report, and this where the horror began.

1. The standard they used for calibration: the power sensor starts 10 MHz, but they measured power down to 300 kHz with that sensor and it passed the test, must be a miracle sensor.
2. I have two pages of testing of LF Output of the signal generator, they did frequency response test, accuracy of output voltage, frequency and so on, here is the kicker both my unit don't have LF output.
3. Another pages was for pulse modulation, my unit does not have pulse modulation option either.
4. SSB phase noise test, they used a spectrum analyzer with phase noise worse then my sig. gen, and yet the test pass superbly, though the limit they stated is wrong.

At first i was thinking, maybe they used a wrong template, till i got the other certificate, same horror show and the result stated is different indicating they must be using a random number generator  |O

In this case my question is how is ANAB certifying all this labs, are they also just giving out accreditation. 
How many people have encountered this kind of laboratories?
Basically the are just churning out fake certificates  :'(



 

Offline DuPe

  • Regular Contributor
  • *
  • Posts: 110
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #1 on: June 21, 2021, 09:25:45 am »
What is their  (cal lab) Statement to your findings?
 

Offline Henrik_V

  • Regular Contributor
  • *
  • Posts: 155
  • Country: de
  • “ground” is a convenient fantasy
Re: How reliable are accredited calibration laboratories
« Reply #2 on: June 21, 2021, 11:32:50 am »
First would be to contact the lab.
Next escalation steps IAAC and finally ILAC...

In the US you have  4 organisations for accreditation (according to IAAC), nice for the labs, but for metrological quality ??

So the best a customer can do is to ask for a quote for your type of instrument including the typical/expected* uncertainty, references and methods.
*) sometimes they could not be reached, but the lab should answer the question why.

Since I am a expert assessor myself, I can only answer your question for Germany and in the area of acceleration....  ;) 

Would like to see follow-ups
« Last Edit: June 21, 2021, 11:59:41 am by Henrik_V »
Greetings from Germany
Henrik

The number you have dialed is imaginary, please turn your phone 90° and dial again!
 

Offline HighVoltage

  • Super Contributor
  • ***
  • Posts: 5453
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #3 on: June 21, 2021, 12:02:46 pm »
If you have found already so many mistakes, I would not be surprised, if you will find more by looking in to their reference equipment. This seems to be very sloppy.

Some companies just need a calibration certificate and they do not care about the numbers shown and they are happy that they have a valid certificate to fulfill their obligation to ISO requirements.

I just had a similar experience with an expensive vacuum gauge that was reading far off.
The gauge showed -960 mbar when in reality it should be reading -999 mbar

It was sent in for calibration with adjustment.
Adjustment was not done!
And a perfectly looking calibration certificate came back with the gauge, showing all values within its specifications.
And this was a well known calibration lab in Germany.
The invoice stated adjustment and total price was several hundred Euros.
 |O
There are 3 kinds of people in this world, those who can count and those who can not.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 9889
  • Country: nz
Re: How reliable are accredited calibration laboratories
« Reply #4 on: June 21, 2021, 12:36:45 pm »
Some companies just need a calibration certificate and they do not care about the numbers shown and they are happy that they have a valid certificate to fulfill their obligation to ISO requirements.

Sad but true. 
Greek letter 'Psi' (not Pounds per Square Inch)
 

Offline binary01

  • Contributor
  • Posts: 37
  • Country: au
Re: How reliable are accredited calibration laboratories
« Reply #5 on: June 21, 2021, 01:10:09 pm »
Yes, I agree with others; like many commercial industries, there are good and bad operators and accreditation doesn't seem to always protect you, unfortunately. 
I think it is a problem in the industry, with some labs trading on neivity of there customers with respect to metrology, issuing poor reports and a new calibration sticker.
I have certainly had problems with larger well-known accredited labs, and the response when raising it with them doesn't always fill me with confidence. Sometimes there are legitimate mistakes, sometimes systematic errors or lack of care.
However, there are definitely good and thorough accredited labs around, if you do some digging (I know because I work in one 🙂, could be a biased opinion)
 

Offline mendip_discovery

  • Frequent Contributor
  • **
  • Posts: 824
  • Country: gb
Re: How reliable are accredited calibration laboratories
« Reply #6 on: June 22, 2021, 04:43:16 pm »
As someone who runs a 17025 accredited lab, I can say that is not ok and you should complain. The accreditation body should be informed if the lab doesn't rectify it to your satisfaction. I know some ISO9001 labs do some stuff that isn't quite right, I worked at one. But in the UK with UKAS we have to be very well behaved as they are on it if stuff isn't right.

When being audited recently my auditor spotted an issue on an external cert, I spoke to the lab they said they can make a statement, was told by my auditor if they do that their auditor will take a look at the worksheet...he is their auditor.
Motorcyclist, Nerd, and I work in a Calibration Lab :-)
--
So everyone is clear, Calibration = Taking Measurement against a known source, Verification = Checking Calibration against Specification, Adjustment = Adjusting the unit to be within specifications.
 

Online bdunham7

  • Super Contributor
  • ***
  • Posts: 7729
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #7 on: June 22, 2021, 11:20:57 pm »
How many people have encountered this kind of laboratories?
Basically the are just churning out fake certificates  :'(

I've heard of labs that may pencil whip a calibration, usually where it is a no-data or OOT-only data certificate and report.  I'd presume that they at least check that the meter works or comes from a client that wouldn't send a broken meter in for calibration.  I think this probably happens where the customer just needs the paperwork and the cal tech or lab knows it.  The certificates are often very brief and have no real information on it. I'd call that a 'good enough for who it's for' job.

Then there's the lab that is underequipped and tries to get by with what they have.  We'll call that the 'doing the best we can' job.  This can range from questionable to downright useless.  In some cases the data may be fake-ish or overly optimistic, as frequently seen in eBay sales. 

However I have not yet run across a supposedly reputable lab with the balls to do what happened to your stuff, for which the only explanation does seem to be that they fabricated or recycled the entire thing.  I'd have a conversation with them and see how it goes--their attitude will tell you a lot.  Keep in mind that it is sometimes permissible to characterize test standards and instruments beyond their primary advertised specifications, so your first and fourth issues could theoretically have a valid explanation.

There have been a few cal-lab discussions in the forums in the past, I think one of them was someone with a similar issue in your country, but I can't find it right now.  It's not a rare issue at all.  I'd stick with an OEM A-brand calibration lab if you can.

« Last Edit: June 22, 2021, 11:26:42 pm by bdunham7 »
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 

Offline Tony_G

  • Frequent Contributor
  • **
  • Posts: 899
  • Country: us
  • Checkout my old test gear channel (link in sig)
    • TGSoapbox
Re: How reliable are accredited calibration laboratories
« Reply #8 on: June 23, 2021, 12:13:53 am »
It's not a rare issue at all.  I'd stick with an OEM A-brand calibration lab if you can.

I have to completely agree with this statement - Once you go outside the OEM cal labs then it really is about finding a 3rd party one that does good work - There isn't anything inherently wrong with 3rd party cal labs but the quality can be so variant that you might need to try a few to find one or speak to other people in your industry to see who they use and their thoughts - If someone called me up and asked about it I'd be happy to share even if they were a competitor as everyone benefits from good cal labs.

TonyG

Offline TimNJ

  • Super Contributor
  • ***
  • Posts: 1649
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #9 on: June 23, 2021, 03:17:38 am »
How many people have encountered this kind of laboratories?
Basically the are just churning out fake certificates  :'(

Then there's the lab that is underequipped and tries to get by with what they have.  We'll call that the 'doing the best we can' job.  This can range from questionable to downright useless.  In some cases the data may be fake-ish or overly optimistic, as frequently seen in eBay sales. 


Our cal lab sent a guy to our office to calibrate our stuff over the course of three days. He left his stuff in the open after he left for the day. Mainly out of curiosity, I just wanted to see what kind of cool metrology stuff they had. Didn't touch anything, but I found no evidence of an inductance standard...and we have something like 3 or 4 high accuracy LCR meters. No decade inductor either. But the LCR meters got a sticker!! >:D

Now...maybe there was some more crates of equipment in the truck, but I don't think so. And maybe newer meters don't need an inductance standard ? I don't know...but most of the older meters put them in the calibration procedure.
 

Online bdunham7

  • Super Contributor
  • ***
  • Posts: 7729
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #10 on: June 23, 2021, 04:08:04 am »
Our cal lab sent a guy to our office to calibrate our stuff over the course of three days. He left his stuff in the open after he left for the day. Mainly out of curiosity, I just wanted to see what kind of cool metrology stuff they had. Didn't touch anything, but I found no evidence of an inductance standard...and we have something like 3 or 4 high accuracy LCR meters. No decade inductor either. But the LCR meters got a sticker!! >:D

Now...maybe there was some more crates of equipment in the truck, but I don't think so. And maybe newer meters don't need an inductance standard ? I don't know...but most of the older meters put them in the calibration procedure.

I had an LCR meter here for service, being resold by a surplus dealer.  After I patched it back together, I considered having it calibrated.  The procedure requires an OEM calibration kit with 4 characterized resistors, an open and a short fixture and that's it.  So no inductors, per se and the machine basically calibrates itself at your command.

In this case, the kit costs $4K https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf] [url]https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf [/url] and the job takes a total of 90 minutes on the bench, not including warmup and acclimatization, but only about 10 minutes of tech time.  I ended up sending it out 'as-is' (it seemed to work pretty well) because the two labs I contacted couldn't answer basic questions about the process, i.e. "do you have this kit at your location".  One eventually, perhaps accidentally, admitted that they would ship the unit to their "Massachusetts location", which happens to be where the OEM is.   The way that thing was put together it could barely survive a car ride in my cushy SUV, let along 3000 miles of UPS.
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 

Offline TimNJ

  • Super Contributor
  • ***
  • Posts: 1649
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #11 on: June 23, 2021, 02:15:03 pm »
Our cal lab sent a guy to our office to calibrate our stuff over the course of three days. He left his stuff in the open after he left for the day. Mainly out of curiosity, I just wanted to see what kind of cool metrology stuff they had. Didn't touch anything, but I found no evidence of an inductance standard...and we have something like 3 or 4 high accuracy LCR meters. No decade inductor either. But the LCR meters got a sticker!! >:D

Now...maybe there was some more crates of equipment in the truck, but I don't think so. And maybe newer meters don't need an inductance standard ? I don't know...but most of the older meters put them in the calibration procedure.

I had an LCR meter here for service, being resold by a surplus dealer.  After I patched it back together, I considered having it calibrated.  The procedure requires an OEM calibration kit with 4 characterized resistors, an open and a short fixture and that's it.  So no inductors, per se and the machine basically calibrates itself at your command.

In this case, the kit costs $4K [url]https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf]https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf] [url]https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf [/url] and the job takes a total of 90 minutes on the bench, not including warmup and acclimatization, but only about 10 minutes of tech time.  I ended up sending it out 'as-is' (it seemed to work pretty well) because the two labs I contacted couldn't answer basic questions about the process, i.e. "do you have this kit at your location".  One eventually, perhaps accidentally, admitted that they would ship the unit to their "Massachusetts location", which happens to be where the OEM is.   The way that thing was put together it could barely survive a car ride in my cushy SUV, let along 3000 miles of UPS.

I figured as much. I know the new meters have simplified calibration, a lot taken care in software/firmware I guess.
 

Offline mendip_discovery

  • Frequent Contributor
  • **
  • Posts: 824
  • Country: gb
Re: How reliable are accredited calibration laboratories
« Reply #12 on: June 23, 2021, 05:18:22 pm »

Our cal lab sent a guy to our office to calibrate our stuff over the course of three days. He left his stuff in the open after he left for the day. Mainly out of curiosity, I just wanted to see what kind of cool metrology stuff they had. Didn't touch anything, but I found no evidence of an inductance standard...and we have something like 3 or 4 high accuracy LCR meters. No decade inductor either. But the LCR meters got a sticker!! >:D

Now...maybe there was some more crates of equipment in the truck, but I don't think so. And maybe newer meters don't need an inductance standard ? I don't know...but most of the older meters put them in the calibration procedure.

Some Multi-Functional Calibrators come with stuff inside them for calibration. So don't be fooled by the state of what is on the desk.

Remember Calibration is not adjustment. Manufacturers have muddied the waters over the years and created confusion for the consumers over what calibration is and they feel that if its been calibrated its been adjusted. A lab should only adjust if its going out of specification and then they should give you before and after adjustment readings.

Quote
"Calibration is the comparison of a measuring instrument, the unit under test (UUT) to an appropriate standard that is more accurate than the UUT."

The consumer should be making efforts to choose a lab with a good uncertainty, there is no point in getting your equipment calibrated by a lab with a 1% uncertainty when your kit has a manufacturers spec of 20ppm otherwise any of the readings they have given you could be either in or out.

In response to bdunham7 talking about needing 4k of kit, would a good Decade box not cover that, looking at the specs and procedure it looks easy enough to DIY. My portable calibrator even has a simulated resistance mode that may have worked.
Motorcyclist, Nerd, and I work in a Calibration Lab :-)
--
So everyone is clear, Calibration = Taking Measurement against a known source, Verification = Checking Calibration against Specification, Adjustment = Adjusting the unit to be within specifications.
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14076
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #13 on: June 23, 2021, 05:49:50 pm »
Simulated resistance is usually only good for DC, not so much for calibrating a RLC bridge, that likely needs a limit on the parasitic inductance / capacitance.

One may get away with a set of resistors in DIY boxes, if they are properly characterized. Already a decade box may be a problem from parasitics.
 

Online bdunham7

  • Super Contributor
  • ***
  • Posts: 7729
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #14 on: June 23, 2021, 05:56:30 pm »
In response to bdunham7 talking about needing 4k of kit, would a good Decade box not cover that, looking at the specs and procedure it looks easy enough to DIY. My portable calibrator even has a simulated resistance mode that may have worked.

Not likely.  The resistor are characterized in some way (they refer to Q) and the characterization constants are marked on the units and must be entered into the machine during calibration.  A decade box isn't going to cut it.  But I'm sure there is a cal lab manager somewhere that would tell their tech to just use what they have.... |O
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 

Offline TimNJ

  • Super Contributor
  • ***
  • Posts: 1649
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #15 on: June 23, 2021, 06:55:48 pm »
Out of curiosity, I checked a few "digi-bridge" style LCR meters from the last 20-30 years. Seems they need the precision resistor set only...I guess this is acceptable because the overall topology is a Whetstone bridge?

I have an old HP 4332A LCR meter at home. It requires two standard inductors, two standard capacitors, and two standard resistors to calibrate (properly). Per my understanding, it does not use a bridge, but has a in-phase/quadrature detector, sort of like a VNA, but just for two test frequencies. Maybe that explains it.

Is bridge the prevailing LCR meter topology these days? What does the IET/ DER DE-5000 use?

Not to get too crazy off-topic here...
 

Online bdunham7

  • Super Contributor
  • ***
  • Posts: 7729
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #16 on: June 23, 2021, 08:10:23 pm »
Out of curiosity, I checked a few "digi-bridge" style LCR meters from the last 20-30 years. Seems they need the precision resistor set only...I guess this is acceptable because the overall topology is a Whetstone bridge?

I have an old HP 4332A LCR meter at home. It requires two standard inductors, two standard capacitors, and two standard resistors to calibrate (properly). Per my understanding, it does not use a bridge, but has a in-phase/quadrature detector, sort of like a VNA, but just for two test frequencies. Maybe that explains it.

Is bridge the prevailing LCR meter topology these days? What does the IET/ DER DE-5000 use?

Not to get too crazy off-topic here...

None of the ones I've seen use an actual old-school bridge with an adjustable resistor, but there are a variety of systems that are in use.  The QuadTech one I linked sends out a signal, that can be fixed or swept, through a resistor and then monitors the signal across the DUT and across the resistor--apparently this is called the I-V method.  I think it is functionally the cousin of a VNA.  Usually the handheld ones have fixed frequencies and do not load-adjust the bias voltage.  Fancier ones let you add DC bias and can amplify the bias to maintain a constant voltage across the DUT.

There are more bridge-like models as well, but AFAIK most LCR meters of either type can have the calibration constants determined with just resistors, just like you calibrate a nanoVNA with an open, a short and a 50R load.  With an LCR meter, you really have to calibrate it with the fixture or leads that you are going to use.  My BK 886, for example, has you run a CAL OPEN and CAL SHORT (with a fixture) any time you want, which typically would be every time you turn it on or change the leads.  I can understand that this is how you set the constants, but this seems to be conditioned on the meter working properly.  It baffles me that there isn't a performance test requirement with L, C and R standards. 
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 
The following users thanked this post: TimNJ

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14076
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #17 on: June 24, 2021, 07:31:50 am »
When using digital signal analysis there is little need to do an extra calibraion for 90 deg. phase shift. So a good resistor with low parasitics is good enough for the calibration to 0 phase shift.  A test with an actual resistor or capacitor from time to time would be good. More to check for defects than just drift of the internal reference resistor. Many defects are expected to be detectable in the open measurement.  If in doubt, one is free to have a small box with a few reference elements with measured values close to the RLC meter and check them from time to time.
 

Offline Oldtestgear

  • Regular Contributor
  • *
  • Posts: 144
  • Country: gb
Re: How reliable are accredited calibration laboratories
« Reply #18 on: June 24, 2021, 10:46:57 am »
I bought a Solartron 7081 off eBay a few years ago that had a current calibration sticker on it. Out of curiousity I asked the qualifying facility what a re-calibration would cost. The Sales Office quoted a low price very quickly & I was doubtful whether they could do the job to my satisfaction.  Anything better than I could do (using a calibrated Keysight 34470A + Datron calbrators) would be great. After checking their UKAS accredittation it was obvious that they could not perform any calibration to better than 6.5 digits. When I queried their abiity to do the job, they suddenly stopped chasing me for an order number.

I assume that the original customer just needed a piece of paper to satisfy an audit trail, especially as the 7081 had an offest on a couple of ranges.  I know the offest could be the reason it ended up on eBay but it is a stable & very usable instrument that I will re-calibrate ( to a limited extent) myself  "one day".
 

Offline Villain

  • Regular Contributor
  • *
  • Posts: 71
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #19 on: June 24, 2021, 02:49:16 pm »
Out of experience i can say that UKAS takes a lot more caution and care auditing and granting accreditations compared to for example ANAB and A2LA.
Although every lab - even accredited can fake as much as they want really (at least until they get busted). I would stick to reputable companies even if price is higher, except you know someone who is working in a smaller cal lab and you can be certain that the quality is good.
« Last Edit: June 24, 2021, 02:51:27 pm by Villain »
 

Offline Teti

  • Regular Contributor
  • *
  • Posts: 79
  • Country: cz
    • www.martenelectric.cz
Re: How reliable are accredited calibration laboratories
« Reply #20 on: June 26, 2021, 07:04:26 pm »
Into Russia metrology problems is same.
I see real/true measurements only into Keysight Moscow metrology lab and VNIIM D.I.Mendeleeva where store some of ours primary metrology standards.
Into other places metrology is fake.

Hi Shodan, love your voltnut YT channel! :) Sorry for OT
Vintage audio gear connoisseur, computer enthusiast, time-nut, music lover, vintage games gamer, nerd, tinkerer and shady electronic projects maker.
http://www.martenelectric.cz/
 
The following users thanked this post: shodan@micron

Offline CrazyTigerTopic starter

  • Regular Contributor
  • *
  • Posts: 95
  • Country: my
Re: How reliable are accredited calibration laboratories
« Reply #21 on: June 27, 2021, 01:55:31 am »
Update:

After sending a long complain email to the calibration provider, i also kept their Principal in US in loop. The company admitted that something went wrong in their process and they have recalled both the unit for re-calibration. They will be barring the cost for both transportation and calibration. Lets see how it goes  :popcorn:

 

Online bdunham7

  • Super Contributor
  • ***
  • Posts: 7729
  • Country: us
Re: How reliable are accredited calibration laboratories
« Reply #22 on: June 27, 2021, 03:46:56 am »
After sending a long complain email to the calibration provider, i also kept their Principal in US in loop. The company admitted that something went wrong in their process and they have recalled both the unit for re-calibration. They will be barring the cost for both transportation and calibration. Lets see how it goes  :popcorn:

Oh dear!  Reading between the lines, perhaps an A-brand OEM lab still wasn't good enough?  If so, at least they have a reputation to protect.
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 9889
  • Country: nz
Re: How reliable are accredited calibration laboratories
« Reply #23 on: June 27, 2021, 12:21:43 pm »
Update:

After sending a long complain email to the calibration provider, i also kept their Principal in US in loop. The company admitted that something went wrong in their process and they have recalled both the unit for re-calibration. They will be barring the cost for both transportation and calibration. Lets see how it goes  :popcorn:

Say that you expect a detailed engineering level explanation about how this situation occurred and that you will not accept a meaningless statement from their PR department.
Greek letter 'Psi' (not Pounds per Square Inch)
 

Offline HighVoltage

  • Super Contributor
  • ***
  • Posts: 5453
  • Country: de
Re: How reliable are accredited calibration laboratories
« Reply #24 on: June 27, 2021, 03:28:00 pm »
I would ask for a detailed 8D-Report.

If they know what that is, it will be easy for them to prepare one.
If they don't know what it is, they have to figure it out.

If you want, you can black out their company details and post this 8D report here for us.
This will be very interesting.
There are 3 kinds of people in this world, those who can count and those who can not.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf