How to create small (mK) temp differences to test cameras?

[Max Planck]

OK, I have checked my reference books.

You really want to be measuring Minimum Resolvable Temperature Difference (MRTD) and Minimum Detectable Temperature Difference (MDTD). NETD can be very misleading !

[...]

I can not agree. MRTD is not a single numerical value, but a curve (MRTD vs cycle/mrad) limited on one side by the camera IFOV and on the other side by the system noise (which can be described using NETD). NETD remains the best solution.
An example of MRTD:
http://publications.drdo.gov.in/ojs/index.php/dsj/article/download/12078/6319/

Max

 

[Fraser]

The question was how to compare manufacturers claimed NETD figures. My reply was in direct response to that request. Stated NETD in a specification sheet has been discredited to some extent by marketing teams who abuse it as a specification. If a user wishes to compare the real world performance of several cameras that they own rather than just believe potentially manipulated NETD figures, MRTD and MDTD are valid tests for the user.

I have absolutely no doubt that NETD, carried out with transparency and honesty, is a valid and accurate indicator of a cameras noise floor. Sadly experience has shown that some manufacturers marketing teams just see NETD as a way to confuse, and in some cases, mislead customers. A simple question must be asked of the manufacturer..... what lens and bias settings were used for the NETD test and is that same specification as used on the product sold to the customer ? I have also known cameras to be fine tuned to achieve best NETD when the production models are not fine tuned to the same degree. On the specification sheet some impressive NETD figures can be present, whether the same performance can be achieved by the user with their camera is a different matter.

I am not arguing against NETD, just highlighting to Ultraviolet that just because one camera claims superior NETD, does not necessarily mean it is better in the real world than a camera declaring a slightly higher (honest) NETD.

Anyway, I have said my piece. Whether NETD, MRTD or MDTD is used, what is important to the user in a comparison of different cameras is the quality of image produced in challenging conditions of low thermal contrast in a scene 

Fraser

[Max Planck]

Sorry, but the question was:

I have a number of cameras that claim various degrees of sensitivity, going down to a NETD of 30mK. How might I go about arranging some sort of test rig to check these claims? I have visions of terribly high accuracy, but realise there may be much simpler methods using, say, samples of different emissivities.

How to proceed, without spending a fortune I could better invest in more  thermotoys?

As I understand, it was about a user personally testing/comparing different cameras. Thus, whatever the manufacturers are doing or claiming is irrelevant here.
I also explained why your suggestion of using MRTD is not a good idea. This was just scratching the surface of the problems related to this parameter.

I one uses the same method of NETD measurement for all tested cameras and is aware of the camera settings that could affect the results, it remains a much easier solution to campare a single NETD value than comparing MRTD curves.

Max   

[Ultrapurple]

As OP, I think the fault is mine for (ab)using terms somewhat loosely.

I fully recognise that there are a number of ways of characterising a system's merit and that these have different, valid uses.

What I'm actually looking for is a way of comparing two or more thermal cameras' response to small temperature differences - what I'd call 'sensitivity' - around room temperature. Although I have a technical background my main interest here lies in the artistic rather than scientific direction. Being able to see two slightly different shades of onscreen pink is more important than knowing the actual temperatures they represent, or for that matter the absolute temperatures.

So, I apologise for my loose phrasing. But I am grateful for all the learned opinions and technical details that have been put forth and grateful also to all those whose ongoing help increases my understanding.

[Fraser]

The Aurora Scientific Inc 825A TEC controller arrived today. It is every bit as good as I had hoped. It appears to have either seen little use or has been in a clean environment and well cared for. I intend to drive one of the Supercool Peltier heads with it. The rear panel has a power output socket to drive the Peltier cooling fan or a fluid coolant pump so I have the required 12V for the Supercools's built in fan  I have the required AD590 temperature sensor and just need to decide upon a cavity design into which I will install the Peltier emission plate    I also have a couple of the high emission Black Body plates that came with the FLIR SC4000 so may use one of them with the Peltier head. I have 4 of the Peltier heads so could even build different Black Body cavities to suit different tasks. I am already in contact with Aurora Scientific Inc regarding possible modification of the units maximum settable temperature. It currently offers 0C to +40C. The separate T type Thermocouple measurement capability covers -50C to +50C

Ultrapurple, keep your eyes open for TEC controllers on eBay. They come up now and again at good prices. The 825A cost me £50 and is a lovely bit of kit intended for medical science lab use. It is well designed and built. See attached pictures.

Fraser

[bugi]

My one supercooler unit should be on the way now, too, comes whenever.

I was thinking either trying DIY control - pt100, misc, MCU with ADC, discrete FETs or if I happen to have some nice driver IC, etc. - or MAX1978 - expensive chip in quite a non-hobbyist package (0.5mm pitch TQFN exposed pad), but contains pretty much everything except the sensor and filtering passives. I do already have some semi-cheap chinese thing stuffed somewhere to try, but I don't even remember its ratings, might not even be suitable for driving TECs nicely.

Also, the "thermal camera (coming hopefully Soon(tm), though apparently not yet even shipped)" came yesterday. The seller just hadn't bothered to notify me about the shipping.

[Fraser]

Bugi,

I am pleased that you ordered a Supercooler. I do not think you will be disappointed 

With regard to driving the Peltier element, a relatively inexpensive controller may be built using cheap Arduino UNO plus an Arduino motor driver H bridge board. The Peltier element is rated at 12V 3A so not a huge current involved. A Google search on Arduino TEC or Peltier controller should bring up other experimenters experiences with such. You can drive the Peltier in simple on/off and polarity mode. This is what many Peltier refrigerator controllers do. Refrigerator controllers are normally limited in accuracy and offer only single polarity drive for cooling, so not much use to us here.

Fraser

[Fraser]

Bugi,

If you do not mind saying, what thermal camera did you buy ?

Fraser

[bugi]

TE-Q1 "large kit". Which, if I have understood it correctly, includes the same body used for both TE-Q1 and TE-Q1 plus, and both the respective lenses. And accessories and SDK. I wouldn't have needed the extras (I've got enough various similar accessories from other stuff already), but the SDK will get into use, sooner or later.

I did look for higher frame rate cameras, but either they seemed to be too much hazzle and/or price jumps a lot and/or seemed to be noisier and/or lower resolution. That TE-Q1 should be enough for most of my use cases; only two (of my current) ideas would benefit directly from higher framerates, and neither idea is anywhere near the top priority in my project list.

So far I've only quick-tested it last night, i.e. few minutes of looking random things with old galaxy S5 as host, just to check that it works properly. I'll probably do a bit more testing and reading through the manuals etc. this evening. And probably start wondering about "black bodies" and known temperatures much later, probably next summer, when I might have some time to dig into the SDK and hack up some DSP.   Having some time is likely wishful thinking, considering all the other projects queued up, but...

Edit: related to this particular topic, couple of them projects include keeping temperature gradients minimized on electronics (and with their inner containers), so sub-kelvin differences would be nice to see, one way or other.

[Fraser]

Good purchase choice 

Fraser

[olivir]

A comment on NEDT, there is a relationship between NEDT and dynamic range.  It is possible to tweak a camera for better NEDT by increasing responsivity of the detector.  This is done by increasing bias in uncooled detectors or integration time in cooled detectors.  The price to be paid here in practical terms is a reduction in the maximum temperature that can be imaged without saturation. When comparing NEDT specs it's good to know what temperature range can be imaged.  The same thing happens with F number.  If you take two identical cameras and install an F1.4 lens on one and an F1 lens on the other, the F1 camera will have a better NEDT (by a factor of 2 for uncooled cameras) but will saturate at a lower scene temperature.

[Ultrapurple]

A comment on NEDT, there is a relationship between NEDT and dynamic range.  It is possible to tweak a camera for better NEDT by increasing responsivity of the detector.  ...   The price to be paid here in practical terms is a reduction in the maximum temperature that can be imaged without saturation.

Very good points. I remember hearing some time ago that the Ulis sensor in the Therm-App had been tweaked (biased) for minimum NETD and it really is rather good in day to day use. But, as my recent comparison with the FLIR SC660 demonstrated, it falls right over when given a particularly hot scene, precisely what you're saying.

On the whole I like the greater sensitivity but, as I've noted many times, I play with thermal imaging for the pretty pictures rather than the diagnostic science.

[Bill W]

A comment on NEDT, there is a relationship between NEDT and dynamic range.  It is possible to tweak a camera for better NEDT by increasing responsivity of the detector.  This is done by increasing bias in uncooled detectors or integration time in cooled detectors.  The price to be paid here in practical terms is a reduction in the maximum temperature that can be imaged without saturation. When comparing NEDT specs it's good to know what temperature range can be imaged.

Agreed, although NETD should only be measured at the sensor, which is both the strength and weakness of it as a camera figure of merit

The same thing happens with F number.  If you take two identical cameras and install an F1.4 lens on one and an F1 lens on the other, the F1 camera will have a better NEDT (by a factor of 2 for uncooled cameras) but will saturate at a lower scene temperature.

Sorry, but no.  NETD should always be measured at f/1, even if the camera is not.
As many camera manufacturers are clueless they just paste the detector supplier NETD in and call it anything they like (sensitivity etc) even when it is not

That is where MDTD (zero frequency MRTD) is useful even though it often uses a human observer.  That then includes the real lens, all the processing, the display gamma, the display itself and finally the human eye-brain interface.

It is possible to measure a camera signal to noise with a raw image grab (Image J has all the tools) and you set the standard deviation of a flat image against the signal of a few degrees differential.
If you allow a 10 frame average for 'eye-brain' and a decent display you should be close to observed monochrome MDTD.
The statistical approach of ImageJ can be used to compare potential gain/integration/bias combinations easily.

What is actually surprising is ending up with a camera MDTD not far off the detector NETD.

Bill

[Max Planck]

As I wrote in another thread, as per the ASTM standard, NETD is measured at the detector preamplifier output, thus taking into account the detector and preamplifier noise, neglecting the lens etc.
The fact is, that today there is no universal NETD measurement standard that would be adapted to FPA based camera designs and the end user is interested in the performance of his entire camera, not just the detector or detectors in the FPA.
The f/1 is a good practice - if we are talking uncooled - but not a must. It makes no sense when talking about uncooled systems.

What is actually surprising is ending up with a camera MDTD not far off the detector NETD.

Bill

No surprise here. Please have a look at Annex A.

https://books.google.pl/books?id=-ZwYxus-Y5sC&pg=PA200&lpg=PA200&dq=mdtd+camera+measurement&source=bl&ots=8glpFkxVfQ&sig=ACfU3U0mjGpNjesGEssJqMey0j9820DeyQ&hl=pl&sa=X&ved=2ahUKEwj95Zjnwt7hAhUplosKHbhvDvYQ6AEwB3oECAwQAQ#v=onepage&q=mdtd%20camera%20measurement&f=false

As I wrote before, MRTD (and MDTD) are curves limited on one end by the camera noise and by system spatial resolution at the other.  The exact numerical values will of course depend on the model/measurement method, but if the IR system/application is noise limited and we are considering not the entire curve but its lowest value, the NETD/MRTD/MDTD fuss ends anyway as NETD vs NETD vs NETD (or close to).

Max

[Bill W]

What is actually surprising is ending up with a camera MDTD not far off the detector NETD.

Bill

No surprise here. Please have a look at Annex A.

https://books.google.pl/books?id=-ZwYxus-Y5sC&pg=PA200&lpg=PA200&dq=mdtd+camera+measurement&source=bl&ots=8glpFkxVfQ&sig=ACfU3U0mjGpNjesGEssJqMey0j9820DeyQ&hl=pl&sa=X&ved=2ahUKEwj95Zjnwt7hAhUplosKHbhvDvYQ6AEwB3oECAwQAQ#v=onepage&q=mdtd%20camera%20measurement&f=false

As I wrote before, MRTD (and MDTD) are curves limited on one end by the camera noise and by system spatial resolution at the other.  The exact numerical values will of course depend on the model/measurement method, but if the IR system/application is noise limited and we are considering not the entire curve but its lowest value, the NETD/MRTD/MDTD fuss ends anyway as NETD vs NETD vs NETD (or close to).

Max

An observational MDTD will vary by a factor of 2 or 3 between colour palettes or LCD panel, when run on the same sensor/lens/biases.  Your assumption of a system limited solely by the pre-amp NeTD is not the case.

Equally one could make a camera with 128 field averaging and easily surpass the sensor NeTD.  Useless for looking at moving things of course.  Just the same as digital averaging on a scope compared to a single shot trace.

Bill

[Max Planck]

And the point of all this is??

An observational MDTD will vary by a factor of 2 or 3 between colour palettes or LCD panel, when run on the same sensor/lens/biases. 

If measured for example according to STANAGs, yes, there is a long list of factors that could have an influence on the final numerical value, including the most important one - the observer, its training, etc...
If other methods are used, it is no longer the case. For example it can be calculated based on system MTF and noise data.

  Your assumption of a system limited solely by the pre-amp NeTD is not the case.

I made no such assumption. Just corrected some information provided in the thread.

Equally one could make a camera with 128 field averaging and easily surpass the sensor NeTD.  Useless for looking at moving things of course.  Just the same as digital averaging on a scope compared to a single shot trace.

Any manufacturer can addany type of signal/image processing, palette optimisation, etc...


The thread starter asked a simple question:

I have a number of cameras that claim various degrees of sensitivity, going down to a NETD of 30mK. How might I go about arranging some sort of test rig to check these claims?

The simplest answer is how to measure NETD. It is just a figure of merit like several others. Its advantage is the fact than there is no pallete, display, observer influence. Pure numerical calculations. Thus, it is easy to repeat/verify the measurements as long as the same method is used. NETD is more axed on temperature measurements, whereas MRTD curve is rather dedicated for IR observation systems.
It does not mean temperature measurement with +/- NETD accuracy, but this is another story.

Max

[Bill W]

The thread starter asked a simple question:

I have a number of cameras that claim various degrees of sensitivity, going down to a NETD of 30mK. How might I go about arranging some sort of test rig to check these claims?

The simplest answer is how to measure NETD. It is just a figure of merit like several others. Its advantage is the fact than there is no pallete, display, observer influence. Pure numerical calculations. Thus, it is easy to repeat/verify the measurements as long as the same method is used. NETD is more axed on temperature measurements, whereas MRTD curve is rather dedicated for IR observation systems.
It does not mean temperature measurement with +/- NETD accuracy, but this is another story.

Max

The thread starters' premise of using a few mK temperature differential to measure NeTD is not how it is done, so in some ways the question is unanswerable. 
The implication that setting up a 30mK difference might mean anything to observations of a "30mK" sensor is the problem.

You can only compare what you have in a consistent manner, which to an end user will mean near or at the output (video, data stream, screen, etc), which pushes you towards MDTD (ie MRTD (0) )

A typical commercial NeTD experiment uses a 19" rack of equipment (16 bit cooled ADC, special sensor PCB, shielded everything, smoothed everything) to measure the sensor IC output noise, and also measuring the signal from a 29.5°C and a 30.5°C target, for all pixels.

regards
Bill

[cnxunuo]

Consider finding one of these
I got one from local electronics junk yard.

[Fraser]

Nice Black Body  . I am envious of you 

Fraser

[CatalinaWOW]

I completely agree with Bill W's comment and will add this - all of the discussions above of the "correct" way to measure any of the many figures of merit of a system have an implicit assumption of how the system is applied.  The question cannot be answered without a good understanding of that.

The OP says that he merely wants good pictures of objects with a relatively small temperature range.  For this he definitely is concerned about monotonicity of the system response, and is somewhat interested in linearity and has an interest in the system noise level.  But might be satisfied with reducing the noise by integrating dozens or even hundreds of images.  Which can be done without loss of dynamic range if the integration is done in an appropriate signal processor after the signal is digitized.  But this integration will only help for truly random noise, so his figure of merit for his application will also need to pay attention to fixed pattern noise stability.

The upshot of all of this is that the equipment required is not necessarily out of reach.  A couple of black bodies carefully characterized for uniformity and temperature and a good computer.  The truly difficult part is thinking through what matters and how to measure it to accomplish the particular purposes at hand.  The answers will be very different for an art camera generating nice pictures at a few frames per day rate than it is for a surveillance system or a military IRST.

[Max Planck]

I completely agree with Bill W's comment and will add this - all of the discussions above of the "correct" way to measure any of the many figures of merit of a system have an implicit assumption of how the system is applied.  The question cannot be answered without a good understanding of that.

Excellent idea. Let's apply it pratically. So, what was Ultrapurple asking about? What was his camera scenario? IRST? No. A noise limited one?   


Let's now forget about it and have a look at IRST.

The upshot of all of this is that the equipment required is not necessarily out of reach.  A couple of black bodies carefully characterized for uniformity and temperature and a good computer.  The truly difficult part is thinking through what matters and how to measure it to accomplish the particular purposes at hand.  The answers will be very different for an art camera generating nice pictures at a few frames per day rate than it is for a surveillance system or a military IRST.

This time we are talking about a spatial resolution limited scenario, with the IR imager to target distance from a certain range of values. Single MRTD value - this is what was advocated all over this thread - would be again a, let's say wrong choice. MRTD curve is what is needed.

By the way, I gave already examples of such curves, but given some bells and whistles hidden by some of the manufacturers inside the equipment, there could be sometimes real surprises.

http://photonics.pl/PLP/index.php/letters/article/view/4-54/253


Regarding equipment, a liquid based blackbody model would easily do the job. Something cheap and easy to make at home.

Max
   

[Bill W]

Here's an MRTD target set:

http://www.sbir.com/images/Target/4BarNoTitle.jpg



You put these in front of your black body (the target stays at ambient or thereabouts) and run the black body up and down in temperature until you can just (make out) and (not make out) the bars - so four temperatures, and then for each bar set.
Then average the four temperature differences.
By using the black body both hotter and colder than ambient the actual temperature of the target ambient is cancelled out.

MDTD is done with a 'big' square or round hole - ie zero frequency MRTD.

Bill

[cnxunuo]

diy a CNC usaf1951?

[CatalinaWOW]

When I was unable to find anything other than nth generation copies of the 1951 USAF test target I created the attached document that at least has crisp, unblurred edges on the smallest bar targets.  Not traceable to anything, but good enough for local use.  The warnings on page 1 are unchanged from when this was created more than a decade ago.  Feel free to use it to CNC or send it off to be etched into a brass or aluminum sheet.

[cnxunuo]

I want to try using laser stainless stensil service from pcb manufacturers and spray a black body paint over it, but unfortunately I am no longer in china...need to ship/carry my black body to where I live now.

fitting an array of pt100 to brass plate and stick some thin ceramic heating elements to it could also work, I did a teardown of Fluke black body some years ago, it's very 'not metrology looking', hence the relatively low price from fluke when new. if pt100 says it's even, it must be enough for IR use, provided that black body paint is sprayed properly.