Thermal Camera SDK, Image Formats

[j0ker1980]

Hi,

I'm considering buying an USB-C Thermal Imaging Camera with at minimum 384x288 Pixels. I also want to be able to write my own Software so a SDK is mandatory.
I know that there is an SDK available for the Thermal Expert Devices. From what I have read in this forum infiray also has it.

Requirements
- SDK
- Temp. Range up to 250°C
- Framerate of 25Hz+ would be nice
-

Those models are currently on my shortlist.

HTI HT-301
High Framerate, Large Temperature Range, SDK???

Thermal Expert TE-Q1Pro
Low Framerate, Medium Temperature Range, SDK, changeable Lens

infiray XTherm T3pro
High Framerate, Large Temperature Range, SDK, changeable Lens

Do the Cameras save the radiometric Data in their Images for later analysis? This would also be nice to have.

From the specs I would go with the T3pro or the HT-301. But I don't know if HTI has an SDK.
The problem with the infiray Cameras is that I have not found a place to buy them in Europe/Germany/Austria. Does anybody know where to buy a T3pro in Germany/Austria?

Thanks.

[Fraser]

Be warned, some SDK’s can be very restricted in what they offer the developer. The API dictates the available options and some are not well written or comprehensive in what they offer you. Do check out the SDK before buying to ensure that it meets your needs. Manufacturers like Seek Thermal make it hard to obtain the SDK for anyone except manufacturers and they require a NDA before releasing it.

[zrq]

For the Xtherm T3s (Pro) and HTI, it just uses the same protocol as USB Webcams. You can use anything works with an UVC camera, from OpenCV to >10 projects on GitHub to obtain the image frames (with some metadata included as additional rows). Commands for the camera for mode change or shutter calibration is sent though the Zoom control channel. The only useful part of the SDK is actually a binary (which means C source code with IDA or GHIDRA) for the conversion from the frames with metadata to the temperature mapping of the frame or YUV/RGB video data.
This camera can not produce radiometric images compatible with existing analysis software, but I think as a developer, we can get enough information from the camera to do any post processing.
Xtherm T3s (without Pro) is only calibrated up to 120 degree Celsius. It does have data output for the wide temperature range up to 400 degree Celsius, but the sensitivity is poor and the output number range is strange. I need futher investigation to find out whether it's useful for wider temperature range.

[Fraser]

Something to be aware of when building your own radiometric analysis software. Some cameras apply histogram equalisation to the image to improve contrast and enhance the image. Such ‘magic’ is often not detailed by the manufacturer and can totally destroy any radiometric data that was present before the image enhancement. I am working with a camera core that does exactly this and the SDK warns that the RGB image cannot be used fir radiometric analysis as many pixel levels have been altered by the histogram equalisation process. A separate radiometric data stream is provided by ghat core fir radiometric analysis.

From looking at several of the phone dongle type cameras, it would appear that there is a lot of image processing and ‘tweaking’ of the image to decrease noise levels and make the images look pretty. If an SDK does not offer access to RAW unmolested pixel data, do not expect to use the camera with radiometric analysis software even if you extract the pixel values from the image.

Fraser

[Fraser]

Regarding exceeding a cameras stated temperature range. Some cores are deliberately hobbled in the cameras firmware to limit their temperature coverage. Others are just single range cameras that commonly top out at around +120C. Why ? you may ask. Well when configuring a thermal camera microbolometer you need to consider  the needs of the intended application. Basically if you want best sensitivity to Delta T you limit the core to a range of around -10C to +120C and this gives you a core that produces excellent imagery at the temperatures commonly found around the home. If higher temperature capability is required, additional microbolometer configurations will be required. With different settings applied to the microbolometer it is able to image temperatures as high as +700C without the need of an aperture reduction in the optical path. Above +700C it is wise to consider reducing the energy that is reaching the microbolomter.

I will not go into how the microbolometer is actually configured for temperature coverage but some googling on the FLIR high and low sensitivity modes in the TAU core may help understand the effects of the configuration. Basically, if you want greater temperature coverage, you have to accept a lower thermal sensitivity. Nothing in life is for free  .

A microbolometer that is configured for maximum sensitivity may still operate above the stated maximum temperature but the calibration table likely becomes invalid. This is why some cores state a maximum radiometric measurement temperature but also state imaging is available at higher temperatures, but for ‘indication only’. A more capable core configuration will automatically switch to a lower sensitivity configuration and provide radiometric measurements at the higher temperatures. There may be 1, 2, 3 or more temperature ranges in Industrial thermal cameras. Some auto range (fire fighting cameras) whilst others let the user manually select the range. If a camera has an additional temperature range that is not calibrated at the factory, it may work, but be using default generic calibration tables. If no calibration table exists, the camera will either indicate no calibration (no picture generated) or provide pixel ‘Count’ data rather than temperature.

Fraser

[zrq]

For Xtherm T3s and very likely for HT-301, it looks like we can get the raw sensor readout before any processing after sending a command. This means we have the maximum freedom to do any math to get the image, but we also have to implement everything by ourself including shutter calibration and dead pixel removal.

The output stream can be switched to other modes by commands, YUV data after applying color plates and 16bits numbers with clear relation to radiometric values (can be converted to temperature mapping) are available.

[bfree]

I recently purchased an Xtherm T3Pro.  I've confirmed that it supports UVC, and have connected to it on Windows and Android.

No luck on Mac yet, but believe it's a cabling issue.  Haven't tried Linux yet.

While I've developed software that talks to UVC cameras, it'd be great to get the SDK that Xtherm claims on their website they have for this camera.  However, there's no links, and they have not responded to my queries.

zrq - if you have suggestions on how to get their SDK, documentation, or have any sample code, that would be greatly appreciated!

[bfree]

Update: I have the Xtherm T3Pro working via UVC on Windows, Linux, MacOS, iOS, and Android.

I have a UVC streaming/filtering app I wrote last year on MacOS; after trying several cable configurations, I was able to get it to recognize the Xtherm, and is streaming/broadcasting filtered thermal images.

An SDK would have been helpful, but now moot.  Surprising that the company hasn't even responded.

[JimM]

Update: I have the Xtherm T3Pro working via UVC on Windows, Linux, MacOS, iOS, and Android.

I would be interested in your software for the Mac. Have you posted it anywhere -- here or GitHub?
Thanks,
Jim

[bfree]

New update:

Just received a response from InfirayOutdoor - they've sent me SDKs for Windows, Android, and Linux.

Documentation is in Chinese, but you can probably get through the code without that.

For other platforms, you'll need a UVC driver/library to write your own software.  On MacOS/iOS, use AVFoundation AVCaptureSession to access UVC devices (it's not very well documented).  On Windows use DirectShow or the Media Foundation interface.  For Linux use libuvc.  There are a number of UVC libraries for Android.

[rajeshmag]

Hi

I bought an HT-301 and received it just a few days ago. Can you please share the software to run it on Windows 10?

Email: rajeshmag1@gmail.com
UploadLink: https://www.dropbox.com/request/A3o8k9QCG5fj1CBSFU82

Thank you,
Rajesh

[jake5566]

Hi bfree,

Could possible share the SDK? It would be great if there is a chance to run it on linux.

Thanks.
--Jake--

[svgurus]

Something to be aware of when building your own radiometric analysis software. Some cameras apply histogram equalisation to the image to improve contrast and enhance the image. Such ‘magic’ is often not detailed by the manufacturer and can totally destroy any radiometric data that was present before the image enhancement. I am working with a camera core that does exactly this and the SDK warns that the RGB image cannot be used fir radiometric analysis as many pixel levels have been altered by the histogram equalisation process. A separate radiometric data stream is provided by ghat core fir radiometric analysis.

From looking at several of the phone dongle type cameras, it would appear that there is a lot of image processing and ‘tweaking’ of the image to decrease noise levels and make the images look pretty. If an SDK does not offer access to RAW unmolested pixel data, do not expect to use the camera with radiometric analysis software even if you extract the pixel values from the image.

Fraser

Came across quite good samples to illustrate Fraser's point. Attached 3 photos of fellow girls palm made by HTI and thermviewer with superresolution, check out visible light-blue halo around lower finger on one photo and less visible dark-blue halo aroun all palm on all 3 photos) It is uncompressed but superresed PNG photo, jpeg still from video would be even worse. I've read Flir tau datasheets, they also tell about radiometric accuracy and AGC magic) speaking of TAU, you can buy radiometric tau 2 640 on ebay from Germany now, ships worlwide, lot № 265197481344. it has lens already but you will need VPC board and tripod mount. Maybe joe-c from this forum can help you with vpc or i will ask some Russian fellas)) I almost bought this tau from ebay, wasn't sure it is fine with ITAR - can anyone tell me is it 100% fine with law to send flir tau 9hz in Russia? proofs with links are the best)
Also you can get tau1 320 core from flir ps-24, it can be found dirt-cheap and has basic radiometry with isotherms. and i have it too lol. Tau user manuals describe serial port commands, my file for 320 is too big for forum engine, search"Tau_320_Users_GuideRev120" or write me an email. More serious flir tau sdk is something sold for money, maybe even serious amount closer to 1000$

[Vaklin]

Hi!

I bought one of these cameras (T2-Search) and I'm very satisfied from the quality. In my plans is to use it on my drones. So I did some research and now I have image transmission to the base station and so on. The missing item is the temperature scale data, due to dynamic wide of colors related to the temperature. As I can see, the metadata is attached in the last few rows in the picture. The question is how to decode it. The really big help for me will be some manual or SDK or something like this. I hope you'll not reject my question.

Regards!

[vmpn]

Is there an open and in actual use standard for thermal imaging images and movies common across tools?

Everything I was able to find so far is targeting a single tool by its maker.

Thanks

[thermalengineer]

Hi, many thanks for this topic ! By any chance do you know where can I find a tutorial on how to use the infiray xtherm app ? Because there are several options I do not understand.

[ahsan]

hello, i purchased infiray T2L camera but the problem is that it only thermal image in YUY422 encoded format with 4 additional row include in the image. HOw to analyze this type of image from thermal camera to extract temperature values of each pixel. Any help regarding to this is highly appreciated.
Thanks

[andrew_salinas_]

I find it very hard to believe you managed to get it working on IOS. iPadOS 16 is the first to bring USB driver support. These cameras are not MFI compliment and would be blocked by apple. You cannot use AVFoundation as you said for IOS. There are workarounds of course but they are quite complicated.

[Krist-win]

I get SDK from seller.

[rebotea]

Hi any SDK for Infiray Windows?
Thanks