EEVblog Electronics Community Forum

Products => Thermal Imaging => Topic started by: gtatters on September 03, 2021, 12:34:12 pm

Title: Radiometric JPG & ATS file formats
Post by: gtatters on September 03, 2021, 12:34:12 pm
Hi all

I have a new FLIR A8581 (MWIR) with Flir Research Studio.  Nice camera, 1280x1024 resolution.  But the software claims to save radiometric jpg and .ats (thermal video) files, that do not seem to resemble radiometric jpg from LWIR cameras nor .seq or .csq file format.  I thought I'd share here.

The jpg yields no thermal data or planck constants using exiftool. 

The jpg file appears to store the raw data (in this case ~8032 bytes offset; see tiff extracted file), but the tiff data are contaminated with 40 intermittent 12 byte long garbage data (it would be nice to find a way to remove).  I have no idea what format the .ats.

3 files in zip file: Snap-0012.jpg (original, radiometric jpg), Snap-0012.tiff (extracted raw data, 16 bit little endian, converted in ImageJ), Rec-0018.ats (brief thermal video)

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=0 (https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=0)

Sharing on a public dropbox folder above.

Cheers
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 03, 2021, 04:35:31 pm
Having trouble getting the files from DropBox. Can you upload just the jpg file to EEVblog directly?
From your description it sounds like a Flir JPEG-R file. But, I would have to look at it to be sure.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 03, 2021, 05:29:46 pm
Thanks for trying.  I don't know what the issue is, since I tested the link with a colleague but they have a dropbox account.

Let's change the dl=0 to dl=1 to direct download to see if that works since this is supposed to be a public link:

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1
 (https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1)

Otherwise, I can't share the .ats file since it is 44Mb, and this forum will only let me post one file.  I'm curious to solve both the JPG-R issue and the .ats issue.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 03, 2021, 06:43:50 pm
Ok, got the files from DropBox. The jpg in the DropBox ( 2.7 Mbytes) seems to be a JPEG-R file. It has the radiometric data split into APP1 segments with the 12-byte headers separating the segments as usual. However, the metadata looks totally different from other FLIR JPEG-R files. You're right, ExifTool doesn't show anything for the FLIR metadata. My own code crashed when I fed it this file, haha. So I guess some reverse Engineering is needed to figure out these files.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 03, 2021, 07:12:37 pm
Thanks JimM.  Yes indeed.  Tomas123 and Phil have been very helpful with other reverse engineering thus why I shared here.  Feel free to post anything you discover.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 04, 2021, 03:20:22 am
There is some xml near the end of the jpg file that has interesting metadata: min and max temperatures, emissivity, etc. Namely:

Code: [Select]
<workspaceFileSettings>
	<segmentation enabled="False" scaledMin="-0.1252924328758" scaledMax="0.3884484193701">
		<segmentationValues>
			<segmentationValue unit="Counts" min="4107.3955078125" max="6884.330078125" />
			<segmentationValue unit="Radiance" min="0.00013401229807641" max="0.00024988286895677" />
			<segmentationValue unit="TemperatureC" min="18.424726486206" max="36.694522857666" />
			<segmentationValue unit="TemperatureK" min="291.57473754883" max="309.84451293945" />
			<segmentationValue unit="TemperatureF" min="65.164505004883" max="98.050140380859" />
			<segmentationValue unit="TemperatureR" min="524.83453369141" max="557.72015380859" />
		</segmentationValues>
	</segmentation>
	<playBounds start="0" end="0" />
	<filterChain enabled="True">
		<filters />
	</filterChain>
	<imageView zoom="0" panX="0" panY="0" hFlip="False" vFlip="False" angle="0" />
	<palette name="" />
	<roiList version="1.6" imageWidth="1280" imageHeight="1024" activeRoiIndex="0">
		<imageRoi color="#808080" paletteIndex="0" />
		<rois />
	</roiList>
	<source unit="Temperature (Factory)" temperatureType="Celsius" applyNUC="True" applyBP="True" showSaturationColors="True" />
	<levelAndSpan mode="FromImage" scaledMin="0.194501" scaledMax="0.616296">
		<levelAndSpanValues>
			<levelAndSpanValue unit="TemperatureC" min="22.8804" max="58.4427" />
			<levelAndSpanValue unit="TemperatureK" min="296.03" max="331.593" />
			<levelAndSpanValue unit="TemperatureF" min="73.1847" max="137.197" />
			<levelAndSpanValue unit="TemperatureR" min="532.855" max="596.867" />
		</levelAndSpanValues>
	</levelAndSpan>
	<scaleOptions mode="Linear" plateauP="2" ddeSharpness="5" linearIgnore="0" useLog="False" lockScale="False" />
	<objectParameters override="False" emissivity="0.92" distance="1" reflectedTemp="293.15" atmosphereTemp="293.15" extOpticsTemp="293.15" extOpticsTransmission="1" estAtmosphericTransmission="0" relativeHumidity="0.3" />
</workspaceFileSettings>

There is similar xml in the ats file, although some of the values are different.

A start towards reverse Engineering might be to see what temperatures FLIR software displays versus the values in the jpg file.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 04, 2021, 12:42:45 pm
I'm not familiar with xml or how you extract that (happy to learn), but I can confirm that the min=22.8804 and max=58.4427 correspond to the min and max temperature on the scale when you open the image up in flir research studio.

See screenshot.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 04, 2021, 06:45:50 pm
Regarding the xml, the most general way to examine files is with a Hex Editor. On the Mac, I use Hex Fiend --probably my favorite application. Below is a screen shot of the Snap-0012.jpg file in Hex Fiend. Note the hex data is shown on the left. Most hex editors try to decode the hex data as ASCII text which is  on the right. So ASCII text stands out. You can then copy the ASCII text out of Hex Fiend and paste it into a text editor such as TextEdit on the Mac. TextEdit seems to automajically format the text in hierarchy form of xml. Course there's hex editors for Linux or PC if you are on those platforms.

If you get a Hex Editor, one thing to be aware of -- you can open and edit any file on your computer. There are certainly files that you would never want to (accidentally) edit. One very good thing about Hex Fiend is that it does not automatically update a file when you edit it. You have to explicitly save the file. I hate software that automatically updates the file when you edit it.

Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 04, 2021, 07:45:36 pm
Ah yes, thanks for that.  I use Hex Fiend as well.  Great program and I was able to see all these xlm tags, just wasn't sure how to automatically pull them out.  I was impressed at your coding in the earlier message and was curious if there was a command line tool for extracting xml.  I don't know the xml framework at all.  I really liked exiftool's capacity to cleanly pull out the meta-data inside these files! 
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 05, 2021, 02:14:05 am
Quote
I use Hex Fiend as well.
Oh, ok. Sorry for the long winded hex editor explanation.
There's probably other ways of finding/extracting xml too.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 05, 2021, 06:12:38 pm
In terms of the 12 byte repeats that are strewn throughout the TIFF data, I think they are ****464C4952**** where 464C4952 corresponds to the Ascii "FLIR", surrounded on either side by 4 bytes of sometimes different values. 

Actually, I just noticed it's possible that the complete byte sequence is FF E1 FF FD 46 4C 49 52 xx xx xx xx, where xx is more variable.

Anyone with good perl or regex coding skills might be able to help me work out how to search and remove these contaminating elements?



Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 05, 2021, 06:39:03 pm
The FF E1 is a JPEG APP1 segment marker. That's followed (usually) by FF FE (65534 decimal). That's the length of  the APP1 segment. The length includes the length parameter itself but not the marker. So you can find the end of the APP1 segment. The last APP1 segment has a length of 29 4B (10571 decimal). I don't know what the 4-bytes following the 'FLIR' characters are, but they seem to increment by 256 decimal in each APP1 segment.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 05, 2021, 07:20:31 pm
Ah, I didn't realise that.  Ok, that helps, somewhat, but FF FE = 65279 doesn't it, not 65534, which doesn't correspond to the true length of the APP1 segment from what I can tell.

I just want to delete 12 bytes whenever there is a FF E1 FF FD, since without these, the TIFF image should be intact, and represent the raw thermal data (or perhaps simply the temperature data).
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 05, 2021, 07:33:27 pm
No, JPEGS are big endian, so FF FE is 65534 decimal. All of the segments are 65534 bytes except the last one. Note the total length of a 65534 byte segment is 65536 bytes including the marker. So when you excise the 12-bytes you have 65524 bytes of data.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 05, 2021, 08:37:41 pm
Sorry about that.  Yes I see what you mean.  I'll see what I can figure out for splicing the 12-bytes out.  Not sure I know how to do that yet.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 05, 2021, 10:08:12 pm
I'm working on some C code to extract the radiometric data also. One problem is, where does the radiometric data start? As you said in an earlier post, it looks like it's around 8032 offset. But lacking other information we don't know for sure. I was looking around for something that had that offset, but no luck. Course as long as we have a total 1024x1280x2 bytes for the image, we will get a reasonable looking image.   
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 06, 2021, 03:58:35 am
Ok, so attached below is a pgm file of the radiometric data. As usual, the radiometric image has very low contrast (basically appears as black) in most image viewers. I passed the pgm image through ImageMagick convert -contrast-stretch with factors of 1%x0.5% to produce the PNG image also attached. Course, the pixel values in the PNG no longer have any relationship to temperatures. It's just a picture.

One odd thing about the radiometric data in the jpg: it appears that the last byte of the data is missing (the least significant byte of the last pixel).

Edit: I had an "off by one byte" error. The radiometric data is little endian, not big endian.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 06, 2021, 04:04:46 am
I forgot, the forum won't accept a pgm file. I put it in a zip file attached below.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 06, 2021, 12:22:36 pm
If it helps, I have added a few more jpgs (taken from the same program/camera), in case the variation in offset helps.  The TIFF style raw data does not start at the same offset in each image. 

I have put the images in the zip file on dropbox.

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1 (https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1)
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 06, 2021, 12:25:01 pm
Can you help me follow how you did this?  Maybe without the imagemagick step - I work in ImageJ and it's very easy to re-stretch the greyscale but keep the data intact.  The png file looks like it has lost some quality in the conversion.  Even just having the raw data - I can import raw image into ImageJ easily - no file header required.
Title: Re: Radiometric JPG & ATS file formats
Post by: Miek on September 06, 2021, 12:52:36 pm
Quote from: JimM on September 05, 2021, 06:39:03 pm
I don't know what the 4-bytes following the 'FLIR' characters are, but they seem to increment by 256 decimal in each APP1 segment.

The latter two bytes of those are 8bit uints with the current chunk number and the total chunks.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 06, 2021, 01:11:14 pm
Aha, clever.  thanks for sleuthing that.  I could see they incremented and that there were 40 chunks, but didn't figure out it was 8 bit. 
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 06, 2021, 03:14:20 pm
Ok, if it helps, attached below is the C-code to extract the radiometric data from the jpg file. There's basically just 4-steps;

The code is hardwired to the Snap-0012.jpg file. There are magic numbers all over the place.
I'm going to be out of pocket for a couple of weeks, so won't be posting here for awhile. If you guys could do some more sleuthing to find out how to get to the start of the radiometric data, that would be useful.

Edit: As mentioned in Reply #16. I had an off by one byte error in the above. The radiometric data starts at offset 8031 or 7999 after the assembly step.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 06, 2021, 05:43:39 pm
This is really helpful!  And much appreciated.  My own progress will be slow, with start of term this week and chaos that comes with that.  I might try to work something out in perl as I don't know c++.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on September 15, 2021, 06:11:23 pm
Hi all,

A minor discovery (at least for finding the pixel starts) is that in 5 different jpgs (technically these are FLIR ATS-US files I guess), the start of the pixel data appears to be 718 bytes offset to the Pixels byte tag (0x50 0x69 0x78 0x65 0x6c 0x73).  See screen shot. 
In this file, then pixel start is 8034 bytes offset from start of file.

I'm still working on other solutions to this.  Then I need to sit down to work out what the pixel data are stored as.  Radiance and thus need converting to temperature (I don't have any calibration constants) or if they are representations mapped using the min/max data in the Workspace meta data.

Cheers
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 22, 2021, 02:52:31 pm
Quote from: gtatters on September 15, 2021, 06:11:23 pm
Hi all,

A minor discovery (at least for finding the pixel starts) is that in 5 different jpgs (technically these are FLIR ATS-US files I guess), the start of the pixel data appears to be 718 bytes offset to the Pixels byte tag (0x50 0x69 0x78 0x65 0x6c 0x73).  See screen shot. 
In this file, then pixel start is 8034 bytes offset from start of file.



You don't mention the file name for the data you posted, but this appears to be the image Snap-0010.jpg. If so, the "Pixels" string starts at 7317, so applying the 718 byte offset puts the start of the radiometric data at 8035 rather than 8034. The first pixels are then: 1CEE 1CF7 1CFE ... This appears to be Big Endian byte order so in decimal we have: 7406 7415 7422 ...

I haven't discovered anything more about these files. However I notice from the XML data that the palette name is always a null (i.e. no name). I'm guessing that this means just a default greyscale palette is being used. Can you select a palette and then we can see where the palette is in the file and what form it has? Also there doesn't appear to be any rois (region of interest ?). Possibly you can select a roi, and we can see how that shows up in the file. Course, this would all be interesting information but it doesn't tell us how to calculate temperatures from the radiometric data. We would need more info from FLIR for that.

Edit: Just realized something -- if the offset to the start of radiometric data is 717 not 718, then the data indeed starts at 8034 and is Little Endian, and the first pixels are: FF1C EE1C F71C ... giving decimal values of 7423 7406 7415.
Title: Re: Radiometric JPG & ATS file formats
Post by: Miek on September 23, 2021, 02:49:34 pm
Just a minor thing, but I would recommend doing this in layers rather than trying to go straight to the pixel data from the jpg. First extract the ATS file from the jpg, then work with the ATS file alone to figure out its format. It'll be much easier to spot any length/offset fields in the ATS that way.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 23, 2021, 05:28:38 pm
Quote from: Miek on September 23, 2021, 02:49:34 pm
Just a minor thing, but I would recommend doing this in layers rather than trying to go straight to the pixel data from the jpg. First extract the ATS file from the jpg, then work with the ATS file alone to figure out its format. It'll be much easier to spot any length/offset fields in the ATS that way.

Agree, in general you need to use a top-down approach to figure out where things are in these files. We want to determine much more than just where the pixel data is. However, as far as I know, these are just jpeg files with FLIR APP1 segments containing the radiometric data and meta data. So far I haven't found anything like a directory with offsets and lengths would help decode the files.  I don't know what an ATS file is and haven't found much information on-line about what an ATS file is. If you have any links to what's in an ATS file, please provide.
Title: Re: Radiometric JPG & ATS file formats
Post by: Miek on September 23, 2021, 07:41:19 pm
If you extract the chunks of data from all the APP1 segments, combine them together, and save them out as a new file then you get a file starting with "FLIR ATS-US File" and it seems to match the format of the 'Rec-0018.ats' file included in the first post. I've attached 'Snap-0012.ats' as an example, and some python code that I used to extract it.

There's a few mentions of it dotted around online. The ResearchIR user guide mentions support for the format: "Supports SEQ, SAF, PTW, ATS, FCF and RJPG file formats".
This page has a Windows tool for working with the file: https://flir.custhelp.com/app/answers/detail/a_id/3043/~/post-calibration-of-recorded-data, I haven't tried it though.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on September 29, 2021, 06:48:25 pm
Just to close the loop on the radiometric images. I corrected the "off by one" error I had in the previous data and changed the output to tiff type files. Attached are two radiometric tiff files in little endian byte order extracted from the Snap-0012.jpg and Snap-0007.jpg files. These tiff files should be compatible with ImageJ application.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 01, 2021, 05:24:27 pm
Sorry that I missed all these responses (from september) - term busier than usual, and I will try to get caught up.

To recap: you would like me to capture images with different palettes (from null) and with ROIs turned on, so we can see how that influences the .ats file structure and possible the RJPG being saved.

Yes, the tiff file you created JimM is beautiful and free of the repeating noise.  Can't say I follow what you have done given my coding skills, but I'll fumble through.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 01, 2021, 06:18:10 pm
Quote from: gtatters on October 01, 2021, 05:24:27 pm
To recap: you would like me to capture images with different palettes (from null) and with ROIs turned on, so we can see how that influences the .ats file structure and possible the RJPG being saved.

Yes, and just one example RJPEG file would be enough.
Thanks.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 05:02:33 pm
Sorry for the delays.  I've attached 2 versions of the same ATS RJPG (for lack of a better name), with different palettes.  Next post will be with ROIs.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 05:03:23 pm
This contains two files but with a box roi as well.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 05:30:42 pm
This might help.  Describes a command line utility that can pull calibration data out of an ats file.

https://flir.custhelp.com/app/answers/detail/a_id/3043/~/post-calibration-of-recorded-data/session/L2F2LzEvdGltZS8xNjMzMTk1NjE0L2dlbi8xNjMzMTk1NjE0L3NpZC9mVTB4T2Nnd0t3SDVkbjY3OUpxJTdFOVdxR0ZpYU9UVUglN0VySkxobkNaN3VHa2JWR3dTbFlTendqazNxNHdTVE5URlJNOENhMmNscGNCWWRFN05NbTlVZm14ZHFLVWNzYSU3RTFrWjBNOHlVS3hQbHc5VVl3bHdqUExPNGclMjElMjE=
 (https://flir.custhelp.com/app/answers/detail/a_id/3043/~/post-calibration-of-recorded-data/session/L2F2LzEvdGltZS8xNjMzMTk1NjE0L2dlbi8xNjMzMTk1NjE0L3NpZC9mVTB4T2Nnd0t3SDVkbjY3OUpxJTdFOVdxR0ZpYU9UVUglN0VySkxobkNaN3VHa2JWR3dTbFlTendqazNxNHdTVE5URlJNOENhMmNscGNCWWRFN05NbTlVZm14ZHFLVWNzYSU3RTFrWjBNOHlVS3hQbHc5VVl3bHdqUExPNGclMjElMjE=)

This link even vaguely suggests what the calibration coefficients are - although not sure if this is the same equation as used in other flir jpgs (with Exiftool extracted Planck constants):

https://flir.custhelp.com/ci/fattach/get/91501/0/filename/How+to+apply+a+RIR4+user+cal+to+existing+data.pdf (https://flir.custhelp.com/ci/fattach/get/91501/0/filename/How+to+apply+a+RIR4+user+cal+to+existing+data.pdf)
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 02, 2021, 06:48:30 pm
Ok, thanks for the files. Did a quick perusal of the files, and it looks like there is no longer the "Workspace" block of data or the associated xml in these files. So I guess that data is only in the files when they first come from the camera, but not after a file has been modified. Probably makes sense. I'll have to look at the files more closely to locate the palette and roi information.

Regarding the ats naming -- I think the .ats file is just another Flir proprietary file format. In other words it has nothing to do with other .ats file information that might be found elsewhere on the internet. This was initially confusing (at least to me), but it's not the first time that Flir has adopted a file extension that is used elsewhere. So I guess ATS is Advanced Thermal Systems? Do you get an .ats file only for videos, and you get a .jpg (i.e. an RJPG) for pictures?

Regarding extraction of temperature data -- the Flir A8581 is a MWIR camera using an InSb Sterling cooled detector (actually I was only able to find the A8580 on-line, assume the A8581 is similar). However, the temperature extraction work that Phil Harvey and Tomas123 did 6 or 7 years ago was for Flir LWIR cameras using uncooled bolometers. I would not expect temperature extraction to be the same. But I am no expert on this. Maybe someone more knowledgeable on this could jump in -- Fraser ?

Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 07:21:39 pm
Yes what you say makes sense.  Using ResearchIR (or the new Research Studio), i can capture a video (.ats) or a single radiometric file (.jpg).  And of course, I can open a file, and then export a new radiometric jpg with different conditions, but invariably, it seems this isn't the same as a live capture, as you say.

Re: ats, I am 99% certain that .ats file extension is intended for video data (high speed or otherwise) with their advanced software.  A single snapshot with ResearchIR is auto saved as a jpg (no choice that I can see).  If you ever wanted to see it in action, feel free to drop a message and I can set up a zoom.

Re: A8581 vs A8580.  Likely a minor difference, perhaps related to year of manufacture or some obscure feature that differs.  Generally, not a big diff in Flir cams between model numbers differing in the single digits.

Working out the calibration to temperature is really what I'd like to figure out.  While you are correct that we can't assume it is identical to the LWIR equations, in principle I would expect it to be similar, but can't say for cert.  The fact that we can see 4 "calibration" tags in the jpg suggests 4 cal constants.  The 5 planck constants from the .seq, .csq, old .jpg files that Exiftool extracts are technically 4 coefficients in an equation, since R = R1/R2. 

https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/2075/ (https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/2075/)

Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 02, 2021, 08:55:35 pm
Quote from: gtatters on October 02, 2021, 07:21:39 pm
Working out the calibration to temperature is really what I'd like to figure out.  While you are correct that we can't assume it is identical to the LWIR equations, in principle I would expect it to be similar, but can't say for cert.  The fact that we can see 4 "calibration" tags in the jpg suggests 4 cal constants.  The 5 planck constants from the .seq, .csq, old .jpg files that Exiftool extracts are technically 4 coefficients in an equation, since R = R1/R2. 

I took the 4 calibration blocks to be for four different conditions like four different lenses or four different temperature ranges. But I'm just guessing.

Regarding the 4 or 5 Planck constants, actually to do the most accurate temperature calculation there were 15 parameters including emissivity, object distance, relative humidity and others.

Going back to the files you uploaded today, as far as I can tell there aren't any palettes or roi's in these files. However, the palettes are used and the roi is shown in the (non-radiometric) jpegs.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 09:53:12 pm
Yes, I've used the extensive calculation used to account for atmospheric, reflected, window transmission (all those object parameters).  But it's the unique calibration constants (Exitfool calls them Planck R1, R2, B, O and F but they are a non-linear conversion of raw radiance to temperature).  Each camera should have some manufacturer calibration constants unique to the camera stored in the files - unless FLIR has devised another scheme.

Ah, sorry, I guess I forgot to include the "colour bar" as ResearchIR refers to it. 
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 02, 2021, 11:18:43 pm
I guess you could search in those calibration blocks using an IEEE-754 hex to decimal converter to see if any of the numbers seem reasonable for the Planck parameters. Course, you have to make an educated guess about where a number starts and whether it's a 4 or 8 byte float. Then also remember it's little endian so you have to enter the digits into the converter in reverse order. But, that's like searching for a needle in a haystack.
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 02, 2021, 11:27:25 pm
Lol. Yeah, that's about the extent of my searching talents.  I've just been looking with hex fiend for anything that looks suspicious but this model stores in 14 bit range not 16 bit range like the LWIR cams and so the cal constants would be completely different.  Need and haystack is indeed what it is like.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 11, 2021, 10:17:44 pm
Regarding earlier questions on model numbers, apparently there are 4 models in the A858x series having the four combinations of two spectral ranges and two apertures. The A8581 has a spectral range of 3 to 5 μm and an aperture of f/2.5. This and other info is in the User Manual attached below.

Regarding temperature calculations and other meta data, I noticed some patterns in the files that were previously uploaded. However it would be difficult reverse Engineering the file lacking more info from Flir. In the User Manual Section 2.3 it says:

"The A6000 and A8500 are stand-alone imaging cameras that interface to a host PC, Mac or Linux system running FLIR Research Studio software using Gigabit Ethernet. An SDK is available, which makes it possible for the system designer to write their own camera controller and acquire image data with their own custom application."

Wouldn't an SDK provide info on temperature calculations, or would the SDK be at a higher level?
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on October 12, 2021, 01:35:26 am
I haven't used their SDK, so I cannot say, but I doubt they will make the conversion process easy.
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 15, 2021, 04:24:08 pm
The question came up earlier about how to locate the start of the radiometric data after the FLIR APP1 segments have been pieced together in these RJPG files. I came up with an algorithm (or maybe its just a heuristic) for accomplishing this. If you look at these files with a Hex editor you will see a number of metadata blocks, or just call them blocks. For example you will see the following blocks:

FLIR ATS-US File
FOV
Workspace
SourceInfo
AbateCfg
NicevilleFactoryCal
ObjectParameters
Pixels

There might also be other blocks, and there can even be no-name (empty) blocks.
The algorithm is simple: walk the blocks and test for the name “Pixels”.  Eventually you arrive at the Pixels block.

As gtatters noted in an earlier post, the radiometric data starts 717 bytes from the Pixels block start. This is true for files that come directly from the camera. If the file has been edited in Research Studio, the radiometric data starts 621 bytes from the Pixels block start. Actually, it looks like there is either a 128-byte header or a 32-byte header on the radiometric data, accounting for 96 byte difference. I haven’t found anything in the files that is a flag for this difference.

I modified my radiometric data extraction program to find the start of the radiometric data using the “walk the blocks” algorithm noted above. The file attached to this post is a Unix executable command-line Mac app of the radiometric data extraction program. It should run on any Mac. The easiest way to use this program is to drag the Unix executable to an open Terminal window, then drag the RJPG file to the window and hit return. A few lines are printed about the RJPG file, and a TIFF file is created with the radiometric data with a .tiff extension. The TIFF file is placed in the same directory as the RJPG file. The program is named “RJPG_tiff_camera” to reflect the fact that it works for files directly from the camera. It won’t work correctly on files after they have been edited in Research Studio.


Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on October 28, 2021, 04:04:15 am
We would like to be able to calculate temperatures from the radiometric data for these Flir MWIR RJPEG files.
To this end, note that the RJPEG camera files have two data points for Counts, Radiance and Temperature in the Workspace xml data:

<segmentationValue unit="Counts" min="4107.3955078125" max="6884.330078125" />
<segmentationValue unit="Radiance" min="0.00013401229807641" max="0.00024988286895677" />
<segmentationValue unit="TemperatureC" min="18.424726486206" max="36.694522857666" />
<segmentationValue unit="TemperatureK" min="291.57473754883" max="309.84451293945" />

These values are the same in all the files, so they are apparently calibration related, rather than scene related.

To calculate temperatures, assume the equation is like the LWIR formulation:

tempK = B/ln((R1/R2)/(S-O) + F)

where: B, R1, R2, O, and F were parameters from the Flir LWIR metadata,  ln is natural log, and S = signal level (or A/D level).
Not surprisingly, the form of this equation is the same as the Inverse Planck function. See the link below to NOAA calculator reference for the Inverse Planck function (wavelength version).

The equation above can be simplified by replacing R1/R2 with R. Also F was between 0.5 and 2.0 with the usual value being 1.0 (like in the Inverse Planck function). Note, F is added to numbers that are in a range of 20,000 to 40,000, so F doesn’t materially change the result and F = 1 may as well be used. Also, for the InSb detectors used in this MWIR camera, the A/D levels are called “Counts”. So we have: 

tempK = B/ln(R/(Counts-O) + 1)

First, B can be solved for by comparing to the terms in the Inverse Planck function in the NOAA reference:

That is, B = c2/λ where c2 = 1.4387752e4
 
First, determine λ from radiance and temperature. Note that the radiance in the NOAA reference is in units of W/m^2 - sr μm, whereas the Flir values seem to be W/cm^2 sr μm. So the Flir values need to be multiplied by 10,000 (or just move the decimal point four places to the right). Plug the Flir min radiance (1.340122…) into the NOAA Inverse Planck calculator and tweak  λ to get temperature = 291.5747… . 
λ = 4.642 μm is pretty close and B = 1.4387752e4/4.642 or, B = 3099.4726 K

Now we need to determine R and O. From the xml data in the RJPEG files, we have two temperatures for two counts.
So to solve for R and O, basically we have two equations with two unknowns.
Solving these equations (not particularly interesting) we get:

R = 1.31769e8
O = 921.5458

Finally, converting to deg C temp we have:

tempC = 3099.4726 / ln(1.31769e8 / (Counts - 921.5458) + 1)  -273.15

For the Snap-0012.jpg file, min counts = 4781 and max counts = 12386, so min temp = 23.78 C and max temp = 58.36 C.
This is for emissivity = 1.0.
For emissivity = 0.92 and reflected temperature = 293.15 K (values in the object parameters part of the Workspace block),
min temp = 24.09 C and max temp = 60.48 C.

These temperature values need to be compared with the values from Research Studio software to determine if this is an accurate method for determining temperatures.


https://ncc.nesdis.noaa.gov/data/planck.html (https://ncc.nesdis.noaa.gov/data/planck.html)
Title: Re: Radiometric JPG & ATS file formats
Post by: gtatters on November 21, 2021, 04:51:07 pm
Hi Jim

I'm still catching up on all the work you have done here.  Would you be willing to share the source code for it, so I can follow what you've done?  I will see if I can make some progress myself in December when I have a bit more time.

Regards


 
Title: Re: Radiometric JPG & ATS file formats
Post by: JimM on November 21, 2021, 09:53:29 pm
Quote from: gtatters on November 21, 2021, 04:51:07 pm
Would you be willing to share the source code for it, so I can follow what you've done?

I can post the code, but it might not be too useful now. The code started out as plain C, but I have modified it to parse the xml. I used NSXMLParser to do the parsing (Objective C). Objective-C is macOS only. So now to port the code to another platform (Linux or PC), the xml parsing code would have to be replaced by some C xml parser library.

In the mean time, it would be good to know if the min and max temperatures that I got for the Snap-0012.jpg image (listed in my last post) are the same values that Research Studio gets.