Radiometric JPG & ATS file formats

[gtatters]

Hi all

I have a new FLIR A8581 (MWIR) with Flir Research Studio.  Nice camera, 1280x1024 resolution.  But the software claims to save radiometric jpg and .ats (thermal video) files, that do not seem to resemble radiometric jpg from LWIR cameras nor .seq or .csq file format.  I thought I'd share here.

The jpg yields no thermal data or planck constants using exiftool. 

The jpg file appears to store the raw data (in this case ~8032 bytes offset; see tiff extracted file), but the tiff data are contaminated with 40 intermittent 12 byte long garbage data (it would be nice to find a way to remove).  I have no idea what format the .ats.

3 files in zip file: Snap-0012.jpg (original, radiometric jpg), Snap-0012.tiff (extracted raw data, 16 bit little endian, converted in ImageJ), Rec-0018.ats (brief thermal video)

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=0

Sharing on a public dropbox folder above.

Cheers

[JimM]

Having trouble getting the files from DropBox. Can you upload just the jpg file to EEVblog directly?
From your description it sounds like a Flir JPEG-R file. But, I would have to look at it to be sure.

[gtatters]

Thanks for trying.  I don't know what the issue is, since I tested the link with a colleague but they have a dropbox account.

Let's change the dl=0 to dl=1 to direct download to see if that works since this is supposed to be a public link:

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1


Otherwise, I can't share the .ats file since it is 44Mb, and this forum will only let me post one file.  I'm curious to solve both the JPG-R issue and the .ats issue.

[JimM]

Ok, got the files from DropBox. The jpg in the DropBox ( 2.7 Mbytes) seems to be a JPEG-R file. It has the radiometric data split into APP1 segments with the 12-byte headers separating the segments as usual. However, the metadata looks totally different from other FLIR JPEG-R files. You're right, ExifTool doesn't show anything for the FLIR metadata. My own code crashed when I fed it this file, haha. So I guess some reverse Engineering is needed to figure out these files.

[gtatters]

Thanks JimM.  Yes indeed.  Tomas123 and Phil have been very helpful with other reverse engineering thus why I shared here.  Feel free to post anything you discover.

[JimM]

There is some xml near the end of the jpg file that has interesting metadata: min and max temperatures, emissivity, etc. Namely:

Code: [Select]

<workspaceFileSettings>
<segmentation enabled="False" scaledMin="-0.1252924328758" scaledMax="0.3884484193701">
<segmentationValues>
<segmentationValue unit="Counts" min="4107.3955078125" max="6884.330078125" />
<segmentationValue unit="Radiance" min="0.00013401229807641" max="0.00024988286895677" />
<segmentationValue unit="TemperatureC" min="18.424726486206" max="36.694522857666" />
<segmentationValue unit="TemperatureK" min="291.57473754883" max="309.84451293945" />
<segmentationValue unit="TemperatureF" min="65.164505004883" max="98.050140380859" />
<segmentationValue unit="TemperatureR" min="524.83453369141" max="557.72015380859" />
</segmentationValues>
</segmentation>
<playBounds start="0" end="0" />
<filterChain enabled="True">
<filters />
</filterChain>
<imageView zoom="0" panX="0" panY="0" hFlip="False" vFlip="False" angle="0" />
<palette name="" />
<roiList version="1.6" imageWidth="1280" imageHeight="1024" activeRoiIndex="0">
<imageRoi color="#808080" paletteIndex="0" />
<rois />
</roiList>
<source unit="Temperature (Factory)" temperatureType="Celsius" applyNUC="True" applyBP="True" showSaturationColors="True" />
<levelAndSpan mode="FromImage" scaledMin="0.194501" scaledMax="0.616296">
<levelAndSpanValues>
<levelAndSpanValue unit="TemperatureC" min="22.8804" max="58.4427" />
<levelAndSpanValue unit="TemperatureK" min="296.03" max="331.593" />
<levelAndSpanValue unit="TemperatureF" min="73.1847" max="137.197" />
<levelAndSpanValue unit="TemperatureR" min="532.855" max="596.867" />
</levelAndSpanValues>
</levelAndSpan>
<scaleOptions mode="Linear" plateauP="2" ddeSharpness="5" linearIgnore="0" useLog="False" lockScale="False" />
<objectParameters override="False" emissivity="0.92" distance="1" reflectedTemp="293.15" atmosphereTemp="293.15" extOpticsTemp="293.15" extOpticsTransmission="1" estAtmosphericTransmission="0" relativeHumidity="0.3" />
</workspaceFileSettings>
There is similar xml in the ats file, although some of the values are different.

A start towards reverse Engineering might be to see what temperatures FLIR software displays versus the values in the jpg file.

[gtatters]

I'm not familiar with xml or how you extract that (happy to learn), but I can confirm that the min=22.8804 and max=58.4427 correspond to the min and max temperature on the scale when you open the image up in flir research studio.

See screenshot.

[JimM]

Regarding the xml, the most general way to examine files is with a Hex Editor. On the Mac, I use Hex Fiend --probably my favorite application. Below is a screen shot of the Snap-0012.jpg file in Hex Fiend. Note the hex data is shown on the left. Most hex editors try to decode the hex data as ASCII text which is  on the right. So ASCII text stands out. You can then copy the ASCII text out of Hex Fiend and paste it into a text editor such as TextEdit on the Mac. TextEdit seems to automajically format the text in hierarchy form of xml. Course there's hex editors for Linux or PC if you are on those platforms.

If you get a Hex Editor, one thing to be aware of -- you can open and edit any file on your computer. There are certainly files that you would never want to (accidentally) edit. One very good thing about Hex Fiend is that it does not automatically update a file when you edit it. You have to explicitly save the file. I hate software that automatically updates the file when you edit it.

[gtatters]

Ah yes, thanks for that.  I use Hex Fiend as well.  Great program and I was able to see all these xlm tags, just wasn't sure how to automatically pull them out.  I was impressed at your coding in the earlier message and was curious if there was a command line tool for extracting xml.  I don't know the xml framework at all.  I really liked exiftool's capacity to cleanly pull out the meta-data inside these files! 

[JimM]

I use Hex Fiend as well.

Oh, ok. Sorry for the long winded hex editor explanation.
There's probably other ways of finding/extracting xml too.

[gtatters]

In terms of the 12 byte repeats that are strewn throughout the TIFF data, I think they are ****464C4952**** where 464C4952 corresponds to the Ascii "FLIR", surrounded on either side by 4 bytes of sometimes different values. 

Actually, I just noticed it's possible that the complete byte sequence is FF E1 FF FD 46 4C 49 52 xx xx xx xx, where xx is more variable.

Anyone with good perl or regex coding skills might be able to help me work out how to search and remove these contaminating elements?

[JimM]

The FF E1 is a JPEG APP1 segment marker. That's followed (usually) by FF FE (65534 decimal). That's the length of  the APP1 segment. The length includes the length parameter itself but not the marker. So you can find the end of the APP1 segment. The last APP1 segment has a length of 29 4B (10571 decimal). I don't know what the 4-bytes following the 'FLIR' characters are, but they seem to increment by 256 decimal in each APP1 segment.

[gtatters]

Ah, I didn't realise that.  Ok, that helps, somewhat, but FF FE = 65279 doesn't it, not 65534, which doesn't correspond to the true length of the APP1 segment from what I can tell.

I just want to delete 12 bytes whenever there is a FF E1 FF FD, since without these, the TIFF image should be intact, and represent the raw thermal data (or perhaps simply the temperature data).

[JimM]

No, JPEGS are big endian, so FF FE is 65534 decimal. All of the segments are 65534 bytes except the last one. Note the total length of a 65534 byte segment is 65536 bytes including the marker. So when you excise the 12-bytes you have 65524 bytes of data.

[gtatters]

Sorry about that.  Yes I see what you mean.  I'll see what I can figure out for splicing the 12-bytes out.  Not sure I know how to do that yet.

[JimM]

I'm working on some C code to extract the radiometric data also. One problem is, where does the radiometric data start? As you said in an earlier post, it looks like it's around 8032 offset. But lacking other information we don't know for sure. I was looking around for something that had that offset, but no luck. Course as long as we have a total 1024x1280x2 bytes for the image, we will get a reasonable looking image.   

[JimM]

Ok, so attached below is a pgm file of the radiometric data. As usual, the radiometric image has very low contrast (basically appears as black) in most image viewers. I passed the pgm image through ImageMagick convert -contrast-stretch with factors of 1%x0.5% to produce the PNG image also attached. Course, the pixel values in the PNG no longer have any relationship to temperatures. It's just a picture.

One odd thing about the radiometric data in the jpg: it appears that the last byte of the data is missing (the least significant byte of the last pixel).

Edit: I had an "off by one byte" error. The radiometric data is little endian, not big endian.

[JimM]

I forgot, the forum won't accept a pgm file. I put it in a zip file attached below.

[gtatters]

If it helps, I have added a few more jpgs (taken from the same program/camera), in case the variation in offset helps.  The TIFF style raw data does not start at the same offset in each image. 

I have put the images in the zip file on dropbox.

https://www.dropbox.com/s/eqzch2dxaw1nlak/Files.zip?dl=1

[gtatters]

Can you help me follow how you did this?  Maybe without the imagemagick step - I work in ImageJ and it's very easy to re-stretch the greyscale but keep the data intact.  The png file looks like it has lost some quality in the conversion.  Even just having the raw data - I can import raw image into ImageJ easily - no file header required.

[Miek]

I don't know what the 4-bytes following the 'FLIR' characters are, but they seem to increment by 256 decimal in each APP1 segment.

The latter two bytes of those are 8bit uints with the current chunk number and the total chunks.

[gtatters]

Aha, clever.  thanks for sleuthing that.  I could see they incremented and that there were 40 chunks, but didn't figure out it was 8 bit. 

[JimM]

Ok, if it helps, attached below is the C-code to extract the radiometric data from the jpg file. There's basically just 4-steps;

• Locate the APP1 segments and put the offsets and lengths of the segments in a table.
• Use the table to assemble the data in the segments, removing the 12-byte headers in each segment.
• The data still has extra bytes ahead of the radiometric data. Need to shave off these extra bytes. Manually you have to find out where the the radiometric data starts. For the case of Snap-0012.jpg file it starts at 8032, but the first 32 bytes have already been removed by the assembly step. So it starts at 8000.
• Write data to file, including a header. I obfuscated the file path in the code.

The code is hardwired to the Snap-0012.jpg file. There are magic numbers all over the place.
I'm going to be out of pocket for a couple of weeks, so won't be posting here for awhile. If you guys could do some more sleuthing to find out how to get to the start of the radiometric data, that would be useful.

Edit: As mentioned in Reply #16. I had an off by one byte error in the above. The radiometric data starts at offset 8031 or 7999 after the assembly step.

[gtatters]

This is really helpful!  And much appreciated.  My own progress will be slow, with start of term this week and chaos that comes with that.  I might try to work something out in perl as I don't know c++.

[gtatters]

Hi all,

A minor discovery (at least for finding the pixel starts) is that in 5 different jpgs (technically these are FLIR ATS-US files I guess), the start of the pixel data appears to be 718 bytes offset to the Pixels byte tag (0x50 0x69 0x78 0x65 0x6c 0x73).  See screen shot. 
In this file, then pixel start is 8034 bytes offset from start of file.

I'm still working on other solutions to this.  Then I need to sit down to work out what the pixel data are stored as.  Radiance and thus need converting to temperature (I don't have any calibration constants) or if they are representations mapped using the min/max data in the Workspace meta data.

Cheers