Author Topic: Radiometric JPG & ATS file formats  (Read 8111 times)

0 Members and 1 Guest are viewing this topic.

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #25 on: September 22, 2021, 02:52:31 pm »
Hi all,

A minor discovery (at least for finding the pixel starts) is that in 5 different jpgs (technically these are FLIR ATS-US files I guess), the start of the pixel data appears to be 718 bytes offset to the Pixels byte tag (0x50 0x69 0x78 0x65 0x6c 0x73).  See screen shot. 
In this file, then pixel start is 8034 bytes offset from start of file.



You don't mention the file name for the data you posted, but this appears to be the image Snap-0010.jpg. If so, the "Pixels" string starts at 7317, so applying the 718 byte offset puts the start of the radiometric data at 8035 rather than 8034. The first pixels are then: 1CEE 1CF7 1CFE ... This appears to be Big Endian byte order so in decimal we have: 7406 7415 7422 ...

I haven't discovered anything more about these files. However I notice from the XML data that the palette name is always a null (i.e. no name). I'm guessing that this means just a default greyscale palette is being used. Can you select a palette and then we can see where the palette is in the file and what form it has? Also there doesn't appear to be any rois (region of interest ?). Possibly you can select a roi, and we can see how that shows up in the file. Course, this would all be interesting information but it doesn't tell us how to calculate temperatures from the radiometric data. We would need more info from FLIR for that.

Edit: Just realized something -- if the offset to the start of radiometric data is 717 not 718, then the data indeed starts at 8034 and is Little Endian, and the first pixels are: FF1C EE1C F71C ... giving decimal values of 7423 7406 7415.
« Last Edit: September 22, 2021, 03:19:56 pm by JimM »
 
The following users thanked this post: gtatters

Offline Miek

  • Regular Contributor
  • *
  • Posts: 75
  • Country: gb
Re: Radiometric JPG & ATS file formats
« Reply #26 on: September 23, 2021, 02:49:34 pm »
Just a minor thing, but I would recommend doing this in layers rather than trying to go straight to the pixel data from the jpg. First extract the ATS file from the jpg, then work with the ATS file alone to figure out its format. It'll be much easier to spot any length/offset fields in the ATS that way.
 
The following users thanked this post: gtatters

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #27 on: September 23, 2021, 05:28:38 pm »
Just a minor thing, but I would recommend doing this in layers rather than trying to go straight to the pixel data from the jpg. First extract the ATS file from the jpg, then work with the ATS file alone to figure out its format. It'll be much easier to spot any length/offset fields in the ATS that way.

Agree, in general you need to use a top-down approach to figure out where things are in these files. We want to determine much more than just where the pixel data is. However, as far as I know, these are just jpeg files with FLIR APP1 segments containing the radiometric data and meta data. So far I haven't found anything like a directory with offsets and lengths would help decode the files.  I don't know what an ATS file is and haven't found much information on-line about what an ATS file is. If you have any links to what's in an ATS file, please provide.
 
The following users thanked this post: gtatters

Offline Miek

  • Regular Contributor
  • *
  • Posts: 75
  • Country: gb
Re: Radiometric JPG & ATS file formats
« Reply #28 on: September 23, 2021, 07:41:19 pm »
If you extract the chunks of data from all the APP1 segments, combine them together, and save them out as a new file then you get a file starting with "FLIR ATS-US File" and it seems to match the format of the 'Rec-0018.ats' file included in the first post. I've attached 'Snap-0012.ats' as an example, and some python code that I used to extract it.

There's a few mentions of it dotted around online. The ResearchIR user guide mentions support for the format: "Supports SEQ, SAF, PTW, ATS, FCF and RJPG file formats".
This page has a Windows tool for working with the file: https://flir.custhelp.com/app/answers/detail/a_id/3043/~/post-calibration-of-recorded-data, I haven't tried it though.
 
The following users thanked this post: gtatters

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #29 on: September 29, 2021, 06:48:25 pm »
Just to close the loop on the radiometric images. I corrected the "off by one" error I had in the previous data and changed the output to tiff type files. Attached are two radiometric tiff files in little endian byte order extracted from the Snap-0012.jpg and Snap-0007.jpg files. These tiff files should be compatible with ImageJ application.
 
The following users thanked this post: gtatters

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #30 on: October 01, 2021, 05:24:27 pm »
Sorry that I missed all these responses (from september) - term busier than usual, and I will try to get caught up.

To recap: you would like me to capture images with different palettes (from null) and with ROIs turned on, so we can see how that influences the .ats file structure and possible the RJPG being saved.

Yes, the tiff file you created JimM is beautiful and free of the repeating noise.  Can't say I follow what you have done given my coding skills, but I'll fumble through.
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #31 on: October 01, 2021, 06:18:10 pm »
To recap: you would like me to capture images with different palettes (from null) and with ROIs turned on, so we can see how that influences the .ats file structure and possible the RJPG being saved.

Yes, and just one example RJPEG file would be enough.
Thanks.
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #32 on: October 02, 2021, 05:02:33 pm »
Sorry for the delays.  I've attached 2 versions of the same ATS RJPG (for lack of a better name), with different palettes.  Next post will be with ROIs.
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #33 on: October 02, 2021, 05:03:23 pm »
This contains two files but with a box roi as well.
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #34 on: October 02, 2021, 05:30:42 pm »
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #35 on: October 02, 2021, 06:48:30 pm »
Ok, thanks for the files. Did a quick perusal of the files, and it looks like there is no longer the "Workspace" block of data or the associated xml in these files. So I guess that data is only in the files when they first come from the camera, but not after a file has been modified. Probably makes sense. I'll have to look at the files more closely to locate the palette and roi information.

Regarding the ats naming -- I think the .ats file is just another Flir proprietary file format. In other words it has nothing to do with other .ats file information that might be found elsewhere on the internet. This was initially confusing (at least to me), but it's not the first time that Flir has adopted a file extension that is used elsewhere. So I guess ATS is Advanced Thermal Systems? Do you get an .ats file only for videos, and you get a .jpg (i.e. an RJPG) for pictures?

Regarding extraction of temperature data -- the Flir A8581 is a MWIR camera using an InSb Sterling cooled detector (actually I was only able to find the A8580 on-line, assume the A8581 is similar). However, the temperature extraction work that Phil Harvey and Tomas123 did 6 or 7 years ago was for Flir LWIR cameras using uncooled bolometers. I would not expect temperature extraction to be the same. But I am no expert on this. Maybe someone more knowledgeable on this could jump in -- Fraser ?

 
The following users thanked this post: gtatters

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #36 on: October 02, 2021, 07:21:39 pm »
Yes what you say makes sense.  Using ResearchIR (or the new Research Studio), i can capture a video (.ats) or a single radiometric file (.jpg).  And of course, I can open a file, and then export a new radiometric jpg with different conditions, but invariably, it seems this isn't the same as a live capture, as you say.

Re: ats, I am 99% certain that .ats file extension is intended for video data (high speed or otherwise) with their advanced software.  A single snapshot with ResearchIR is auto saved as a jpg (no choice that I can see).  If you ever wanted to see it in action, feel free to drop a message and I can set up a zoom.

Re: A8581 vs A8580.  Likely a minor difference, perhaps related to year of manufacture or some obscure feature that differs.  Generally, not a big diff in Flir cams between model numbers differing in the single digits.

Working out the calibration to temperature is really what I'd like to figure out.  While you are correct that we can't assume it is identical to the LWIR equations, in principle I would expect it to be similar, but can't say for cert.  The fact that we can see 4 "calibration" tags in the jpg suggests 4 cal constants.  The 5 planck constants from the .seq, .csq, old .jpg files that Exiftool extracts are technically 4 coefficients in an equation, since R = R1/R2. 

https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/2075/

 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #37 on: October 02, 2021, 08:55:35 pm »
Working out the calibration to temperature is really what I'd like to figure out.  While you are correct that we can't assume it is identical to the LWIR equations, in principle I would expect it to be similar, but can't say for cert.  The fact that we can see 4 "calibration" tags in the jpg suggests 4 cal constants.  The 5 planck constants from the .seq, .csq, old .jpg files that Exiftool extracts are technically 4 coefficients in an equation, since R = R1/R2. 

I took the 4 calibration blocks to be for four different conditions like four different lenses or four different temperature ranges. But I'm just guessing.

Regarding the 4 or 5 Planck constants, actually to do the most accurate temperature calculation there were 15 parameters including emissivity, object distance, relative humidity and others.

Going back to the files you uploaded today, as far as I can tell there aren't any palettes or roi's in these files. However, the palettes are used and the roi is shown in the (non-radiometric) jpegs.
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #38 on: October 02, 2021, 09:53:12 pm »
Yes, I've used the extensive calculation used to account for atmospheric, reflected, window transmission (all those object parameters).  But it's the unique calibration constants (Exitfool calls them Planck R1, R2, B, O and F but they are a non-linear conversion of raw radiance to temperature).  Each camera should have some manufacturer calibration constants unique to the camera stored in the files - unless FLIR has devised another scheme.

Ah, sorry, I guess I forgot to include the "colour bar" as ResearchIR refers to it. 
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #39 on: October 02, 2021, 11:18:43 pm »
I guess you could search in those calibration blocks using an IEEE-754 hex to decimal converter to see if any of the numbers seem reasonable for the Planck parameters. Course, you have to make an educated guess about where a number starts and whether it's a 4 or 8 byte float. Then also remember it's little endian so you have to enter the digits into the converter in reverse order. But, that's like searching for a needle in a haystack.
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #40 on: October 02, 2021, 11:27:25 pm »
Lol. Yeah, that's about the extent of my searching talents.  I've just been looking with hex fiend for anything that looks suspicious but this model stores in 14 bit range not 16 bit range like the LWIR cams and so the cal constants would be completely different.  Need and haystack is indeed what it is like.
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #41 on: October 11, 2021, 10:17:44 pm »
Regarding earlier questions on model numbers, apparently there are 4 models in the A858x series having the four combinations of two spectral ranges and two apertures. The A8581 has a spectral range of 3 to 5 μm and an aperture of f/2.5. This and other info is in the User Manual attached below.

Regarding temperature calculations and other meta data, I noticed some patterns in the files that were previously uploaded. However it would be difficult reverse Engineering the file lacking more info from Flir. In the User Manual Section 2.3 it says:

"The A6000 and A8500 are stand-alone imaging cameras that interface to a host PC, Mac or Linux system running FLIR Research Studio software using Gigabit Ethernet. An SDK is available, which makes it possible for the system designer to write their own camera controller and acquire image data with their own custom application."

Wouldn't an SDK provide info on temperature calculations, or would the SDK be at a higher level?
« Last Edit: October 11, 2021, 10:36:18 pm by JimM »
 
The following users thanked this post: gtatters

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #42 on: October 12, 2021, 01:35:26 am »
I haven't used their SDK, so I cannot say, but I doubt they will make the conversion process easy.
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #43 on: October 15, 2021, 04:24:08 pm »
The question came up earlier about how to locate the start of the radiometric data after the FLIR APP1 segments have been pieced together in these RJPG files. I came up with an algorithm (or maybe its just a heuristic) for accomplishing this. If you look at these files with a Hex editor you will see a number of metadata blocks, or just call them blocks. For example you will see the following blocks:

FLIR ATS-US File
FOV
Workspace
SourceInfo
AbateCfg
NicevilleFactoryCal
ObjectParameters
Pixels

There might also be other blocks, and there can even be no-name (empty) blocks.
The algorithm is simple: walk the blocks and test for the name “Pixels”.  Eventually you arrive at the Pixels block.

As gtatters noted in an earlier post, the radiometric data starts 717 bytes from the Pixels block start. This is true for files that come directly from the camera. If the file has been edited in Research Studio, the radiometric data starts 621 bytes from the Pixels block start. Actually, it looks like there is either a 128-byte header or a 32-byte header on the radiometric data, accounting for 96 byte difference. I haven’t found anything in the files that is a flag for this difference.

I modified my radiometric data extraction program to find the start of the radiometric data using the “walk the blocks” algorithm noted above. The file attached to this post is a Unix executable command-line Mac app of the radiometric data extraction program. It should run on any Mac. The easiest way to use this program is to drag the Unix executable to an open Terminal window, then drag the RJPG file to the window and hit return. A few lines are printed about the RJPG file, and a TIFF file is created with the radiometric data with a .tiff extension. The TIFF file is placed in the same directory as the RJPG file. The program is named “RJPG_tiff_camera” to reflect the fact that it works for files directly from the camera. It won’t work correctly on files after they have been edited in Research Studio.


« Last Edit: October 15, 2021, 06:10:43 pm by JimM »
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #44 on: October 28, 2021, 04:04:15 am »
We would like to be able to calculate temperatures from the radiometric data for these Flir MWIR RJPEG files.
To this end, note that the RJPEG camera files have two data points for Counts, Radiance and Temperature in the Workspace xml data:

<segmentationValue unit="Counts" min="4107.3955078125" max="6884.330078125" />
<segmentationValue unit="Radiance" min="0.00013401229807641" max="0.00024988286895677" />
<segmentationValue unit="TemperatureC" min="18.424726486206" max="36.694522857666" />
<segmentationValue unit="TemperatureK" min="291.57473754883" max="309.84451293945" />

These values are the same in all the files, so they are apparently calibration related, rather than scene related.

To calculate temperatures, assume the equation is like the LWIR formulation:

tempK = B/ln((R1/R2)/(S-O) + F)

where: B, R1, R2, O, and F were parameters from the Flir LWIR metadata,  ln is natural log, and S = signal level (or A/D level).
Not surprisingly, the form of this equation is the same as the Inverse Planck function. See the link below to NOAA calculator reference for the Inverse Planck function (wavelength version).

The equation above can be simplified by replacing R1/R2 with R. Also F was between 0.5 and 2.0 with the usual value being 1.0 (like in the Inverse Planck function). Note, F is added to numbers that are in a range of 20,000 to 40,000, so F doesn’t materially change the result and F = 1 may as well be used. Also, for the InSb detectors used in this MWIR camera, the A/D levels are called “Counts”. So we have: 

tempK = B/ln(R/(Counts-O) + 1)

First, B can be solved for by comparing to the terms in the Inverse Planck function in the NOAA reference:

That is, B = c2/λ where c2 = 1.4387752e4
 
First, determine λ from radiance and temperature. Note that the radiance in the NOAA reference is in units of W/m^2 - sr μm, whereas the Flir values seem to be W/cm^2 sr μm. So the Flir values need to be multiplied by 10,000 (or just move the decimal point four places to the right). Plug the Flir min radiance (1.340122…) into the NOAA Inverse Planck calculator and tweak  λ to get temperature = 291.5747… . 
λ = 4.642 μm is pretty close and B = 1.4387752e4/4.642 or, B = 3099.4726 K

Now we need to determine R and O. From the xml data in the RJPEG files, we have two temperatures for two counts.
So to solve for R and O, basically we have two equations with two unknowns.
Solving these equations (not particularly interesting) we get:

R = 1.31769e8
O = 921.5458

Finally, converting to deg C temp we have:

tempC = 3099.4726 / ln(1.31769e8 / (Counts - 921.5458) + 1)  -273.15

For the Snap-0012.jpg file, min counts = 4781 and max counts = 12386, so min temp = 23.78 C and max temp = 58.36 C.
This is for emissivity = 1.0.
For emissivity = 0.92 and reflected temperature = 293.15 K (values in the object parameters part of the Workspace block),
min temp = 24.09 C and max temp = 60.48 C.

These temperature values need to be compared with the values from Research Studio software to determine if this is an accurate method for determining temperatures.


https://ncc.nesdis.noaa.gov/data/planck.html
 

Offline gtattersTopic starter

  • Contributor
  • Posts: 44
Re: Radiometric JPG & ATS file formats
« Reply #45 on: November 21, 2021, 04:51:07 pm »
Hi Jim

I'm still catching up on all the work you have done here.  Would you be willing to share the source code for it, so I can follow what you've done?  I will see if I can make some progress myself in December when I have a bit more time.

Regards


 
 

Offline JimM

  • Regular Contributor
  • *
  • Posts: 64
  • Country: us
Re: Radiometric JPG & ATS file formats
« Reply #46 on: November 21, 2021, 09:53:29 pm »
Would you be willing to share the source code for it, so I can follow what you've done?

I can post the code, but it might not be too useful now. The code started out as plain C, but I have modified it to parse the xml. I used NSXMLParser to do the parsing (Objective C). Objective-C is macOS only. So now to port the code to another platform (Linux or PC), the xml parsing code would have to be replaced by some C xml parser library.

In the mean time, it would be good to know if the min and max temperatures that I got for the Snap-0012.jpg image (listed in my last post) are the same values that Research Studio gets.   
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf