Flir Ex: Realtime raw radiometric data streaming via UVC

[Bud]

Ya baby, now we are talking! Thanks tomas123!

Though I had to update to Ubuntu 14.whatever, otherwise the OpenCV library did not compile. That is why I avoid Linux, the developers have no clue as to what flavor of Linux and what version their product works with. Guessing game every time and you better have all Developers packages in the Universe installed to mitigate the pain.

Anyway, your latest code worked and I took a liberty to tailor it a little. I made use of the original buffer initialization from the original test file, as there seemed to be some duplication in last version. I also commented out the frame data length message that was flooding the screen - I think we got the code working to the point we do not need to monitor that anymore. And a nice thing: I replaced CV_WINDOW_AUTOSIZE with CV_WINDOW_NORMAL, and now the window is resizable.



I also returned to waiting for a key press to end streaming vs a timeout.

BTW, why is this code only falls thru if ENTER key is pressed but not any other key ?

Code: [Select]

          // stream until key is pressed.
          char c;
          read(0, &c, 1);
          /* End the stream. Blocks until last callback is serviced */
          uvc_stop_streaming(devh);
          puts("Done streaming.");
 
So this actually is becoming a useful piece of software. You can make snapshots from the window (a sample attached). I think autoleveling may need to be improved, it did not always work well.

EDIT: attached the source

[Bud]

@tomas123

OK some more awkward code added to track mouse coordinates and display pixel value under the cursor using a slider control as a gauge meter



So this pretty much completes a Proof of Concept and now the actual pixel temperature can be calculated and displayed, you will find my comments in the code. I recall you were the one who reverse engineered FLIR temperature calculation? This at the moment is beyond my ability so pls see if you can update the code with it. Thks!

What else could be added is perhaps a center spot and box measures. I think this can be done with OpenCV.
For now you can position mouse cursor at the center of the image to emulate a center spot and then moving the camera will read temperature from it.

EDIT: just in case: I compile using the following command:

Code: [Select]

gcc -o flirfox2 flirfox2.c -luvc -lm `pkg-config --cflags --libs opencv`

[Bud]

@tomas123

And here is something interesting from me playing with window modes. When I used CV_WINDOW_OPENGL, moved the window on the screen, exploded to full screen then back, noticed something weird (selection is mine):



For some reason the camera changed frame data length and it seemed to stayed consistent until I returned from full screen mode. What the heck the camera was transmitting and how the heck the code managed to display a proper image??? 

Did not have time to play more with that, may be after adding streaming data to file will see if can replicate it and look what is in the extra data.

[tomas123]

I can confirm this additional bytes.
In my Ubuntu virtual machine I often have spontaneous glitches associated with changes of usb frame lenght.
With camera connected direct to host machine USB this glitches are rarely.
The additional bytes are without informations (most zeros)

I solved this glitches in the juchsjagd code by hardcoding the frame size with 320x240x2 Byte (crop the tail).

So this pretty much completes a Proof of Concept and now the actual pixel temperature can be calculated and displayed, you will find my comments in the code. I recall you were the one who reverse engineered FLIR temperature calculation? This at the moment is beyond my ability so pls see if you can update the code with it.

this is really hard
You can't grep a Flir temperature value from a 8 Bit grayscale video stream without additional informations (high/low value etc)
Therefore the best way is to enlarge the image size and imprint the temperature values (like max/min) outside the visible video.

the nice running fuchsjagd code from tmbinc contains all necessary code snippsets (font.h  )
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344

the mathematical equations for calculating RAW to temperature I posted here:
- sample with excel sheet: http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg27546.html#msg27546
- sample with php code https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072

As attachment I post my excel sheet with all formulas and some unpublished  infos from Flir

The required Flir parameters are calibrated constants for every camera. You get this values from every radiometric jpg with this exiftool command:

Code: [Select]

>exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* sample.jpg
Atmospheric Temperature         : 20.0 C
Atmospheric Trans Alpha 1       : 0.006569
Atmospheric Trans Alpha 2       : 0.012620
Atmospheric Trans Beta 1        : -0.002276
Atmospheric Trans Beta 2        : -0.006670
Atmospheric Trans X             : 1.900000
Planck R1                       : 15314.136
Planck B                        : 1404.6
Planck F                        : 1
Planck O                        : -978
Planck R2                       : 0.011565135
Object Distance                 : 1.00 m
Emissivity                      : 0.60
Reflected Apparent Temperature  : 20.0 C
Relative Humidity               : 50.0 %

[tmbinc]

What do you mean with this code snippset from fuchsjagd?
It's like a 20x20 pixel watermark.
You can see it in your video at the bottom on the left side.

Code: [Select]

  unsigned short *pix = frame->data;
  int x, y;
 
  for (y = 160; y < 180; ++y) {
    for (x = 60; x < 80; ++x) {
      pix[y * 320 + x] = pix[y * 320 + 60];
    }
  }

Haha, I did forget about that code. Sorry. I have some dirt on my lens (or sensor) which I didn't cleanup yet (out of fear to break something). This created a spot that read a slightly larger temperature, which triggered the "Fuchs temperature threshold" ("Ftt").

The, erm, solution was to just copy the area next to it over the spot.

[Pantheron]

Hello guys,

i tried this also, but i cant get a stream with the first flir.c release.
I also got some other streaming descriptors than you, probably you can help me out?

Thank you in advance.

It is an FLIR E4 wit FW 2.30 and hacked for Resolution and Menu (Firmware was original at 2.30)

Code: [Select]

root@iJudge:/home/pantheron/libuvc# build/flir e4x.raw
UVC initialized
Device found
unsupported descriptor subtype: 3
unsupported descriptor subtype: 13
unsupported descriptor subtype: 3
unsupported descriptor subtype: 13
Device opened
DEVICE CONFIGURATION (05ac:8510/CC2CAV0DA1DN9KE0) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 5
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 00
  size: 160x120
  bit rate: 24576000-147456000
  max frame size: 38400
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(2)
  capabilities: 00
  size: 176x144
  bit rate: 24576000-147456000
  max frame size: 50688
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(3)
  capabilities: 00
  size: 320x240
  bit rate: 24576000-147456000
  max frame size: 153600
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(4)
  capabilities: 00
  size: 352x288
  bit rate: 24576000-147456000
  max frame size: 202752
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(5)
  capabilities: 00
  size: 640x480
  bit rate: 24576000-147456000
  max frame size: 614400
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(6)
  capabilities: 00
  size: 480x360
  bit rate: 24576000-147456000
  max frame size: 345600
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(7)
  capabilities: 00
  size: 1280x720
  bit rate: 24576000-147456000
  max frame size: 1843200
  default interval: 1/10
  interval[0]: 1/10
MJPEGFormat(2)
  bits per pixel: 0
  GUID: 4d4a5047000000000000000000000000 (MJPG)
  default frame: 2
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 00
  size: 960x544
  bit rate: 24576000-147456000
  max frame size: 1044480
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(2)
  capabilities: 00
  size: 1024x576
  bit rate: 24576000-147456000
  max frame size: 1179648
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
FrameDescriptor(3)
  capabilities: 00
  size: 1280x720
  bit rate: 24576000-147456000
  max frame size: 1843200
  default interval: 1/29
  interval[0]: 1/29
  interval[1]: 1/25
  interval[2]: 1/23
  interval[3]: 1/14
END DEVICE CONFIGURATION
bmHint: af90
bFormatIndex: 5
bFrameIndex: 157
dwFrameInterval: 32766
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 4176
dwMaxVideoFrameSize: 0
dwMaxPayloadTransferSize: 4197312
bInterfaceNumber: 254
get_mode: Invalid mode (-51)
Device closed
UVC exited
root@iJudge:/home/pantheron/libuvc#

[Pantheron]

Okay,

it tooks the isight camera of the macbook...
So i added the vendor and product id directly into the code for the flir e4 and now it works

[Bud]

Good point. Can you post the changed code line , thks.

[Bud]

Here is a version that streams data in a uncompressed 8-bit grayscale AVI file.  It can then be opened and colorized in ImageJ.

[tomas123]

Good point. Can you post the changed code line , thks.

Code: [Select]

  res = uvc_find_device(
      ctx, &dev,
      0, 0, NULL);
change 0,0 to vid,pid

https://int80k.com/libuvc/doc/group__device.html

uvc_error_t uvc_find_device   (   uvc_context_t *    ctx,
uvc_device_t **    dev,
int    vid,
int    pid,
const char *    sn
)      
Finds a camera identified by vendor, product and/or serial number.

Parameters
[in]   ctx   UVC context in which to search for the camera
[out]   dev   Reference to the camera, or NULL if not found
[in]   vid   Vendor ID number, optional
[in]   pid   Product ID number, optional
[in]   sn   Serial number or NULL

[Bud]

Thanks tomas123 for your input and information/files in your previous post!

[Bud]

Here is a version that supports both RAW 16-bit and grayscale 8-bit streaming to file

Usage:

./flirfox4               :Display video, no streaming  to file
./flirfox4 -v      :same as above
./flirfox4 -r  filename   :stream 16-bit RAW thermal video  to file, no header
./flirfox4 -g filename   :stream 8-bit grayscale auto-leveled video  to file with AVI format header

Both can be opened in ImageJ program, color table applied and can be saved in a variety of formats.

EDIT:   and it also has E4 PID/VID hardcoded, so it only be looking for E4 camera.
EDIT:   do not expect well written code from me. First, I am Linux impaired. Second, I heard about C language that there is such thing and that is about it. Third, I am not a programmer. 

[Bud]

Temperature calculation added. Try it folks if it works for you.
You will need a file called e4data.txt with your camera's calibration data. You create the file using exiftool as tomas123 explained here: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673

You also need to make a sample image from your camera, exiftool extracts the data from it (google for exiftool download if you do not have it).

Run exiftool command against the sample E4 file:

Code: [Select]

exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ*  FLIR_sample.jpg >e4data.txt
Once made and inspected, place the e4data.txt file in the folder with the compiled executable. For now the program just displays temperature (at mouse pointer coordinates) in the top left corner. Compare it with readings on your camera by pointing the mouse and camera's crosshair to a same hot spot.

Thanks tomas123 for the formulas!

[tomas123]

  great work
unfortunately now I have not enough time to go deeper in your code

a fast check with Ubuntu results in a "nan" value for temperature (see screenshot)

Code: [Select]

$ exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ*  FLIR3344.jpg >e4data.txt

$ cat e4data.txt
Atmospheric Temperature         : 20.0 C
Planck R1                       : 13061.76
Planck B                        : 1352.2
Planck F                        : 2.7
Planck O                        : -7334
Planck R2                       : 0.027282318
Object Distance                 : 1.00 m
Emissivity                      : 0.95
Reflected Apparent Temperature  : 20.0 C
Relative Humidity               : 50.0 %

$ /usr/bin/cc  -o flirfox6 flirfox6.c -luvc `pkg-config --cflags --libs opencv` -lm
$ sudo ./flirfox6

I will tomorrow debug your nice code.


another note:

I compiled successful the libuvc patch on mac OS X 10.9.

my raw note for interested users (use macports):

Code: [Select]

//check
$ port installed | grep active
  cmake @3.3.1_0 (active)
  libusb @1.0.19_0 (active)
  jpeg @9a_1 (active)

$ git clone https://github.com/ktossell/libuvc
Cloning into 'libuvc'...
remote: Counting objects: 526, done.
remote: Total 526 (delta 0), reused 0 (delta 0), pack-reused 526
Receiving objects: 100% (526/526), 263.90 KiB | 309.00 KiB/s, done.
Resolving deltas: 100% (305/305), done.
Checking connectivity... done.

$ cd libuvc/

// now use  patch in the src folder
$ patch < libuvc.patch
   patching file stream.c

cd ..
$ mkdir build
$ cd build
$ cmake ..
-- The C compiler identification is AppleClang 5.1.0.5030040
-- The CXX compiler identification is AppleClang 5.1.0.5030040
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Release
-- No target type selected, default to shared library
-- Found JPEG: /opt/local/lib/libjpeg.dylib 
-- Building libuvc with JPEG support.
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   uvc

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /Users/jfried/Documents/Flir/uvc/libuvc/build

$ make
Scanning dependencies of target uvc
[ 10%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o
[ 20%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o
[ 30%] Building C object CMakeFiles/uvc.dir/src/device.c.o
[ 40%] Building C object CMakeFiles/uvc.dir/src/diag.c.o
[ 50%] Building C object CMakeFiles/uvc.dir/src/frame.c.o
[ 60%] Building C object CMakeFiles/uvc.dir/src/init.c.o
[ 70%] Building C object CMakeFiles/uvc.dir/src/stream.c.o
[ 80%] Building C object CMakeFiles/uvc.dir/src/misc.c.o
[ 90%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o
[100%] Linking C shared library libuvc.dylib
[100%] Built target uvc
the RAW streaming of Flir E4  works fine under OS X (sample code from tmbinc)

Code: [Select]

// fast check with my full pathes
$ cc -o flir flir.c -I/opt/local/include/ -I/Users/tomas/Documents/Flir/uvc/libuvc/build/include -I/Users/tomas/Documents/Flir/uvc/libuvc/include/  -L/Users/tomas/Documents/Flir/uvc/libuvc/build -luvc

$ ./flir
UVC initialized
usage: ./flir out.raw

./flir 1
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 1
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 03
  size: 320x240
  bit rate: 912384-912384
  max frame size: 153600
  default interval: 1/3
  interval[0]: 1/15
  interval[1]: 1/7
  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1441305922:212093, 153600 bytes
got frame 1441305922:308068, 153600 bytes
got frame 1441305922:404825, 153600 bytes
got frame 1441305922:500120, 153600 bytes
got frame 1441305922:594571, 153600 bytes
...

now I have some trouble with OpenCV under OS X ...

[Bud]

@tomas123

Oh-oh, seems a problem with exiftool outputting less data when run in linux vs windows. I ran exiftool in Windows, it dumps 15 parameters in total. Yours seems to dump only 10, that is what caused the nan error.
I will check what version of exiftool executable i have in windows.

[Bud]

So my exiftool Windows version is 9.9, this is what it outputs:

Code: [Select]

Atmospheric Temperature         : 20.0 C
Atmospheric Trans Alpha 1       : 0.00656
Atmospheric Trans Alpha 2       : 0.01262
Atmospheric Trans Beta 1        : -0.0022
Atmospheric Trans Beta 2        : -0.0066
Atmospheric Trans X             : 1.90000
Planck R1                       : 14901.4
Planck B                        : 1400.9
Planck F                        : 2.5
Planck O                        : -7187
Planck R2                       : 0.02804
Object Distance                 : 1.00 m
Emissivity                      : 0.95
Reflected Apparent Temperature  : 20.0 C
Relative Humidity               : 50.0 %
@tomas123: seems yours does not output alpha/beta/X, not sure if this is how  exiftool works in Linux 

In the program I used the formulas with Distance from your Excel spreadsheet, that required I believe all 15 entries.
BTW, the last version is just using atmospheric temperature, object distance, reflected temperature and humidity read from the sample file. I intended later to replace them with sliders added to the video window so it would be possible to adjust for the current scene, since this information is not in the video stream.

[Bud]

Latest version,

• Temperature label moved to its own overlay so it will not appear in recorded files
• A crosshair cursor is added to read temperature, use the mouse to drag the cursor. Left mouse Double click resets position to the image center.
• The cursor appears in 8-bit grayscale video files, won't appear in 16-bit RAW video files.
• Added sliders to adjust a few ambient parameters .

For now I reverted to a fixed size window, as the sliders eat window space, shrinking the image. Would probably add a command line switch later to allow for window resize.

[tomas123]

great work
unfortunately now I have not enough time to go deeper in your code

sorry, that I'm wasted your time

with this steps on my virtual machine

Code: [Select]

$ lsb_release -a
Description: Ubuntu 14.04.1 LTS

$ sudo apt-get install exiftool
Note, selecting 'libimage-exiftool-perl' instead of 'exiftool'

$ sudo apt-get install libimage-exiftool-perl

$ exiftool -ver
9.46
I got an old exiftool version from Jan. 11, 2014: "Version 9.46 production release" (without the values for atmosphere transmissivity)
... and to make matters worse, I'm a coder of the Flir part of exiftool

I downloaded the current version 10.0.1 from Phils homepage and now all works fine 


If you prefer to parse XML in C than you can use the X parameter in exiftool:

Code: [Select]

Image-ExifTool-10.01/exiftool -X -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR3344.jpg
<?xml version='1.0' encoding='UTF-8'?>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
<rdf:Description rdf:about='FLIR3344.jpg'
  xmlns:et='http://ns.exiftool.ca/1.0/' et:toolkit='Image::ExifTool 10.01'
  xmlns:FLIR='http://ns.exiftool.ca/APP1/FLIR/1.0/'>
 <FLIR:AtmosphericTemperature>20.0 C</FLIR:AtmosphericTemperature>
 <FLIR:AtmosphericTransAlpha1>0.006569</FLIR:AtmosphericTransAlpha1>
 <FLIR:AtmosphericTransAlpha2>0.012620</FLIR:AtmosphericTransAlpha2>
 <FLIR:AtmosphericTransBeta1>-0.002276</FLIR:AtmosphericTransBeta1>
 <FLIR:AtmosphericTransBeta2>-0.006670</FLIR:AtmosphericTransBeta2>
 <FLIR:AtmosphericTransX>1.900000</FLIR:AtmosphericTransX>
 <FLIR:PlanckR1>18453.355</FLIR:PlanckR1>
 <FLIR:PlanckB>1460.6</FLIR:PlanckB>
 <FLIR:PlanckF>1</FLIR:PlanckF>
 <FLIR:PlanckO>-7003</FLIR:PlanckO>
 <FLIR:PlanckR2>0.013993904</FLIR:PlanckR2>
 <FLIR:ObjectDistance>1.00 m</FLIR:ObjectDistance>
 <FLIR:Emissivity>0.95</FLIR:Emissivity>
 <FLIR:Emissivity>0.95</FLIR:Emissivity>
 <FLIR:ReflectedApparentTemperature>20.0 C</FLIR:ReflectedApparentTemperature>
 <FLIR:RelativeHumidity>50.0 %</FLIR:RelativeHumidity>
</rdf:Description>
</rdf:RDF>

[tomas123]

Because there are some minor differences between the temperature of your crosshair cursor and the E4 screen I rechecked my excel sheet with a sample image.

(1) load image in Flir Tools and export the temperature off all pixels in a csv file
(2) with exiftool and imagemagick export the raw values of some pixels
(3) with Excel compare calculated temperature with exported values from Flir Tools

The values in Flir Tools and my Excel sheet are identical with all digits (milliKelvin) 

The calculations in your code are also right. I checked it with:

Code: [Select]

185: sprintf(szText, "%3.1f C / RAW: %i",t, pix[cpY * 320 + cpX]);The temperature differences under crosshair I can't explain with averaging of 2x2 or 3x3 pixels on the E4 display 

all steps in bash

Code: [Select]

// export with Flir Tools the temperature of all Pixels (Flir Tools export as *.csv)
$ head -n11 FLIR0080.csv
File: ;C:\Users\admin\Desktop\1\FLIR0080.jpg

Parameters:;Emissivity:;0,95
;Refl. temp.:;20 °C
;Distance:;1 m
;Atmospheric temp.:;20 °C
;Ext. optics temp.:;20 °C
;Ext. optics trans.:;1
;Relative humidity:;50 %
;
Frame 1;18,359;18,373;18,373;18,359;18,387;18,429;18,401;18,457;18,457;18,443;18,443;18,429;18,415;18,415;18,401;18,401;18,401;18,387;18,387;18,373;18,359;18,345;18,331

// print a readable form
s jfried$ grep Frame FLIR0080.csv | tr ";" "\n" | head -n 17
Frame 1
18,359
18,373
18,373
18,359
18,387
18,429
18,401
18,457
18,457
18,443
18,443
18,429
18,415
18,415
18,401
18,401

// get Flir parameters for excel sheet
$ /Image-ExifTool-10.01/exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR0080.jpg
Atmospheric Temperature         : 20.0 C
Atmospheric Trans Alpha 1       : 0.006569
Atmospheric Trans Alpha 2       : 0.012620
Atmospheric Trans Beta 1        : -0.002276
Atmospheric Trans Beta 2        : -0.006670
Atmospheric Trans X             : 1.900000
Planck R1                       : 15119.786
Planck B                        : 1405.2
Planck F                        : 2.5
Planck O                        : -7652
Planck R2                       : 0.027600825
Object Distance                 : 1.00 m
Emissivity                      : 0.95
Reflected Apparent Temperature  : 20.0 C
Relative Humidity               : 50.0 %

// export RAW values of radiometric image
$ exiftool -b -RawThermalImage FLIR0080.jpg > FLIR0080.png

// swap bytes
$ convert -define png:swap-bytes=on FLIR0080.png FLIR0080swap.png

// print RAW values of first 16 Pixel (top left)
$ for i in $(seq 0 15); do convert FLIR0080swap.png -crop 1x1+$i+0 -format "%[fx:round(65535*u.r)]" info:;echo ; done
12167
12168
12168
12167
12169
12172
12170
12174
12174
12173
12173
12172
12171
12171
12170
12170
a pixel around the center of image (the crosshair) 
$ convert FLIR0080swap.png -crop 1x1+160+120 -format "%[fx:round(65535*u.r)]" info:
12644
=> 24,785 °C (see imprinted spot 24,8°C)

see my links in signature for details
and https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg350250/#msg350250

[Bud]

Thanks tomas123. Could the difference be because of rounding errors in Linux math, the formulas have power of 2 and 3 as well as square roots, so rounding errors may accumulate. I think what can be done is to run the temp calculation routine with a known input value and check results of each step in the routine against your spreadsheet. This way we can catch where the difference occurs.

[Bud]

@tomas123
That is interesting the doctored libuvc works on Mac OS. Nice. I guess may also work on Win10, i heard it overcomes the limitations that prevented libuvc from working with camera streaming in Win7.

[tomas123]

The differences are only visible in rare situations at higher temperatures and only small (<0,5K).
I'm not sure, but it doesn't look like a average of 4x4 pixels.
Also this differences are not rounding errors.

It's possible, that the E4 raw video stream contains less "magic Flir" post procession filters like the radiometric images (jpg).

[Bud]

Must be some secret sauce in E4, since temperature calculation in  the program matches pretty well your Excel spreadsheet, I checked for a pixel level 10000:

T_Obj   -8.397801     -8.397789

Anyway, attached is the version with  MJPG compression added ( -m switch, will save 8-bit MJPG compressed file).
I will probably stop here for now.

EDIT: I think this program with some additions may be useful as a data logger for rapidly changing temperatures, since readings are available for each frame. Even this version can be used for that. Write video to a 8-bit file, open the file in any video viewer and step through the frames to read temperature at the cursor.

[tomas123]

I will probably stop here for now.

You've done a great job 
I like your solution for temperature measurement under the free movable crosshair.

I did not have enough time to work with my raspberry pi (spycam for  animals).
The OpenVC window is not the best solution for a ssh remote session disconnected with screen.
As a next step, I'll try to send the 8 Bit video stream to a v4l2loopback device (a video for linux loopback device like /dev/video1).

PS: I wasted many time for running your c code under Mac OS X.
The OpenCV library under OS X is a pain. I have unresolved problems with the OpenCV heap inside the function cb(frame,ptr) and the Qt library.
I think the best solution is switching to the newer opencv c++ api
But I give up 

As I wrote here, the patched libuvc library works stable under OS X:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg747569/#msg747569

[Bud]

Yes I did not exactly like OpenCV but was easier for an inexperienced person like me to come up with some code. I run Ubuntu 14 on a Win7 guest using Oracle VirtualBox (32Gb RAM) and OpenCV appears to be glitchy, not sure if this is because it runs on a VM. It also seems to write 9 fps video with a 15fps header which causing the video to play too fast. Not sure how it ends up with 9 fps.

Anyway, this is what I call "better than nothing" and I tried to summarize the steps below:

The purpose of this topic and this program was to get FLIR E4 camera to stream RAW thermal 16-bit video that can be written in a file as uncompressed 16-bit grayscale video for further processing elsewhere. As a by side feature, the program can display a rescaled 8-bit video in a window and can also write 8-bit grayscale uncompressed and MJPG compressed files that include temperature measurement under a spot cursor that can be moved/set in the program using the mouse. The program also have a few adjustment controls that allow to change several environmental parameters and obtain corrected temperature measurements in real time.

The program seems to only work in Linux at this time because the underlying libraries (libuvc and OpenCV) seem to not provide required functionality in Win7 or MacOS.

Steps to get it running in Linux (I used Ubuntu 14):

• Download libuvc source and patch it as in this post from tomas123:
  https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604
  (use the patch file provided by user tmbinc in the zip archive attached to the very first message in this topic)
• Download and Compile OpenCV library as in this post from tomas123 (step 2)
  https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg742931/#msg742931
  
• Compile the program code
     gcc -o flirfox9 flirfox9.c -luvc -lm `pkg-config --cflags --libs opencv`
• Take a sample image from your camera, use exiftool to extract calibration constants for your camera from it (google for exiftool download if you do not have it).
  Run exiftool command against the sample E4 file:
     exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ*  FLIR_sample.jpg >e4data.txt
  Place the produced e4data.txt file in the folder with the compiled executable.
• Connect the camera and run the program
  ./flirfox9  [options]

Options:
 -r   filename  :write 16-bit RAW data to file
 -g  filename  :write 8-bit grayscale data to file
 -m filename  :write 8-bit grayscale MJPG compressed data to file
 -v                :only display 8-bit grayscale video

Use the mouse to click-n-drug the temperature cursor to the desired position. left mouse dbl-click resets the cursor to the image center.

For post processing you may want to take a look at a nice piece of software called ImageJ
http://imagej.nih.gov/ij/

It has bunch of image processing options plus bunch of color palettes (LUTs) to color the video and resave in various formats.