EEVblog Electronics Community Forum
Products => Thermal Imaging => Topic started by: tmbinc on March 05, 2015, 11:20:56 pm
-
[Apologize if this is common knowledge and I just missed the memo. As everyone knows, the main thread is _huge_ and there was some talk about this, but I didn't find the final verdict of whether it's possible or not.]
I was trying to catch a fox that ate our pet rabbits from our garden on camera (people say they return to the scene of the crime). I wasn't successful (yet!), but I learned to hate the FLIR UVC implementation since it's randomly stops working during the night.
In an attempt to rectify the situation, I looked at the UVC implementation on the camera, and figured out that there's a dormant function to stream 9Hz raw radiometric data instead of just capturing the screen. With Windows and the regular UVC drivers, the best I could capture was the screen with 3.3Hz (for whatever reason; 15Hz should be possible) at (effectively) 8-bit over a pre-selected temp range.
The catch is that the UVC driver on the Flir explicitely disables radiometric streaming for the Z3 (=Flir Ex). It enables it for other cameras, though - i think the Flir Ex0 suports it out-of-the-box. The UVC stack is implemented in usbfnvideo.dll, which unfortunately is part of nk.bin, so it can't be easily patched (at least, I can't with my limited WinCE knowledge and my unwillingness to brick the E4).
The great thing is that FLIR sucks at implementing device restrictions; they have a dedicated grabber thread for the Flir Ex-series, and it _does_ indeed implement grabbing the raw data (via FVD1: instead of capturing the screen), and they _let_ you select capturing that (via setting bFormatIndex in the stream control), but they don't announce the ability for that in the UVC descriptors, so any "well-behaving" UVC client will not allow you to select it.
I've hacked libuvc (which is based on libusb) and - tada, it just worked. I patched libuvc pretty crudely so I'm sure there's a better way to do it. I've attached the patch and the crude sample (which just dumps out raw data into a file).
Let me know if you find this useful.
-
First time I have seen any info on this myself - nice work!
Would love to see what the output can turn out like.
-
A few hundred lines of code later (simply record all frames with a max temp of higher than some threshold, re-mapping to 8-bit, then encoding as mp4), I got a good picture of the beast:
https://www.youtube.com/watch?v=Gv5Fyg_1660 (https://www.youtube.com/watch?v=Gv5Fyg_1660)
Mission accomplished.
-
Hi tmbinc, if you could post a compiled executable and a patched dll that would help us try. ;)
-
edit 08.09.2015
----------------------link list for this thread: ----------------------
You need a E4 firmware 2.3.0!! FW 1.19.8 doesn't work (see post below)
source code for above video fuchsjagd
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344)
calculate temperature from 16 bit RAW values
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748218/#msg748218 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748218/#msg748218)
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673)
rebuild a radiometric flir jpeg from a single RAW frame
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732177/#msg732177 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732177/#msg732177)
use ImageJ for reading a RAW video file
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732349/#msg732349 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732349/#msg732349)
stream RAW thermal 16-bit video and display a rescaled 8-bit video in a OpenCV window:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg749464/#msg749464 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg749464/#msg749464)
source code: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748654/#msg748654 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748654/#msg748654)
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=170013;image)
stream a 8-bit video to V4L2 loopback devices ( /dev/video0 ):
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530)
an older post for streaming the E4 screen video to a raspberry pi (UVC, no RAW video)
https://www.eevblog.com/forum/testgear/capturing-video-from-flir-e4-to-portable-device/msg368083/#msg368083 (https://www.eevblog.com/forum/testgear/capturing-video-from-flir-e4-to-portable-device/msg368083/#msg368083)
---------------------- end link list ----------------------
only for information:
I'm played around with the uvc patch, but it doesn't work with my E4 (RAW Video).
E4
Modell: E4 1.1
Software: 1.19.8
first compile patched UVC library
$ uname -a
Linux ubu64 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
$ apt-get install CMake
$ apt --installed list| grep -i usb
libusb-0.1-4/trusty,now 2:0.1.12-23.3ubuntu1 amd64 [installed]
libusb-1.0-0/trusty,now 2:1.0.17-1ubuntu2 amd64 [installed]
$ sudo apt-get update && sudo apt-get install build-essential
$ sudo apt-get install libjpeg-dev
$ sudo apt-get install libusb-1.0-0-dev
$ git clone [url]https://github.com/ktossell/libuvc[/url]
// don't forget to apply the patch from tmbinc in /src/
$ patch < libuvc.patch
patching file stream.c
$ cd libuvc
$ mkdir build
$ cd build
$ cmake ..
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- No build type selected, default to Release
-- No target type selected, default to shared library
-- Building libuvc with JPEG support.
-- Configuring done
-- Generating done
-- Build files have been written to: ../flir/libuvc/build
$ make
Scanning dependencies of target uvc
[ 11%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o
[ 22%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o
[ 33%] Building C object CMakeFiles/uvc.dir/src/device.c.o
[ 44%] Building C object CMakeFiles/uvc.dir/src/diag.c.o
[ 55%] Building C object CMakeFiles/uvc.dir/src/frame.c.o
[ 66%] Building C object CMakeFiles/uvc.dir/src/init.c.o
[ 77%] Building C object CMakeFiles/uvc.dir/src/stream.c.o
[ 88%] Building C object CMakeFiles/uvc.dir/src/misc.c.o
[100%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o
Linking C shared library libuvc.so
[100%] Built target uvc
$ sudo make install
[100%] Built target uvc
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/libuvc.so
-- Installing: /usr/local/include/libuvc/libuvc.h
-- Installing: /usr/local/include/libuvc/libuvc_config.h
-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfig.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfigVersion.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets-release.cmake
renew library cache and compile flir.c tool from user tmbinc :-+
$ sudo ldconfig
$ /usr/bin/cc -o flir ../src/flir.c -luvc
check Flir E4 USB announcements
$ lsusb -v
Bus 001 Device 006: ID 09cb:1007
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x09cb
idProduct 0x1007
bcdDevice 0.00
iManufacturer 7 FLIR Systems
iProduct 8 FLIR Ex-Series
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 241
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 10 FLIR USB Video
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 2 FLIR Ex-Series
VideoControl Interface Descriptor:
bLength 13
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 85
dwClockFrequency 0.001000MHz
bInCollection 1
baInterfaceNr( 0) 1
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 0
wObjectiveFocalLengthMax 0
wOcularFocalLength 0
bControlSize 3
bmControls 0x00020260
Focus (Absolute)
Focus (Relative)
Zoom (Absolute)
Focus, Auto
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 3
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 6
iTerminal 0
VideoControl Interface Descriptor:
bLength 7
bDescriptorType 36
bDescriptorSubtype 4 (SELECTOR_UNIT)
bUnitID 4
bNrInPins 1
baSource( 0) 1
iSelector 0
VideoControl Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 5
bSourceID 4
wMaxMultiplier 0
bControlSize 2
bmControls 0x00000217
Brightness
Contrast
Hue
Sharpness
Gain
iProcessing 0
bmVideoStandards 0x1b
None
NTSC - 525/60
SECAM - 625/50
NTSC - 625/50
VideoControl Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 6
guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}
bNumControl 4
bNrPins 1
baSourceID( 0) 5
bControlSize 2
bmControls( 0) 0x0f
bmControls( 1) 0x00
iExtension 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 16
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 1
wTotalLength 79
bEndPointAddress 130
bmInfo 0
bTerminalLink 3
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 27
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 1
bNumFrameDescriptors 1
guidFormat {59555932-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 38
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 320
wHeight 240
dwMinBitRate 912384
dwMaxBitRate 912384
dwMaxVideoFrameBufferSize 153600
dwDefaultFrameInterval 2666664
bFrameIntervalType 3
dwFrameInterval( 0) 666666
dwFrameInterval( 1) 1333332
dwFrameInterval( 2) 2666664
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x04 EP 4 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0000
(Bus Powered)
now grep a stream$ sudo libuvc/flir e4-1.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439397994:439760, 153600 bytes
got frame 1439397994:533467, 153600 bytes
got frame 1439397994:629556, 153600 bytes
got frame 1439397994:726339, 153600 bytes
got frame 1439397994:820088, 153600 bytes
got frame 1439397994:916235, 153600 bytes
got frame 1439397995:22432, 153600 bytes
got frame 1439397995:115699, 153600 bytes
got frame 1439397995:209889, 153600 bytes
got frame 1439397995:304196, 153600 bytes
split the stream and convert a single frame
$ split -b 153600 e4-1.raw
$ convert -size 320x240 yuv:xaa -separate 4.png
the result is the known screen image and not a RAW image:(
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=164996;image)
-
the same process with an E40 works fine
E40
Modell E40 0.10
Software: 2.23.14
USB annoucement
$ lsusb -v
Bus 001 Device 008: ID 09cb:1005
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x09cb
idProduct 0x1005
bcdDevice 0.00
iManufacturer 6 FLIR Systems
iProduct 7 FLIR Exx Series
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 345
bNumInterfaces 4
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xc0
Self Powered
MaxPower 0mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 2 Communications
bFunctionSubClass 2 Abstract (modem)
bFunctionProtocol 255 Vendor Specific (MSFT RNDIS?)
iFunction 9 FLIR Network Camera
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 2 Communications
bInterfaceSubClass 2 Abstract (modem)
bInterfaceProtocol 255 Vendor Specific (MSFT RNDIS?)
iInterface 0
CDC Header:
bcdCDC 1.10
CDC Call Management:
bmCapabilities 0x00
bDataInterface 1
CDC ACM:
bmCapabilities 0x00
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 4
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 10 CDC Data
bInterfaceSubClass 0 Unused
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x03 EP 3 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 2
bInterfaceCount 2
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 10 FLIR USB Video
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 4 FLIR Exx Series
VideoControl Interface Descriptor:
bLength 13
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 85
dwClockFrequency 0.001000MHz
bInCollection 1
baInterfaceNr( 0) 3
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 0
wObjectiveFocalLengthMax 0
wOcularFocalLength 0
bControlSize 3
bmControls 0x00020260
Focus (Absolute)
Focus (Relative)
Zoom (Absolute)
Focus, Auto
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 3
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 6
iTerminal 0
VideoControl Interface Descriptor:
bLength 7
bDescriptorType 36
bDescriptorSubtype 4 (SELECTOR_UNIT)
bUnitID 4
bNrInPins 1
baSource( 0) 1
iSelector 0
VideoControl Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 5
bSourceID 4
wMaxMultiplier 0
bControlSize 2
bmControls 0x00000217
Brightness
Contrast
Hue
Sharpness
Gain
iProcessing 0
bmVideoStandards 0x1b
None
NTSC - 525/60
SECAM - 625/50
NTSC - 625/50
VideoControl Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 6
guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}
bNumControl 4
bNrPins 1
baSourceID( 0) 5
bControlSize 2
bmControls( 0) 0x0f
bmControls( 1) 0x00
iExtension 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x84 EP 4 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 16
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 3
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 15
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 2
wTotalLength 145
bEndPointAddress 133
bmInfo 0
bTerminalLink 3
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 27
bmaControls( 1) 27
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 1
bNumFrameDescriptors 1
guidFormat {59555932-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 38
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 320
wHeight 240
dwMinBitRate 912384
dwMaxBitRate 912384
dwMaxVideoFrameBufferSize 153600
dwDefaultFrameInterval 1333332
bFrameIntervalType 3
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 666666
dwFrameInterval( 2) 1333332
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 2
bNumFrameDescriptors 1
guidFormat {46374d30-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 38
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 320
wHeight 246
dwMinBitRate 912384
dwMaxBitRate 912384
dwMaxVideoFrameBufferSize 157440
dwDefaultFrameInterval 1333332
bFrameIntervalType 3
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 666666
dwFrameInterval( 2) 1333332
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x85 EP 5 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0001
Self Powered
grep a stream
$ sudo libuvc/flir e40.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1004/0a7f4000-c632-0000-0000-000000000000) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/7
interval[0]: 1/30
interval[1]: 1/15
interval[2]: 1/7
UncompressedFormat(2)
bits per pixel: 16
GUID: 46374d3000001000800000aa00389b71 (F7M0)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x246
bit rate: 912384-912384
max frame size: 157440
default interval: 1/7
interval[0]: 1/30
interval[1]: 1/15
interval[2]: 1/7
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 157440
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439394751:346415, 157440 bytes
got frame 1439394751:411283, 157440 bytes
got frame 1439394751:479331, 157440 bytes
got frame 1439394751:546262, 157440 bytes
got frame 1439394751:610415, 157440 bytes
got frame 1439394751:679598, 157440 bytes
got frame 1439394751:741424, 157440 bytes
got frame 1439394751:810494, 157440 bytes
got frame 1439394751:874681, 157440 bytes
got frame 1439394751:938867, 157440 bytes
....
the streams contains RAW FFF frames
$ hexdump -C e40.raw
00000000 f4 48 f6 48 f7 48 03 49 04 49 02 49 01 49 00 49 |.H.H.H.I.I.I.I.I|
00000010 14 49 fc 48 ff 48 04 49 fe 48 0a 49 fd 48 0d 49 |.I.H.H.I.H.I.H.I|
...
000257f0 09 49 19 49 18 49 0e 49 0e 49 0c 49 13 49 32 49 |.I.I.I.I.I.I.I2I|
00025800 46 46 46 00 55 56 43 00 00 00 00 00 00 00 00 00 |FFF.UVC.........|
00025810 00 00 00 00 64 00 00 00 40 00 00 00 02 00 00 00 |....d...@.......|
split the stream and convert a frame
$ split -b 157440 e40.raw
$ convert -depth 16 -size 320x240 -auto-level gray:xaa 8.png
it's a RAW image :-+
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=165005;image)
check frame rate with exiftool (15 fps)
$ dd if=e40.raw of=e40allframe.ff bs=153600 skip=1
$ exiftool -Date* x*
======== xaa
Date/Time Original : 2015:08:12 17:39:48.413+01:00
======== xab
Date/Time Original : 2015:08:12 17:39:48.479+01:00
======== xac
Date/Time Original : 2015:08:12 17:39:48.545+01:00
======== xad
Date/Time Original : 2015:08:12 17:39:48.611+01:00
======== xae
Date/Time Original : 2015:08:12 17:39:48.677+01:00
======== xaf
Date/Time Original : 2015:08:12 17:39:48.743+01:00
======== xag
Date/Time Original : 2015:08:12 17:39:48.809+01:00
======== xah
Date/Time Original : 2015:08:12 17:39:48.875+01:00
======== xai
Date/Time Original : 2015:08:12 17:39:48.941+01:00
======== xaj
Date/Time Original : 2015:08:12 17:39:49.007+01:00
======== xak
Date/Time Original : 2015:08:12 17:39:49.073+01:00
======== xal
Date/Time Original : 2015:08:12 17:39:49.139+01:00
======== xam
Date/Time Original : 2015:08:12 17:39:49.205+01:00
======== xan
Date/Time Original : 2015:08:12 17:39:49.271+01:00
======== xao
Date/Time Original : 2015:08:12 17:39:49.337+01:00
======== xap
Date/Time Original : 2015:08:12 17:39:49.403+01:00
or all values:
$ exiftool xaa
ExifTool Version Number : 9.54
File Name : xaa
Directory : .
File Size : 154 kB
File Modification Date/Time : 2015:08:12 18:16:45+02:00
File Access Date/Time : 2015:08:12 18:27:35+02:00
File Inode Change Date/Time : 2015:08:12 18:16:45+02:00
File Permissions : rw-r--r--
File Type : FLIR
MIME Type : application/unknown
Emissivity : 0.95
Object Distance : 1.00 m
Reflected Apparent Temperature : 20.0 C
Atmospheric Temperature : -4.0 C
IR Window Temperature : 20.0 C
IR Window Transmission : 1.00
Relative Humidity : 50.0 %
Planck R1 : 14345.714
Planck B : 1347.7
Planck F : 1
Atmospheric Trans Alpha 1 : 0.006569
Atmospheric Trans Alpha 2 : 0.012620
Atmospheric Trans Beta 1 : -0.002276
Atmospheric Trans Beta 2 : -0.006670
Atmospheric Trans X : 1.900000
Camera Temperature Range Max : 120.0 C
Camera Temperature Range Min : -20.0 C
Camera Model : FLIR E40
Camera Part Number : 49001-2001
Camera Serial Number : 490337633
Camera Software : 20.0.0
Lens Model : FOL18
Lens Part Number :
Lens Serial Number :
Field Of View : 25.0 deg
Filter Model :
Filter Part Number :
Filter Serial Number :
Planck O : -5711
Planck R2 : 0.01234
Raw Value Median : 18457
Raw Value Range : 780
Date/Time Original : 2015:08:12 17:39:48.413+01:00
Focus Step Count : 1526
Focus Distance : 5.3 m
-
Meh. Okay.
I'm on 2.3, so maybe it was added later. Could someone with a Flir Ex with 2.3 firmware give it a try? I wouldn't know what else it could be.
-
Do you see differences in the usb announcements "VideoControl Interface Descriptor" between my firmware 1.19.8 and your firmware 2.3?
$ lsusb -v
Bus 001 Device 006: ID 09cb:1007
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x09cb
idProduct 0x1007
bcdDevice 0.00
iManufacturer 7 FLIR Systems
iProduct 8 FLIR Ex-Series
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 241
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 10 FLIR USB Video
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 2 FLIR Ex-Series
VideoControl Interface Descriptor:
bLength 13
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 85
dwClockFrequency 0.001000MHz
bInCollection 1
baInterfaceNr( 0) 1
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 0
wObjectiveFocalLengthMax 0
wOcularFocalLength 0
bControlSize 3
bmControls 0x00020260
Focus (Absolute)
Focus (Relative)
Zoom (Absolute)
Focus, Auto
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 3
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 6
iTerminal 0
VideoControl Interface Descriptor:
bLength 7
bDescriptorType 36
bDescriptorSubtype 4 (SELECTOR_UNIT)
bUnitID 4
bNrInPins 1
baSource( 0) 1
iSelector 0
VideoControl Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 5
bSourceID 4
wMaxMultiplier 0
bControlSize 2
bmControls 0x00000217
Brightness
Contrast
Hue
Sharpness
Gain
iProcessing 0
bmVideoStandards 0x1b
None
NTSC - 525/60
SECAM - 625/50
NTSC - 625/50
VideoControl Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 6
guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}
bNumControl 4
bNrPins 1
baSourceID( 0) 5
bControlSize 2
bmControls( 0) 0x0f
bmControls( 1) 0x00
iExtension 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 16
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 1
wTotalLength 79
bEndPointAddress 130
bmInfo 0
bTerminalLink 3
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 27
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 1
bNumFrameDescriptors 1
guidFormat {59555932-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 38
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 320
wHeight 240
dwMinBitRate 912384
dwMaxBitRate 912384
dwMaxVideoFrameBufferSize 153600
dwDefaultFrameInterval 2666664
bFrameIntervalType 3
dwFrameInterval( 0) 666666
dwFrameInterval( 1) 1333332
dwFrameInterval( 2) 2666664
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x04 EP 4 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0000
(Bus Powered)
A few hundred lines of code later (simply record all frames with a max temp of higher than some threshold, re-mapping to 8-bit, then encoding as mp4)
I'm very interested in your Fuchsjagd code snippets :)
-
Since mine is 2.3.0 I spun up a Ubuntu 14.04.3 VM (First time every trying my hand at compiling anything) and got as far at the line listed above before I for the following error:
orby@ubuntu:~/libuvc/build$ /usr/bin/cc -o flir ../src/flir.c -luvc
../src/flir.c: In function ‘cb’:
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘__time_t’ [-Wformat=]
printf("got frame %d:%d, %d bytes\n", tv.tv_sec, tv.tv_usec, frame->data_bytes);
^
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘__suseconds_t’ [-Wformat=]
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]
Everything seemed to go word for word as outlined in tomas123 detailed instructions till that point.
I have no issues saying that I am in way over my head so unless anyone has any suggestions I don't think I can test further.
I would love to get a libuvc with this patch with this working on windows but my limited looking into it seems to indicate that the USB driver wrangling situation to accomplish this would be even more complex.
-
the result is the known screen image and not a RAW image:(
Same here, with FW 2.3.0
root@Ubuntu32:/home/some/Temp/libuvc# ./myflir test6.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439442632:196459, 153600 bytes
got frame 1439442632:290426, 153600 bytes
got frame 1439442632:384196, 153600 bytes
got frame 1439442632:478789, 153600 bytes
got frame 1439442632:573349, 153600 bytes
Done streaming.
Device closed
UVC exited
An extracted frame then is just a screen capture.
-
Since mine is 2.3.0 I spun up a Ubuntu 14.04.3 VM (First time every trying my hand at compiling anything) and got as far at the line listed above before I for the following error:
orby@ubuntu:~/libuvc/build$ /usr/bin/cc -o flir ../src/flir.c -luvc
../src/flir.c: In function ‘cb’:
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘__time_t’ [-Wformat=]
printf("got frame %d:%d, %d bytes\n", tv.tv_sec, tv.tv_usec, frame->data_bytes);
^
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘__suseconds_t’ [-Wformat=]
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]
Everything seemed to go word for word as outlined in tomas123 detailed instructions till that point.
I have no issues saying that I am in way over my head so unless anyone has any suggestions I don't think I can test further.
I would love to get a libuvc with this patch with this working on windows but my limited looking into it seems to indicate that the USB driver wrangling situation to accomplish this would be even more complex.
This is only a compiler warning, not an error. I got the excact same warning.
Simple use the successful compiled flir binary.
-
I don't have my E4 here right now. I'll investigate. Sorry for wasting everyone's time. (It definitely worked for me, as you can see in the youtube video)
-
Sorry, but I'm don't understand the "bFormatIndex - 1" hack.
Can you give us some more informations about the patch?
Which stream from the lsusb tree (see above) we are grabbing?
Thanks for help!
diff --git a/src/stream.c b/src/stream.c
index 571be16..aabb6ff 100644
--- a/src/stream.c
+++ b/src/stream.c
@@ -183,6 +183,7 @@ uvc_error_t uvc_query_stream_ctrl(
/* prepare for a SET transfer */
if (req == UVC_SET_CUR) {
SHORT_TO_SW(ctrl->bmHint, buf);
+ ctrl->bFormatIndex = 2;
buf[2] = ctrl->bFormatIndex;
buf[3] = ctrl->bFrameIndex;
INT_TO_DW(ctrl->dwFrameInterval, buf + 4);
@@ -222,6 +223,7 @@ uvc_error_t uvc_query_stream_ctrl(
if (req != UVC_SET_CUR) {
ctrl->bmHint = SW_TO_SHORT(buf);
ctrl->bFormatIndex = buf[2];
+ ctrl->bFormatIndex = 1;
ctrl->bFrameIndex = buf[3];
ctrl->dwFrameInterval = DW_TO_INT(buf + 4);
ctrl->wKeyFrameRate = SW_TO_SHORT(buf + 8);
@@ -828,7 +830,7 @@ uvc_error_t uvc_stream_start(
strmh->pts = 0;
strmh->last_scr = 0;
- frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex, ctrl->bFrameIndex);
+ frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex - 1, ctrl->bFrameIndex);
if (!frame_desc) {
ret = UVC_ERROR_INVALID_PARAM;
goto fail;
@@ -1037,7 +1039,7 @@ void _uvc_populate_frame(uvc_stream_handle_t *strmh) {
* is going to be reopen_on_change anyway
*/
- frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex,
+ frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex - 1,
strmh->cur_ctrl.bFrameIndex);
frame->frame_format = strmh->frame_format;
-
I think, the RAW Video of E40 is
bFormatIndex= 2
dwMaxVideoFrameBufferSize = 157440
but the flir.c log:
bFormatIndex: 1
bFrameIndex: 1
dwMaxVideoFrameSize: 157440
...very magic
compress output from lsusb -v
E40
VideoStreaming Interface Descriptor:
bFormatIndex 1
bBitsPerPixel 16
wWidth 320
wHeight 240
dwMaxVideoFrameBufferSize 153600 // = 320x240x2 Byte ( Screen Shot Video)
VideoStreaming Interface Descriptor:
bFormatIndex 2
bBitsPerPixel 16
wWidth 320
wHeight 246 // 6 extra lines for the FFF Header (see exiftool output above)
dwMaxVideoFrameBufferSize 157440 // 157440 is same size as the output from the RAW Video grab tool
E4
VideoStreaming Interface Descriptor:
bFormatIndex 1
bBitsPerPixel 16
wWidth 320
wHeight 240
dwMaxVideoFrameBufferSize 153600 // = 320x240x2 Byte ( Screen Shot Video)
-
This is only a compiler warning, not an error. I got the excact same warning.
Simple use the successful compiled flir binary.
I will spin that VM back up and test it out when I get off work later today then.
I don't have my E4 here right now. I'll investigate. Sorry for wasting everyone's time. (It definitely worked for me, as you can see in the youtube video)
I don't think anyone thinks you have wasted their time. (I don't) There could be different configurations or variables that could be at play here.
What USB mode is your camera in when capturing? (just UVC or one of the mixed modes?)
-
What USB mode is your camera in when capturing? (just UVC or one of the mixed modes?)
I checked all UVC variants without success.
-
Just took it for a little spin but getting some odd results:
orby@ubuntu:~$ sudo libuvc/build/flir e4-1.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439507727:121939, 153600 bytes
got frame 1439507727:122047, 153600 bytes
orby@ubuntu:~$ split -b 153600 e4-1.raw
orby@ubuntu:~$ convert -size 320x240 yuv:xaa -separate 4.png
convert.im6: unexpected end-of-file `xaa': No such file or directory @ error/yuv.c/ReadYUVImage/428.
It seems like it may be the raw feed at least for me.
Attached are the 2 png's it spat out and the zip has the raw and a xaa file it in it.
-
Meh. Okay.
I'm on 2.3, so maybe it was added later. Could someone with a Flir Ex with 2.3 firmware give it a try? I wouldn't know what else it could be.
Can you share your patched stream.c ?
-
@OrBy
I am getting same error and same result (Ubuntu 11.10 on a VM). But the png produced is not radiometric. Yours does not seem to be it either, it does not contain radiometric metadata.
-
@Orby
your png-image is radiometric with swapped bytes
see my tutorial here: https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398)
>identify 4-0.png
4-0.png PNG 320x240 320x240+0+0 16-bit sRGB 67.3KB 0.000u 0:00.001
>convert -define png:swap-bytes=on 4-0.png -auto-level raw.png
better use the splitted part
convert -depth 16 -size 320x240 -auto-level gray:xaa raw.png
@Orby + @Bud
You have both the firmware 2.3.0.
Find the difference between the configuration ;)
check "lsusb -v"
our logs from flir.c are 100% identical :-[
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
A interesting difference to my E40 stream:
The E4 RAW stream contains no FFF Header:
Image Size: 320x240x2 Byte = frame size = 153600 Byte
The E40 RAW Stream contains full FFF Imgages
frame size 157440 byte
see sample in my post https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688)
-
@Orby
your png-image is radiometric with swapped bytes
see my tutorial here: https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398)
>identify 4-0.png
4-0.png PNG 320x240 320x240+0+0 16-bit sRGB 67.3KB 0.000u 0:00.001
>convert -define png:swap-bytes=on 4-0.png -auto-level raw.png
better use the splitted part
convert -depth 16 -size 320x240 -auto-level gray:xaa raw.png
@Orby + @Bud
You have both the firmware 2.3.0.
Find the difference between the configuration ;)
check "lsusb -v"
our logs from flir.c are 100% identical :-[
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
A interesting difference to my E40 stream:
The E4 RAW stream contains no FFF Header:
Image Size: 320x240x2 Byte = frame size = 153600 Byte
The E40 RAW Stream contains full FFF Imgages
frame size 157440 byte
see sample in my post https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688)
Yes I kinda thought that it had the look like it was backwards and that it needed just a bit more processing to look right. I ran out of time to play last night but I was going to try messing with it a little more when time permitted.
It's interesting how the E4 is missing the header info - guess they cant make it too easy!
I will post up my lsusb -v log when I get some time back on my PC.
-
The only missing header infos are the time stamps.
All other Flir values to calculate the real temperature of a pixel are (calibrated) constants.
You can get them from every radiometric jpg of the same (!) camera.
I hope tmbinc give us some explanations to the bFormatIndex-hack
and some code snippets for our own Fuchsjagd ;)
-
By non-radiometric i mean no temperature scaling information is in the file. I do not have access to my computer at the moment but if you load the image in Flir Tools i think temperature measurements will be incorrect and all the way between -40 +300 or something. To get it right i think the image levels need to be scaled against a known reference, same as you do when creating panoramas after stitching step.
Anyway, try loading it in Flir Tools to see if it can be correctly measured.
-
@Bud
Currently you have not radiometric raw values in your stream. Therefore Flir Tools can't read anything.
You need a RAW stream, like Orby E4.
Then you can inject a extracted 16 bit raw png in a radiometric jpg (from the same E4!) and the temperature measurement of Flir Tools works fine
simple use my (panorama) script splitjpg.php for injecting
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715)
// no -auto-level ;)
$ convert -depth 16 -size 320x240 gray:xaa raw.png
$ php splitjpg.php -i IR_0xxx.jpg -r raw.png -o sample
Load the sample.jpg in Flir Tools
-
Yes i have already tried that with my non- radiometric streamed images and if i use your splitjpg script and a reference radiometric jpg, i get a proper thermal image in Flir Tools but temperature readings are still somewhat off, probably because the reference jpg was not from the same scene. I will try tonight without the scaling step.
Having said that, i begin to believe that may not be difference between the E4 hacked radiometric streamed images vs screen capture streamed ones, providing i turn measurements and everything else off on the camera, and also turn MSX off so the screen only displays the thermal image. I can do this on my doctored E4 2.3.0.
I will still try to get the camera stream same way as OrBy's so i can post processes images received using the two methods and compare the end result.
-
I will still try to get the camera stream same way as OrBy's so i can post processes images received using the two methods and compare the end result.
I can grab some more out of each stream and put them up for comparison later today if needed/wanted.
-
Having said that, i begin to believe that may not be difference between the E4 hacked radiometric streamed images vs screen capture streamed ones
...except perhaps swapped bytes in radiometric stream.
-
i begin to believe that may not be difference between the E4 hacked radiometric streamed images vs screen capture streamed ones, providing i turn measurements and everything else off on the camera, and also turn MSX off so the screen only displays the thermal image.
Stream of the monitor view is nice but a raw stream is really cool (surveillance camera, measurements)
See the great video from tmbinc.
I hope that tmbinc give us the code for converting a raw stream. I see a nice application for my raspberry pi.
Don't forget that Flir Tools can grab a raw stream from E40 (switch from video to signal)
(https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/?action=dlattach;attach=78866;image)
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg377140/#msg377140 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg377140/#msg377140)
the video stream of E4 grab with the free FLIR IR Camera Player
(https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/?action=dlattach;attach=79308;image)
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg378926/#msg378926 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg378926/#msg378926)
-
@OrBy
Your upgraded the E4 from 1.18.7 to 2.3.0.
Interesting!
Maybe the (hidden) raw stream of your E4 is activated by a switch in your config file.
Just wanted to give a quick :-+ to janekivi and everyone else that's been working on 2.3.0.
I decided to upgrade from 1.18.7 to 2.3.0 today on my 1.0 E4.
I just flashed the 2.3.0 FIF and rebooted and it retained my "upgraded" status. Installed the 2014 menu FIF and had to add a little tweak to my e8.cfg and recrc it to get the manual adjustments working.
So far I really like the clean no measurements screen! Makes for "truer" video capture in UVC mode and allows me to "frame" my shots better! Also the thermal blending is a nice addition.
Cant wait till I have some more time to play with the advanced ed. :)
-
Yes I did upgrade mine to 2.3.0
Here is my lsusb -v dump:
(I hope it's the right section required)
Bus 001 Device 002: ID 09cb:1007
Couldn't open device, some information will be missing
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x09cb
idProduct 0x1007
bcdDevice 0.00
iManufacturer 7
iProduct 8
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 241
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 10
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 2
VideoControl Interface Descriptor:
bLength 13
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 85
dwClockFrequency 0.001000MHz
bInCollection 1
baInterfaceNr( 0) 1
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 0
wObjectiveFocalLengthMax 0
wOcularFocalLength 0
bControlSize 3
bmControls 0x00020260
Focus (Absolute)
Focus (Relative)
Zoom (Absolute)
Focus, Auto
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 3
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 6
iTerminal 0
VideoControl Interface Descriptor:
bLength 7
bDescriptorType 36
bDescriptorSubtype 4 (SELECTOR_UNIT)
bUnitID 4
bNrInPins 1
baSource( 0) 1
iSelector 0
VideoControl Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 5
bSourceID 4
wMaxMultiplier 0
bControlSize 2
bmControls 0x00000217
Brightness
Contrast
Hue
Sharpness
Gain
iProcessing 0
bmVideoStandards 0x1b
None
NTSC - 525/60
SECAM - 625/50
NTSC - 625/50
VideoControl Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 6
guidExtensionCode {d41f59fa-5094-463a-b3bb-e7858a831fa3}
bNumControl 4
bNrPins 1
baSourceID( 0) 5
bControlSize 2
bmControls( 0) 0x0f
bmControls( 1) 0x00
iExtension 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 16
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 1
wTotalLength 79
bEndPointAddress 130
bmInfo 0
bTerminalLink 3
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 27
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 1
bNumFrameDescriptors 1
guidFormat {59555932-0000-1000-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 38
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x03
Still image supported
Fixed frame-rate
wWidth 320
wHeight 240
dwMinBitRate 912384
dwMaxBitRate 912384
dwMaxVideoFrameBufferSize 153600
dwDefaultFrameInterval 2666664
bFrameIntervalType 3
dwFrameInterval( 0) 666666
dwFrameInterval( 1) 1333332
dwFrameInterval( 2) 2666664
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x04 EP 4 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 0
And here are the contents of my e8.cfg:
.caps entry
.caps.config entry
.caps.config.name text "app E8"
.caps.config.revision text "1.0"
.caps.config.image entry
.caps.config.image.framegrab entry
.caps.config.image.framegrab.fusion entry
.caps.config.image.framegrab.fusion.enabled bool true
.caps.config.image.framegrab.fusion.pip entry
.caps.config.image.framegrab.fusion.pip.enabled bool true
.caps.config.image.framegrab.fusion.hcf entry
.caps.config.image.framegrab.fusion.hcf.enabled bool true
.caps.config.image.services entry
.caps.config.image.services.store entry
.caps.config.image.services.store.enabled bool true
.caps.config.image.services.store.radiometric entry
.caps.config.image.services.store.radiometric.enabled bool true
.caps.config.image.services.store.incompatible entry
.caps.config.image.services.store.incompatible.enabled bool false
.caps.config.image.services.store.incompatible.level int32 0
.caps.config.image.settings entry
.caps.config.image.settings.enabled bool true
.caps.config.image.settings.IRwidth int32 320
.caps.config.image.settings.IRheight int32 240
.caps.config.image.sysimg entry
.caps.config.image.sysimg.alarms entry
.caps.config.image.sysimg.alarms.enabled bool true
.caps.config.image.sysimg.alarms.measfunc entry
.caps.config.image.sysimg.alarms.measfunc.enabled bool true
.caps.config.image.sysimg.alarms.measfunc.maxCount int32 3
.caps.config.image.sysimg.alarms.humidity entry
.caps.config.image.sysimg.alarms.humidity.enabled bool true
.caps.config.image.sysimg.alarms.humidity.maxCount int32 1
.caps.config.image.sysimg.alarms.insulation entry
.caps.config.image.sysimg.alarms.insulation.enabled bool true
.caps.config.image.sysimg.alarms.insulation.maxCount int32 1
.caps.config.image.sysimg.irMarkers entry
.caps.config.image.sysimg.irMarkers.enabled bool true
.caps.config.image.sysimg.irMarkers.spot entry
.caps.config.image.sysimg.irMarkers.spot.enabled bool false
.caps.config.image.sysimg.irMarkers.spot.maxCount int32 0
.caps.config.image.sysimg.irMarkers.arrow entry
.caps.config.image.sysimg.irMarkers.arrow.enabled bool true
.caps.config.image.sysimg.irMarkers.arrow.maxCount int32 4
.caps.config.image.sysimg.irMarkers.box entry
.caps.config.image.sysimg.irMarkers.box.enabled bool false
.caps.config.image.sysimg.irMarkers.box.maxCount int32 0
.caps.config.image.sysimg.measureFuncs entry
.caps.config.image.sysimg.measureFuncs.enabled bool true
.caps.config.image.sysimg.measureFuncs.diff entry
.caps.config.image.sysimg.measureFuncs.diff.enabled bool true
.caps.config.image.sysimg.measureFuncs.diff.maxCount int32 1
.caps.config.image.sysimg.measureFuncs.diff.calcMask int32 65526
.caps.config.image.sysimg.measureFuncs.isotherm entry
.caps.config.image.sysimg.measureFuncs.isotherm.enabled bool true
.caps.config.image.sysimg.measureFuncs.isotherm.calcMask int32 20
.caps.config.image.sysimg.measureFuncs.isotherm.dual bool false
.caps.config.image.sysimg.measureFuncs.isotherm.fixScale bool false
.caps.config.image.sysimg.measureFuncs.isotherm.interval bool true
.caps.config.image.sysimg.measureFuncs.isotherm.invInterval bool false
.caps.config.image.sysimg.measureFuncs.isotherm.maxCount int32 1
.caps.config.image.sysimg.measureFuncs.mbox entry
.caps.config.image.sysimg.measureFuncs.mbox.enabled bool true
.caps.config.image.sysimg.measureFuncs.mbox.calcMask int32 1924
.caps.config.image.sysimg.measureFuncs.mbox.maxCount int32 5
.caps.config.image.sysimg.measureFuncs.mcircle entry
.caps.config.image.sysimg.measureFuncs.mcircle.enabled bool false
.caps.config.image.sysimg.measureFuncs.mcircle.calcMask int32 1924
.caps.config.image.sysimg.measureFuncs.mcircle.maxCount int32 0
.caps.config.image.sysimg.measureFuncs.mline entry
.caps.config.image.sysimg.measureFuncs.mline.enabled bool false
.caps.config.image.sysimg.measureFuncs.mline.calcMask int32 1924
.caps.config.image.sysimg.measureFuncs.mline.maxCount int32 0
.caps.config.image.sysimg.measureFuncs.reftemp entry
.caps.config.image.sysimg.measureFuncs.reftemp.enabled bool true
.caps.config.image.sysimg.measureFuncs.reftemp.calcMask int32 1924
.caps.config.image.sysimg.measureFuncs.reftemp.maxCount int32 1
.caps.config.image.sysimg.measureFuncs.script entry
.caps.config.image.sysimg.measureFuncs.script.enabled false
.caps.config.image.sysimg.measureFuncs.script.maxCount int32 0
.caps.config.image.sysimg.measureFuncs.spot entry
.caps.config.image.sysimg.measureFuncs.spot.enabled bool true
.caps.config.image.sysimg.measureFuncs.spot.calcMask int32 514
.caps.config.image.sysimg.measureFuncs.spot.maxCount int32 5
.caps.config.image.sysimg.visualMarkers entry
.caps.config.image.sysimg.visualMarkers.enabled bool true
.caps.config.image.sysimg.visualMarkers.spot entry
.caps.config.image.sysimg.visualMarkers.spot.enabled bool false
.caps.config.image.sysimg.visualMarkers.spot.maxCount int32 0
.caps.config.image.sysimg.visualMarkers.arrow entry
.caps.config.image.sysimg.visualMarkers.arrow.enabled bool true
.caps.config.image.sysimg.visualMarkers.arrow.maxCount int32 4
.caps.config.image.sysimg.visualMarkers.box entry
.caps.config.image.sysimg.visualMarkers.box.enabled bool false
.caps.config.image.sysimg.visualMarkers.box.maxCount int32 0
.caps.config.image.contadj entry
.caps.config.image.contadj.minSpanFactor entry
.caps.config.image.contadj.minSpanFactor.enabled bool true
.caps.config.image.contadj.minSpanFactor.factorAuto double 2.0
.caps.config.image.contadj.minSpanFactor.factorManual double 2.0
.caps.config.image.targetNoise entry
.caps.config.image.targetNoise.enabled bool false
.caps.config.image.targetNoise.targetNoiseMk int32 0
.caps.config.image.zoom entry
.caps.config.image.zoom.enabled bool true
.caps.config.image.zoom.maxFactor double 8
.caps.config.system entry
.caps.config.system.focus entry
.caps.config.system.focus.laser entry
.caps.config.system.focus.laser.updateFocus entry
.caps.config.system.focus.laser.updateFocus.enabled bool true
.caps.config.ui.image entry
.caps.config.ui.image.adjust entry
.caps.config.ui.image.adjust.enabled bool true
.caps.config.ui.image.adjust.manual bool true
.caps.config.ui entry
.caps.config.ui.fusion entry
.caps.config.ui.fusion.PIP entry
.caps.config.ui.fusion.PIP.enabled bool true
.caps.hw entry
.caps.hw.sdcard entry
.caps.hw.sdcard.enabled bool false
-
I followed tomas123's steps to re-compile libuvc library and now it works for me too (thanks tomas123 for posting the compilation log, I think I can learn Chinese faster than Linux ).
Attached is a frame from the video stream, loaded in FLIR Tools and some measurements enabled. All makes sense and matches what is measured by the camera. For the record, the camera is a native E4 fw 2.3.0 doctored for resolution and enhanced menu.
So yes the patch works, it is possible to stream radiometric video from E4 (oh well, for now just in a file and using Linux), thanks user tmbinc :-+
-
Great!
Please describe all steps, how you can load a frame in Flir Tools and attach a splitted single raw frame (as .zip)
I think, I must make a firmware update from 1.19 to 2.3.0
don't touch a running system :palm:
-
All as previously described:
Pre-requisits for image processing software:
- imagemagick
- php
- splitjpg.php script
source: https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715)
- a sample image from the same E4 camera
- FLIR Tools
workflow:
- capture a RAW video
- split to individual frames
split -b 153600 test1.raw
- convert the required frame (in this example the first frame) to a RAW png
convert -depth 16 -size 320x240 gray:xaa raw1.png
convert the RAW png to radiometric jpg (for a template use a sample jpg image, e.g. FLIR0123.jpg, from the same camera:
php splitjpg.php -i FLIR0123.jpg -r raw1.png -o test1.jpg
note: after this step the resulting test1.jpg preview will appear the same as the template image, do not worry: the actual RAW data is hidden behind the image.
Now load this JPG into FLIR Tools to rebuild the preview image, place temperature measurements, apply a desired palette and resave.
I do video capture, split and convert on a Ubuntu VM, everything else on a Windows 7.
-
Found a great little utility called ImageJ
http://imagej.nih.gov/ij/ (http://imagej.nih.gov/ij/)
Holy sh!t, have not seen it before, it is a jewel. For the purpose of this thread it can directly open raw files created by tmbinc's software and resave in a bunch of different formats including individual split frames. Also has a pile of built-in pallettes (Look Up Table they call it) and can colorize the video and even save the video in AVI format ready to be played ! :-+
A sample is attached - a radiometric stream capture from the E4, colorized using ImageJ.
test13.avi (http://wikisend.com/download/544416/test13.avi) <- click here
With this sort of post processing software radiometric streaming from E4 all of a sudden makes sense.
-
:-+ Great!!!
ImageJ read my E40 raw stream with little endian byte order and 3840 Byte Flir-FFF-Header between frames
@Bud
I linked your last two posts in my E4 link list:
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072)
see topic 15
Mike linked my post to the the head of the infinite thread
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/ (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/)
"Tomas123's links on post- processing and measurements using E4 images"
Please add in your second last post a link to my splitjpg.php source
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715)
@tmbinc
I hope you can give us your Fuchsjagd code snippets ;)
-
@Bud
Please add in your second last post a link to my splitjpg.php source
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715)
Done.
I wish ImageJ could combine multiple frames in a single image for superresolution, but have not figured out yet if it can.
A newer version is here:
http://fiji.sc/Downloads (http://fiji.sc/Downloads)
Better palettes, +tons of image manipulation menus, a holy grail of image processing :-+
-
a little bit off topic: superresolution with RAW thermal video
I think, you know my posts to superresolution.
(see my link list #17 -> https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg363688/#msg363688 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg363688/#msg363688))
I have had good experiences with:
- Registax (sharp with wavelets)
- AviStack (sharp with wavelets)
- Photoshop CS3 -> automate -> merge to HDR (automat. align images)
-> quality like Registax/AviStax (CS3 works better as CS5)
... do not expect too much and resize your images/video before stacking :)
-
I wish ImageJ could combine multiple frames in a single image for superresolution, but have not figured out yet if it can.
A newer version is here:
http://fiji.sc/Downloads (http://fiji.sc/Downloads)
Better palettes, +tons of image manipulation menus, a holy grail of image processing :-+
OK I think there is a few ways to experiment with superresolution in ImageJ. For the purpose of this topic the workflow to create a combined image from a sequence of video frames may look as follows:
- capture a few seconds of a raw video as already described above
- open the video in ImageJ (File->Import->RAW)
- if necessary, delete bad or unwanted frames (use scroll bar to select a frame, then "Image->Stacks->Delete Slice"
- combine the frames (Image->Stacks->Z Project)
-
Sorry for not keeping up with this thread.
The idea is that even though the E4 doesn't announce the bFormatIndex=2 RAW stream via the USB descriptor, it still allows to select it. The code that restricts the raw mode based on the camera type does cripple the descriptor, but it doesn't prevent you from still selecting it.
The libuvc patch is what I've posted (i just double-checked against what's still on my raspi2), I applied it on top of https://github.com/ktossell/libuvc/commits/6ff39f17e3f3a26f437bc6b3c09cc6c289db7891 . It does this in a very lame and stupid way by switching it back and forth, to make the library take the original stream descriptor, but send the right UVC command.
I attached the "fuchsjags" source, but it's really not great - it calculates min and max, scales that to 0..255, inserts the texts and the pointer, converts it to ARGB for the framebuffer, and also dumps the raw and converted values into files. I have a different script that used avconv to encode it to MP4 ("avconv -f rawvideo -pix_fmt gray -s:v 320x240 -r 9 -i /mnt/fox/r1_proc.raw -c:v libx264 /mnt/fox/output.mp4").
-
Thank You :-+
My next project name is Marderjagd ;)
-
I think, I must make a firmware update from 1.19.8 to 2.3.0
don't touch a running system :palm:
successful upgraded the E4 from 1.19.8 to 2.3.0 with Flir Tools
The resolution hack survived the update (nothing else to do).
the RAW stream grabbing now works fine :)
-
Great, so RAW streaming did not work in 1.19.8 ...
-
@tomas123
I wonder if you then could take a challenge and compile the other sample from libuvc which supposedly creates a window and displays the streamed video (source modified for 320x240x15 is attached). It requires a library called opencv to be installed. I tried but as I said my Chinese is much better than my Linux, no matter how hard I tried I got the dreaded opencv "Rebuild the library with Windows, GTK+ 2.x or Carbon support" error. I am sure I am missing something obvious but I just do not know what.
If this code works and if combined with tmbinc's code this may become a usable program for E4 that displays, saves and processes E4 radiometric video.
-
By the way, my test tool already shows the output on a framebuffer. It's not great and was just meant for debugging... Should be easy though to add a LUT and some color scheme, as well as "calibrated" temperature. I'm not sure though that the radiometric metadata is available.
I also tried doing a 2D-FFT to assist in fine-tuning focus. But the results haven't been that useful. Does someone have a better idea how to do this? (I.e. how to evaluate focus based on image processing)
-
l upgraded the E4 from 1.19.8 to 2.3.0 with Flir Tools
The resolution hack survived the update (nothing else to do).
Temped to do the same a long time ago, didn´t have time yet:
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg607838/#msg607838 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg607838/#msg607838)
and do the janekivi menu mod. Hope the raw capturing will work regardless the menu mod.
-
I also tried doing a 2D-FFT to assist in fine-tuning focus. But the results haven't been that useful. Does someone have a better idea how to do this? (I.e. how to evaluate focus based on image processing)
The opencv library is a tool for computer vision, so I'd think it may have something in it for what you need.
-
@tomas123
I wonder if you then could take a challenge and compile the other sample from libuvc which supposedly creates a window and displays the streamed video (source modified for 320x240x15 is attached). It requires a library called opencv to be installed. I tried but as I said my Chinese is much better than my Linux, no matter how hard I tried I got the dreaded opencv "Rebuild the library with Windows, GTK+ 2.x or Carbon support" error. I am sure I am missing something obvious but I just do not know what.
If this code works and if combined with tmbinc's code this may become a usable program for E4 that displays, saves and processes E4 radiometric video.
Hi Bud,
your code snippet works fine, no changes are necessary :)
edit 30.08.15: found the source https://github.com/ktossell/libuvc/blob/master/src/test.c (https://github.com/ktossell/libuvc/blob/master/src/test.c) |O
the steps:
(1) check version
$ lsb_release -a
Description: Ubuntu 14.04.1 LTS
(2) compile opencv
use this tutorial
http://www.bogotobogo.com/OpenCV/opencv_3_tutorial_ubuntu14_install_cmake.php (http://www.bogotobogo.com/OpenCV/opencv_3_tutorial_ubuntu14_install_cmake.php)
$ mkdir OpenCV
$ cd OpenCV
$ git clone [url]https://github.com/Itseez/opencv.git[/url]
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON ..
$ make
$ sudo make install
$ sudo ldconfig
(3) compile your sample
// ignore warnings
// note http://stackoverflow.com/questions/9614872/opencv-application-not-library-linking-errors (http://stackoverflow.com/questions/9614872/opencv-application-not-library-linking-errors)
$ /usr/bin/cc -o mytest mytest.c -luvc `pkg-config --cflags --libs opencv`
mytest.c: In function ‘cb’:
mytest.c:50:66: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);
^
mytest.c:50:3: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘size_t’ [-Wformat=]
printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);
(4) run your code
$ sudo ./mytest
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming for 10 seconds...
set_ae_mode: Pipe (-9)
callback! length = 153600, ptr = -886544208
set_exp_abs: Pipe (-9)
init done
opengl support available
callback! length = 157690, ptr = -886544208
callback! length = 157690, ptr = -886544208
callback! length = 157690, ptr = -886544208
callback! length = 157690, ptr = -886544208
and you get a window with a nice false color video ;)
-
[Apologize if this is common knowledge and I just missed the memo. As everyone knows, the main thread is _huge_ and there was some talk about this, but I didn't find the final verdict of whether it's possible or not.]
I was trying to catch a fox that ate our pet rabbits from our garden on camera (people say they return to the scene of the crime). I wasn't successful (yet!), but I learned to hate the FLIR UVC implementation since it's randomly stops working during the night.
In an attempt to rectify the situation, I looked at the UVC implementation on the camera, and figured out that there's a dormant function to stream 9Hz raw radiometric data instead of just capturing the screen. With Windows and the regular UVC drivers, the best I could capture was the screen with 3.3Hz (for whatever reason; 15Hz should be possible) at (effectively) 8-bit over a pre-selected temp range.
The catch is that the UVC driver on the Flir explicitely disables radiometric streaming for the Z3 (=Flir Ex). It enables it for other cameras, though - i think the Flir Ex0 suports it out-of-the-box. The UVC stack is implemented in usbfnvideo.dll, which unfortunately is part of nk.bin, so it can't be easily patched (at least, I can't with my limited WinCE knowledge and my unwillingness to brick the E4).
The great thing is that FLIR sucks at implementing device restrictions; they have a dedicated grabber thread for the Flir Ex-series, and it _does_ indeed implement grabbing the raw data (via FVD1: instead of capturing the screen), and they _let_ you select capturing that (via setting bFormatIndex in the stream control), but they don't announce the ability for that in the UVC descriptors, so any "well-behaving" UVC client will not allow you to select it.
I've hacked libuvc (which is based on libusb) and - tada, it just worked. I patched libuvc pretty crudely so I'm sure there's a better way to do it. I've attached the patch and the crude sample (which just dumps out raw data into a file).
Let me know if you find this useful.
Cool, but could you please attach the compiled file instead of the C source code and patch? I want a binary to use directly. I don't have much experience with C programming, but I really would like to get to use this (I'm considering getting a Flir E4, part of the Flir Ex series, but whether or not I get it will depend on what features I can access, via EASY to use hacks, and having to compile a hack is NOT easy). So can you pleas compile this, and then post the compiled binary file here?
-
this a generic linux source code and you must compile it for your CPU (ARM / Intel etc) and your Linux version
tmbinc compiled the code on a raspberry pi 2 (quad-core ARM7) ;)
you need a firmware 2.3.0
read my step by step tutorial for ubuntu x64
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604[/url
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604)
-
@Bud
just for fun I changed the sample code from libuvc for 16 Bit RAW and appended the auto-level code from tmbinc
https://github.com/ktossell/libuvc/blob/master/src/test.c (https://github.com/ktossell/libuvc/blob/master/src/test.c)
compile the code like above described
# /usr/bin/cc -o mytest mytest.c -luvc `pkg-config --cflags --libs opencv`
... some warnings
# sudo ./mytest
UVC initialized
Device found
Device opened
...
the source:
/*********************************************************************
* Software License Agreement (BSD License)
*
* Copyright (C) 2010-2012 Ken Tossell
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials provided
* with the distribution.
* * Neither the name of the author nor other contributors may be
* used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
* ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*********************************************************************/
#include <stdio.h>
#include <opencv/highgui.h>
#include "libuvc/libuvc.h"
struct cb_context {
FILE *out;
struct timeval tv_start;
int frames;
};
void cb(uvc_frame_t *frame, void *ptr) {
uvc_frame_t *rgb;
uvc_error_t ret;
IplImage* cvImg;
printf("callback! length = %u, ptr = %d\n", frame->data_bytes, (int) ptr);
rgb = uvc_allocate_frame(frame->width * frame->height * 3);
if (!rgb) {
printf("unable to allocate rgb frame!");
return;
}
ret = uvc_any2rgb(frame, rgb);
if (ret) {
uvc_perror(ret, "uvc_any2rgb");
uvc_free_frame(rgb);
return;
}
cvImg = cvCreateImageHeader(
cvSize(rgb->width, rgb->height),
IPL_DEPTH_8U,
3);
unsigned short *pix = frame->data;
unsigned char *fb_proc;
fb_proc = malloc(320 * 240);
int i, x, y;
// calc min, max, rms
int min = 0x10000, max = 0;
for (i = 0; i < 320 * 240; ++i) {
if (pix[i] < min) min = pix[i];
if (pix[i] > max) { max = pix[i]; }
}
int scale = 0x10000 / (max - min);
// stretch level and shift 16 to 8 Bit
for (y = 0; y < 240; ++y)
{
for (x = 0; x < 320; ++x) {
int v = (pix[y * 320 + x] - min) * scale >> 8;
fb_proc[y * 320 + x] = v;
}
}
CvSize image_size;
image_size.height = 240;
image_size.width = 320;
int channels = 1;
IplImage *image = cvCreateImageHeader(image_size, IPL_DEPTH_8U, channels);
cvSetData(image, fb_proc, image->widthStep);
cvNamedWindow("Test", CV_WINDOW_AUTOSIZE);
cvShowImage("Test", image);
cvWaitKey(10);
cvReleaseImageHeader(&cvImg);
uvc_free_frame(rgb);
}
int main(int argc, char **argv) {
uvc_context_t *ctx;
uvc_device_t *dev;
uvc_device_handle_t *devh;
uvc_stream_ctrl_t ctrl;
uvc_error_t res;
struct cb_context cb_ctx = {0};
/* Initialize a UVC service context. Libuvc will set up its own libusb
* context. Replace NULL with a libusb_context pointer to run libuvc
* from an existing libusb context. */
res = uvc_init(&ctx, NULL);
if (res < 0) {
uvc_perror(res, "uvc_init");
return res;
}
puts("UVC initialized");
res = uvc_find_device(
ctx, &dev,
0, 0, NULL);
if (res < 0) {
uvc_perror(res, "uvc_find_device");
} else {
puts("Device found");
res = uvc_open(dev, &devh);
if (res < 0) {
uvc_perror(res, "uvc_open");
} else {
puts("Device opened");
uvc_print_diag(devh, stderr);
/* Try to negotiate a 640x480 30 fps YUYV stream profile */
res = uvc_get_stream_ctrl_format_size(
devh, &ctrl, UVC_FRAME_FORMAT_YUYV, 320, 240, 15 /* YUV 422, aka YUV 4:2:2. try _COMPRESSED */
);
/* Print out the result */
uvc_print_stream_ctrl(&ctrl, stderr);
if (res < 0) {
uvc_perror(res, "get_mode");/* device doesn't provide a matching stream */
} else {
/* Start the video stream. The library will call user function cb:
* cb(frame, (void*) 12345)
*/
res = uvc_start_streaming(devh, &ctrl, cb, &cb_ctx, 0);
if (res < 0) {
uvc_perror(res, "start_streaming"); /* unable to start stream */
} else {
puts("Streaming for 10 seconds...");
uvc_error_t resAEMODE = uvc_set_ae_mode(devh, 1);
uvc_perror(resAEMODE, "set_ae_mode");
int i;
for (i = 1; i <= 10; i++) {
/* uvc_error_t resPT = uvc_set_pantilt_abs(devh, i * 20 * 3600, 0); */
/* uvc_perror(resPT, "set_pt_abs"); */
uvc_error_t resEXP = uvc_set_exposure_abs(devh, 20 + i * 5);
uvc_perror(resEXP, "set_exp_abs");
sleep(1);
}
sleep(20);
uvc_stop_streaming(devh);
puts("Done streaming.");
}
}
uvc_close(devh);
puts("Device closed");
}
uvc_unref_device(dev);
}
uvc_exit(ctx);
puts("UVC exited");
return 0;
}
RAW live stream:
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=168545;image)
@ tmbinc
What do you mean with this code snippset from fuchsjagd?
It's like a 20x20 pixel watermark.
You can see it in your video at the bottom on the left side.
unsigned short *pix = frame->data;
int x, y;
for (y = 160; y < 180; ++y) {
for (x = 60; x < 80; ++x) {
pix[y * 320 + x] = pix[y * 320 + 60];
}
}
Thank you for the fuchsjagd code :-+
I changed it for Ubuntu x64, only the frame buffer part it's a little bit tricky.
-
Ya baby, now we are talking! Thanks tomas123!
Though I had to update to Ubuntu 14.whatever, otherwise the OpenCV library did not compile. That is why I avoid Linux, the developers have no clue as to what flavor of Linux and what version their product works with. Guessing game every time and you better have all Developers packages in the Universe installed to mitigate the pain.
Anyway, your latest code worked and I took a liberty to tailor it a little. I made use of the original buffer initialization from the original test file, as there seemed to be some duplication in last version. I also commented out the frame data length message that was flooding the screen - I think we got the code working to the point we do not need to monitor that anymore. And a nice thing: I replaced CV_WINDOW_AUTOSIZE with CV_WINDOW_NORMAL, and now the window is resizable.
(http://i.imgur.com/KgRZS2b.png)
I also returned to waiting for a key press to end streaming vs a timeout.
BTW, why is this code only falls thru if ENTER key is pressed but not any other key ?
// stream until key is pressed.
char c;
read(0, &c, 1);
/* End the stream. Blocks until last callback is serviced */
uvc_stop_streaming(devh);
puts("Done streaming.");
So this actually is becoming a useful piece of software. You can make snapshots from the window (a sample attached). I think autoleveling may need to be improved, it did not always work well.
EDIT: attached the source
-
@tomas123
OK some more awkward code added to track mouse coordinates and display pixel value under the cursor using a slider control as a gauge meter
(http://i.imgur.com/UjUwmZs.png)
So this pretty much completes a Proof of Concept and now the actual pixel temperature can be calculated and displayed, you will find my comments in the code. I recall you were the one who reverse engineered FLIR temperature calculation? This at the moment is beyond my ability so pls see if you can update the code with it. Thks!
What else could be added is perhaps a center spot and box measures. I think this can be done with OpenCV.
For now you can position mouse cursor at the center of the image to emulate a center spot and then moving the camera will read temperature from it.
EDIT: just in case: I compile using the following command:
gcc -o flirfox2 flirfox2.c -luvc -lm `pkg-config --cflags --libs opencv`
-
@tomas123
And here is something interesting from me playing with window modes. When I used CV_WINDOW_OPENGL, moved the window on the screen, exploded to full screen then back, noticed something weird (selection is mine):
(http://i.imgur.com/JLovpLA.png)
For some reason the camera changed frame data length and it seemed to stayed consistent until I returned from full screen mode. What the heck the camera was transmitting and how the heck the code managed to display a proper image??? :-//
Did not have time to play more with that, may be after adding streaming data to file will see if can replicate it and look what is in the extra data.
-
I can confirm this additional bytes.
In my Ubuntu virtual machine I often have spontaneous glitches associated with changes of usb frame lenght.
With camera connected direct to host machine USB this glitches are rarely.
The additional bytes are without informations (most zeros)
I solved this glitches in the juchsjagd code by hardcoding the frame size with 320x240x2 Byte (crop the tail).
So this pretty much completes a Proof of Concept and now the actual pixel temperature can be calculated and displayed, you will find my comments in the code. I recall you were the one who reverse engineered FLIR temperature calculation? This at the moment is beyond my ability so pls see if you can update the code with it.
this is really hard :)
You can't grep a Flir temperature value from a 8 Bit grayscale video stream without additional informations (high/low value etc)
Therefore the best way is to enlarge the image size and imprint the temperature values (like max/min) outside the visible video.
the nice running fuchsjagd code from tmbinc contains all necessary code snippsets (font.h :-+ )
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344)
the mathematical equations for calculating RAW to temperature I posted here:
- sample with excel sheet: http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg27546.html#msg27546 (http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg27546.html#msg27546)
- sample with php code https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072)
As attachment I post my excel sheet with all formulas and some unpublished ;) infos from Flir
The required Flir parameters are calibrated constants for every camera. You get this values from every radiometric jpg with this exiftool command:
>exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* sample.jpg
Atmospheric Temperature : 20.0 C
Atmospheric Trans Alpha 1 : 0.006569
Atmospheric Trans Alpha 2 : 0.012620
Atmospheric Trans Beta 1 : -0.002276
Atmospheric Trans Beta 2 : -0.006670
Atmospheric Trans X : 1.900000
Planck R1 : 15314.136
Planck B : 1404.6
Planck F : 1
Planck O : -978
Planck R2 : 0.011565135
Object Distance : 1.00 m
Emissivity : 0.60
Reflected Apparent Temperature : 20.0 C
Relative Humidity : 50.0 %
-
What do you mean with this code snippset from fuchsjagd?
It's like a 20x20 pixel watermark.
You can see it in your video at the bottom on the left side.
unsigned short *pix = frame->data;
int x, y;
for (y = 160; y < 180; ++y) {
for (x = 60; x < 80; ++x) {
pix[y * 320 + x] = pix[y * 320 + 60];
}
}
Haha, I did forget about that code. Sorry. I have some dirt on my lens (or sensor) which I didn't cleanup yet (out of fear to break something). This created a spot that read a slightly larger temperature, which triggered the "Fuchs temperature threshold" ("Ftt").
The, erm, solution was to just copy the area next to it over the spot.
-
Hello guys,
i tried this also, but i cant get a stream with the first flir.c release.
I also got some other streaming descriptors than you, probably you can help me out?
Thank you in advance.
It is an FLIR E4 wit FW 2.30 and hacked for Resolution and Menu (Firmware was original at 2.30)
root@iJudge:/home/pantheron/libuvc# build/flir e4x.raw
UVC initialized
Device found
unsupported descriptor subtype: 3
unsupported descriptor subtype: 13
unsupported descriptor subtype: 3
unsupported descriptor subtype: 13
Device opened
DEVICE CONFIGURATION (05ac:8510/CC2CAV0DA1DN9KE0) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 5
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 160x120
bit rate: 24576000-147456000
max frame size: 38400
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(2)
capabilities: 00
size: 176x144
bit rate: 24576000-147456000
max frame size: 50688
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 24576000-147456000
max frame size: 153600
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(4)
capabilities: 00
size: 352x288
bit rate: 24576000-147456000
max frame size: 202752
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(5)
capabilities: 00
size: 640x480
bit rate: 24576000-147456000
max frame size: 614400
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(6)
capabilities: 00
size: 480x360
bit rate: 24576000-147456000
max frame size: 345600
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(7)
capabilities: 00
size: 1280x720
bit rate: 24576000-147456000
max frame size: 1843200
default interval: 1/10
interval[0]: 1/10
MJPEGFormat(2)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 2
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 960x544
bit rate: 24576000-147456000
max frame size: 1044480
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(2)
capabilities: 00
size: 1024x576
bit rate: 24576000-147456000
max frame size: 1179648
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
FrameDescriptor(3)
capabilities: 00
size: 1280x720
bit rate: 24576000-147456000
max frame size: 1843200
default interval: 1/29
interval[0]: 1/29
interval[1]: 1/25
interval[2]: 1/23
interval[3]: 1/14
END DEVICE CONFIGURATION
bmHint: af90
bFormatIndex: 5
bFrameIndex: 157
dwFrameInterval: 32766
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 4176
dwMaxVideoFrameSize: 0
dwMaxPayloadTransferSize: 4197312
bInterfaceNumber: 254
get_mode: Invalid mode (-51)
Device closed
UVC exited
root@iJudge:/home/pantheron/libuvc#
-
Okay,
it tooks the isight camera of the macbook...
So i added the vendor and product id directly into the code for the flir e4 and now it works :)
-
Good point. Can you post the changed code line , thks.
-
Here is a version that streams data in a uncompressed 8-bit grayscale AVI file. It can then be opened and colorized in ImageJ.
-
Good point. Can you post the changed code line , thks.
res = uvc_find_device(
ctx, &dev,
0, 0, NULL);
change 0,0 to vid,pid
https://int80k.com/libuvc/doc/group__device.html
uvc_error_t uvc_find_device ( uvc_context_t * ctx,
uvc_device_t ** dev,
int vid,
int pid,
const char * sn
)
Finds a camera identified by vendor, product and/or serial number.
Parameters
[in] ctx UVC context in which to search for the camera
[out] dev Reference to the camera, or NULL if not found
[in] vid Vendor ID number, optional
[in] pid Product ID number, optional
[in] sn Serial number or NULL
-
Thanks tomas123 for your input and information/files in your previous post!
-
Here is a version that supports both RAW 16-bit and grayscale 8-bit streaming to file
Usage:
./flirfox4 :Display video, no streaming to file
./flirfox4 -v :same as above
./flirfox4 -r filename :stream 16-bit RAW thermal video to file, no header
./flirfox4 -g filename :stream 8-bit grayscale auto-leveled video to file with AVI format header
Both can be opened in ImageJ program, color table applied and can be saved in a variety of formats.
EDIT: and it also has E4 PID/VID hardcoded, so it only be looking for E4 camera.
EDIT: do not expect well written code from me. First, I am Linux impaired. Second, I heard about C language that there is such thing and that is about it. Third, I am not a programmer.
-
Temperature calculation added. Try it folks if it works for you.
You will need a file called e4data.txt with your camera's calibration data. You create the file using exiftool as tomas123 explained here: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673)
You also need to make a sample image from your camera, exiftool extracts the data from it (google for exiftool download if you do not have it).
Run exiftool command against the sample E4 file:
exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR_sample.jpg >e4data.txt
Once made and inspected, place the e4data.txt file in the folder with the compiled executable. For now the program just displays temperature (at mouse pointer coordinates) in the top left corner. Compare it with readings on your camera by pointing the mouse and camera's crosshair to a same hot spot.
Thanks tomas123 for the formulas!
-
:-+ great work
unfortunately now I have not enough time to go deeper in your code
a fast check with Ubuntu results in a "nan" value for temperature (see screenshot)
$ exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR3344.jpg >e4data.txt
$ cat e4data.txt
Atmospheric Temperature : 20.0 C
Planck R1 : 13061.76
Planck B : 1352.2
Planck F : 2.7
Planck O : -7334
Planck R2 : 0.027282318
Object Distance : 1.00 m
Emissivity : 0.95
Reflected Apparent Temperature : 20.0 C
Relative Humidity : 50.0 %
$ /usr/bin/cc -o flirfox6 flirfox6.c -luvc `pkg-config --cflags --libs opencv` -lm
$ sudo ./flirfox6
I will tomorrow debug your nice code.
another note:
I compiled successful the libuvc patch on mac OS X 10.9.
my raw note for interested users (use macports):
//check
$ port installed | grep active
cmake @3.3.1_0 (active)
libusb @1.0.19_0 (active)
jpeg @9a_1 (active)
$ git clone https://github.com/ktossell/libuvc
Cloning into 'libuvc'...
remote: Counting objects: 526, done.
remote: Total 526 (delta 0), reused 0 (delta 0), pack-reused 526
Receiving objects: 100% (526/526), 263.90 KiB | 309.00 KiB/s, done.
Resolving deltas: 100% (305/305), done.
Checking connectivity... done.
$ cd libuvc/
// now use patch in the src folder
$ patch < libuvc.patch
patching file stream.c
cd ..
$ mkdir build
$ cd build
$ cmake ..
-- The C compiler identification is AppleClang 5.1.0.5030040
-- The CXX compiler identification is AppleClang 5.1.0.5030040
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Release
-- No target type selected, default to shared library
-- Found JPEG: /opt/local/lib/libjpeg.dylib
-- Building libuvc with JPEG support.
-- Configuring done
CMake Warning (dev):
Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake
--help-policy CMP0042" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
MACOSX_RPATH is not specified for the following targets:
uvc
This warning is for project developers. Use -Wno-dev to suppress it.
-- Generating done
-- Build files have been written to: /Users/jfried/Documents/Flir/uvc/libuvc/build
$ make
Scanning dependencies of target uvc
[ 10%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o
[ 20%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o
[ 30%] Building C object CMakeFiles/uvc.dir/src/device.c.o
[ 40%] Building C object CMakeFiles/uvc.dir/src/diag.c.o
[ 50%] Building C object CMakeFiles/uvc.dir/src/frame.c.o
[ 60%] Building C object CMakeFiles/uvc.dir/src/init.c.o
[ 70%] Building C object CMakeFiles/uvc.dir/src/stream.c.o
[ 80%] Building C object CMakeFiles/uvc.dir/src/misc.c.o
[ 90%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o
[100%] Linking C shared library libuvc.dylib
[100%] Built target uvc
the RAW streaming of Flir E4 works fine under OS X (sample code from tmbinc)
// fast check with my full pathes
$ cc -o flir flir.c -I/opt/local/include/ -I/Users/tomas/Documents/Flir/uvc/libuvc/build/include -I/Users/tomas/Documents/Flir/uvc/libuvc/include/ -L/Users/tomas/Documents/Flir/uvc/libuvc/build -luvc
$ ./flir
UVC initialized
usage: ./flir out.raw
./flir 1
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1441305922:212093, 153600 bytes
got frame 1441305922:308068, 153600 bytes
got frame 1441305922:404825, 153600 bytes
got frame 1441305922:500120, 153600 bytes
got frame 1441305922:594571, 153600 bytes
...
now I have some trouble with OpenCV under OS X ... :(
-
@tomas123
Oh-oh, seems a problem with exiftool outputting less data when run in linux vs windows. I ran exiftool in Windows, it dumps 15 parameters in total. Yours seems to dump only 10, that is what caused the nan error.
I will check what version of exiftool executable i have in windows.
-
So my exiftool Windows version is 9.9, this is what it outputs:
Atmospheric Temperature : 20.0 C
Atmospheric Trans Alpha 1 : 0.00656
Atmospheric Trans Alpha 2 : 0.01262
Atmospheric Trans Beta 1 : -0.0022
Atmospheric Trans Beta 2 : -0.0066
Atmospheric Trans X : 1.90000
Planck R1 : 14901.4
Planck B : 1400.9
Planck F : 2.5
Planck O : -7187
Planck R2 : 0.02804
Object Distance : 1.00 m
Emissivity : 0.95
Reflected Apparent Temperature : 20.0 C
Relative Humidity : 50.0 %
@tomas123: seems yours does not output alpha/beta/X, not sure if this is how exiftool works in Linux :-//
In the program I used the formulas with Distance from your Excel spreadsheet, that required I believe all 15 entries.
BTW, the last version is just using atmospheric temperature, object distance, reflected temperature and humidity read from the sample file. I intended later to replace them with sliders added to the video window so it would be possible to adjust for the current scene, since this information is not in the video stream.
-
Latest version,
- Temperature label moved to its own overlay so it will not appear in recorded files
- A crosshair cursor is added to read temperature, use the mouse to drag the cursor. Left mouse Double click resets position to the image center.
- The cursor appears in 8-bit grayscale video files, won't appear in 16-bit RAW video files.
- Added sliders to adjust a few ambient parameters .
For now I reverted to a fixed size window, as the sliders eat window space, shrinking the image. Would probably add a command line switch later to allow for window resize.
(http://i.imgur.com/7IZLRIS.png)
-
:-+ great work
unfortunately now I have not enough time to go deeper in your code
sorry, that I'm wasted your time
with this steps on my virtual machine
$ lsb_release -a
Description: Ubuntu 14.04.1 LTS
$ sudo apt-get install exiftool
Note, selecting 'libimage-exiftool-perl' instead of 'exiftool'
$ sudo apt-get install libimage-exiftool-perl
$ exiftool -ver
9.46
I got an old exiftool version from Jan. 11, 2014: "Version 9.46 production release" (without the values for atmosphere transmissivity)
... and to make matters worse, I'm a coder of the Flir part of exiftool |O
I downloaded the current version 10.0.1 from Phils homepage and now all works fine ;)
If you prefer to parse XML in C than you can use the X parameter in exiftool:
Image-ExifTool-10.01/exiftool -X -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR3344.jpg
<?xml version='1.0' encoding='UTF-8'?>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
<rdf:Description rdf:about='FLIR3344.jpg'
xmlns:et='http://ns.exiftool.ca/1.0/' et:toolkit='Image::ExifTool 10.01'
xmlns:FLIR='http://ns.exiftool.ca/APP1/FLIR/1.0/'>
<FLIR:AtmosphericTemperature>20.0 C</FLIR:AtmosphericTemperature>
<FLIR:AtmosphericTransAlpha1>0.006569</FLIR:AtmosphericTransAlpha1>
<FLIR:AtmosphericTransAlpha2>0.012620</FLIR:AtmosphericTransAlpha2>
<FLIR:AtmosphericTransBeta1>-0.002276</FLIR:AtmosphericTransBeta1>
<FLIR:AtmosphericTransBeta2>-0.006670</FLIR:AtmosphericTransBeta2>
<FLIR:AtmosphericTransX>1.900000</FLIR:AtmosphericTransX>
<FLIR:PlanckR1>18453.355</FLIR:PlanckR1>
<FLIR:PlanckB>1460.6</FLIR:PlanckB>
<FLIR:PlanckF>1</FLIR:PlanckF>
<FLIR:PlanckO>-7003</FLIR:PlanckO>
<FLIR:PlanckR2>0.013993904</FLIR:PlanckR2>
<FLIR:ObjectDistance>1.00 m</FLIR:ObjectDistance>
<FLIR:Emissivity>0.95</FLIR:Emissivity>
<FLIR:Emissivity>0.95</FLIR:Emissivity>
<FLIR:ReflectedApparentTemperature>20.0 C</FLIR:ReflectedApparentTemperature>
<FLIR:RelativeHumidity>50.0 %</FLIR:RelativeHumidity>
</rdf:Description>
</rdf:RDF>
-
Because there are some minor differences between the temperature of your crosshair cursor and the E4 screen I rechecked my excel sheet with a sample image.
(1) load image in Flir Tools and export the temperature off all pixels in a csv file
(2) with exiftool and imagemagick export the raw values of some pixels
(3) with Excel compare calculated temperature with exported values from Flir Tools
The values in Flir Tools and my Excel sheet are identical with all digits (milliKelvin) :-+
The calculations in your code are also right. I checked it with:
185: sprintf(szText, "%3.1f C / RAW: %i",t, pix[cpY * 320 + cpX]);
The temperature differences under crosshair I can't explain with averaging of 2x2 or 3x3 pixels on the E4 display :-\
all steps in bash
// export with Flir Tools the temperature of all Pixels (Flir Tools export as *.csv)
$ head -n11 FLIR0080.csv
File: ;C:\Users\admin\Desktop\1\FLIR0080.jpg
Parameters:;Emissivity:;0,95
;Refl. temp.:;20 °C
;Distance:;1 m
;Atmospheric temp.:;20 °C
;Ext. optics temp.:;20 °C
;Ext. optics trans.:;1
;Relative humidity:;50 %
;
Frame 1;18,359;18,373;18,373;18,359;18,387;18,429;18,401;18,457;18,457;18,443;18,443;18,429;18,415;18,415;18,401;18,401;18,401;18,387;18,387;18,373;18,359;18,345;18,331
// print a readable form
s jfried$ grep Frame FLIR0080.csv | tr ";" "\n" | head -n 17
Frame 1
18,359
18,373
18,373
18,359
18,387
18,429
18,401
18,457
18,457
18,443
18,443
18,429
18,415
18,415
18,401
18,401
// get Flir parameters for excel sheet
$ /Image-ExifTool-10.01/exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR0080.jpg
Atmospheric Temperature : 20.0 C
Atmospheric Trans Alpha 1 : 0.006569
Atmospheric Trans Alpha 2 : 0.012620
Atmospheric Trans Beta 1 : -0.002276
Atmospheric Trans Beta 2 : -0.006670
Atmospheric Trans X : 1.900000
Planck R1 : 15119.786
Planck B : 1405.2
Planck F : 2.5
Planck O : -7652
Planck R2 : 0.027600825
Object Distance : 1.00 m
Emissivity : 0.95
Reflected Apparent Temperature : 20.0 C
Relative Humidity : 50.0 %
// export RAW values of radiometric image
$ exiftool -b -RawThermalImage FLIR0080.jpg > FLIR0080.png
// swap bytes
$ convert -define png:swap-bytes=on FLIR0080.png FLIR0080swap.png
// print RAW values of first 16 Pixel (top left)
$ for i in $(seq 0 15); do convert FLIR0080swap.png -crop 1x1+$i+0 -format "%[fx:round(65535*u.r)]" info:;echo ; done
12167
12168
12168
12167
12169
12172
12170
12174
12174
12173
12173
12172
12171
12171
12170
12170
a pixel around the center of image (the crosshair)
$ convert FLIR0080swap.png -crop 1x1+160+120 -format "%[fx:round(65535*u.r)]" info:
12644
=> 24,785 °C (see imprinted spot 24,8°C)
see my links in signature for details
and https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg350250/#msg350250 (https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg350250/#msg350250)
-
Thanks tomas123. Could the difference be because of rounding errors in Linux math, the formulas have power of 2 and 3 as well as square roots, so rounding errors may accumulate. I think what can be done is to run the temp calculation routine with a known input value and check results of each step in the routine against your spreadsheet. This way we can catch where the difference occurs.
-
@tomas123
That is interesting the doctored libuvc works on Mac OS. Nice. I guess may also work on Win10, i heard it overcomes the limitations that prevented libuvc from working with camera streaming in Win7.
-
The differences are only visible in rare situations at higher temperatures and only small (<0,5K).
I'm not sure, but it doesn't look like a average of 4x4 pixels.
Also this differences are not rounding errors.
It's possible, that the E4 raw video stream contains less "magic Flir" post procession filters like the radiometric images (jpg).
-
Must be some secret sauce in E4, since temperature calculation in the program matches pretty well your Excel spreadsheet, I checked for a pixel level 10000:
T_Obj -8.397801 -8.397789
Anyway, attached is the version with MJPG compression added ( -m switch, will save 8-bit MJPG compressed file).
I will probably stop here for now.
EDIT: I think this program with some additions may be useful as a data logger for rapidly changing temperatures, since readings are available for each frame. Even this version can be used for that. Write video to a 8-bit file, open the file in any video viewer and step through the frames to read temperature at the cursor.
-
I will probably stop here for now.
You've done a great job :-+
I like your solution for temperature measurement under the free movable crosshair.
I did not have enough time to work with my raspberry pi (spycam for animals).
The OpenVC window is not the best solution for a ssh remote session disconnected with screen.
As a next step, I'll try to send the 8 Bit video stream to a v4l2loopback device (a video for linux loopback device like /dev/video1).
PS: I wasted many time for running your c code under Mac OS X.
The OpenCV library under OS X is a pain. I have unresolved problems with the OpenCV heap inside the function cb(frame,ptr) and the Qt library.
I think the best solution is switching to the newer opencv c++ api
But I give up :(
As I wrote here, the patched libuvc library works stable under OS X:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg747569/#msg747569 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg747569/#msg747569)
-
Yes I did not exactly like OpenCV but was easier for an inexperienced person like me to come up with some code. I run Ubuntu 14 on a Win7 guest using Oracle VirtualBox (32Gb RAM) and OpenCV appears to be glitchy, not sure if this is because it runs on a VM. It also seems to write 9 fps video with a 15fps header which causing the video to play too fast. Not sure how it ends up with 9 fps.
Anyway, this is what I call "better than nothing" and I tried to summarize the steps below:
The purpose of this topic and this program was to get FLIR E4 camera to stream RAW thermal 16-bit video that can be written in a file as uncompressed 16-bit grayscale video for further processing elsewhere. As a by side feature, the program can display a rescaled 8-bit video in a window and can also write 8-bit grayscale uncompressed and MJPG compressed files that include temperature measurement under a spot cursor that can be moved/set in the program using the mouse. The program also have a few adjustment controls that allow to change several environmental parameters and obtain corrected temperature measurements in real time.
The program seems to only work in Linux at this time because the underlying libraries (libuvc and OpenCV) seem to not provide required functionality in Win7 or MacOS.
Steps to get it running in Linux (I used Ubuntu 14):
- Download libuvc source and patch it as in this post from tomas123:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604)
(use the patch file provided by user tmbinc in the zip archive attached to the very first message in this topic) - Download and Compile OpenCV library as in this post from tomas123 (step 2)
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg742931/#msg742931 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg742931/#msg742931)
- Compile the program code
gcc -o flirfox9 flirfox9.c -luvc -lm `pkg-config --cflags --libs opencv` - Take a sample image from your camera, use exiftool to extract calibration constants for your camera from it (google for exiftool download if you do not have it).
Run exiftool command against the sample E4 file:
exiftool -Atmos* -Planck* -Object* -Emis* -Reflected* -Relativ* FLIR_sample.jpg >e4data.txt
Place the produced e4data.txt file in the folder with the compiled executable. - Connect the camera and run the program
./flirfox9 [options]
Options:
-r filename :write 16-bit RAW data to file
-g filename :write 8-bit grayscale data to file
-m filename :write 8-bit grayscale MJPG compressed data to file
-v :only display 8-bit grayscale video
Use the mouse to click-n-drug the temperature cursor to the desired position. left mouse dbl-click resets the cursor to the image center.
For post processing you may want to take a look at a nice piece of software called ImageJ
http://imagej.nih.gov/ij/ (http://imagej.nih.gov/ij/)
It has bunch of image processing options plus bunch of color palettes (LUTs) to color the video and resave in various formats.
-
a great tutorial
I edited my entry post on the first page and added a link list for new readers:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604)
-
some results from playing around with the code from tmbinc :-+:
hardware: a raspberry pi 2 with a 5 inch hdmi display from waveshare
http://www.waveshare.com/wiki/5inch_HDMI_LCD (http://www.waveshare.com/wiki/5inch_HDMI_LCD)
raspberry image: 2015-05-05-raspbian-wheezy.img
first install patched uvclib to get the raw video stream from a Flir Ex
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604)
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg623251/#msg623251 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg623251/#msg623251)
I changed the source from tmbinc for a 32 bit framebuffer...
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344 (https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344)
...for my display with 800x480 in 16 Bit depth (RGB565) and added some comments for using the framebuffer.
For better viewing I rescaled the video to 640x480.
source: flirfox2x.c
compile with
/usr/bin/cc -o flirfox2x ./flirfox2x.c -luvc -lm
using
$ sudo ./flirfox2x out.raw 20000
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
[0b397000] got frame 1442439612:475769, 153600 bytes, [12419..14340 rms 12865.158203 rmax 143]
[0b397000] got frame 1442439612:564221, 153600 bytes, [12415..14340 rms 12864.794922 rmax 284]
[0b397000] got frame 1442439612:666238, 153600 bytes, [12416..14339 rms 12864.424805 rmax 424]
[0b397000] got frame 1442439612:764188, 153600 bytes, [12417..14342 rms 12864.861328 rmax 563]
[0b397000] got frame 1442439612:866825, 153600 bytes, [12417..14339 rms 12864.305664 rmax 700]
you can clean your screen from frame buffer output with
xrefresh -d :0.0
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=171452;image)
next step is writing the video to a V4L2 loopback devices ( /dev/video0 )
https://github.com/umlaeute/v4l2loopback (https://github.com/umlaeute/v4l2loopback)
The installation of v4l2loopback on a raspberry 2 with 2015-05-05-raspbian-wheezy.img was tricky, because there are no sources included for building kernel modules.
(no /lib/modules/$(uname -r)/build )
All downloads to get the kernel sources for 3.18.7-v7+ fails :(
I found here a working solution with roll back from 3.18.7-v7+ to the older kernel 3.18.0
https://github.com/umlaeute/v4l2loopback/issues/84 (https://github.com/umlaeute/v4l2loopback/issues/84)
sudo aptitude install -y linux-image-rpi2-rpfv linux-headers-rpi2-rpfv
echo -e "kernel=vmlinuz-3.18.0-trunk-rpi2\ninitramfs=initrd.img-3.18.0-trunk-rpi followkernel" | sudo tee -a /boot/config.txt
sudo reboot
the next steps for installing v4l2loopback are simple
$ uname -a
Linux raspberryLCD 3.18.0-trunk-rpi2 #1 SMP PREEMPT Debian 3.18.5-1~exp1+rpi19 (2015-08-08) armv7l GNU/Linux
$ git clone [url]https://github.com/umlaeute/v4l2loopback[/url]
$ cd v4l2loopback
$ make
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
CC [M] /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.mod.o
LD [M] /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
$ sudo su
# make install
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules_install
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
INSTALL /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
DEPMOD 3.18.0-trunk-rpi2
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
depmod -a `uname -r`
$ sudo modprobe v4l2loopback
$ lsmod
Module Size Used by
v4l2loopback 23372 0
...
source: flirfox-loop.c
compile
/usr/bin/cc -o flirfox-loop ./flirfox-loop.c -luvc -lm
I simple mixed tmbinc source with the v4l2loopback test.c file ;)
source: https://github.com/umlaeute/v4l2loopback/blob/master/examples/test.c (https://github.com/umlaeute/v4l2loopback/blob/master/examples/test.c)
using
$ sudo ./flirfox-loop out.raw 20000
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
bits per pixel: 16
GUID: 5955593200001000800000aa00389b71 (YUY2)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x240
bit rate: 912384-912384
max frame size: 153600
default interval: 1/3
interval[0]: 1/15
interval[1]: 1/7
interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
using output device: /dev/video0
vid_format->type =2
vid_format->fmt.pix.width =320
vid_format->fmt.pix.height =240
vid_format->fmt.pix.pixelformat =1497715271
vid_format->fmt.pix.sizeimage =76800
vid_format->fmt.pix.field =1
vid_format->fmt.pix.bytesperline=320
vid_format->fmt.pix.colorspace =8
Streaming...
[0b34c000] got frame 1442439057:790436, 153600 bytes, [12283..12491 rms 12423.203125 rmax 124]
[0b34c000] got frame 1442439057:887940, 153600 bytes, [12284..12485 rms 12422.550781 rmax 247]
[0b34c000] got frame 1442439057:988181, 153600 bytes, [12280..12489 rms 12420.328125 rmax 369]
[0b34c000] got frame 1442439058:88068, 153600 bytes, [12279..12486 rms 12418.430664 rmax 490]
[0b34c000] got frame 1442439058:187958, 153600 bytes, [12281..12484 rms 12418.650391 rmax 609]
[0b34c000] got frame 1442439058:297959, 153600 bytes, [12281..12481 rms 12419.558594 rmax 727]
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=171450;image)
You can start flirfox-loop in the background (as sample a screen session) and log out.
The program writes the screen video to /dev/video0 device.
From this point you can use all video players or webcam programs to pass the video from /dev/video0 to a html page etc...
As sample screenshot from a ssh session with x11 forwarding the mplayer
$ ssh -X pi@192.168.1.70
$ mplayer tv:// -tv driver=v4l2:device=/dev/video0
MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
Playing tv://.
TV file format detected.
Selected driver: v4l2
name: Video 4 Linux 2 input
author: Martin Olschewski <olschewski@zpr.uni-koeln.de>
comment: first try, more to come ;-)
v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.
Selected device: Dummy video device (0x0000)
Capabilities: video capture read/write streaming
supported norms:
inputs: 0 = loopback;
Current input: 0
Current format: GREY
(https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/?action=dlattach;attach=171446;image)
-
unfortunately I have some trouble with charging the Ex with a raspberry 2
the max_usb_current switch doesn't work:
$ sudo nano /boot/config.txt
safe_mode_gpio=4
max_usb_current=1
the only solution for me was using a USB hub with a power supply :(
-
Wow, very nice!
:)
-
try the V4L2 loopback devices under Ubuntu ( /dev/video0 )
it's simple
$ sudo apt-get install v4l2loopback-dkms
Setting up v4l2loopback-dkms (0.8.0-1) ...
$ sudo modprobe v4l2loopback
// check your video devices video0 / video1
$ ls /dev/video*
- edit flirfox-loop.c for your loopback device video0 / video1
- compile flirfox-loop.c with
/usr/bin/cc -o flirfox-loop ./flirfox-loop.c -luvc -lm
read the v4l2loopback wiki for nice samples
https://github.com/umlaeute/v4l2loopback/wiki
- GStreamer
- FFmpeg
- MPlayer
- Skype
ffserver is a nice tool for a webcam server
google for:
ffserver /dev/video0
-
unfortunately I have some trouble with charging the Ex with a raspberry 2
It is not always working even with a conventional computer. I think FLIR did a poor job with implementation of the E4 USB charging circuit.
-
@admin
Please move this thread with essential informations about Flir E4 RAW streaming to the new subforum "Thermal Imaging"
Thank You
-
Can someone please help me? I'm on Mac. Trying to stream raw data from my E30bx. the frame size is 153600 and I'm just getting a stream of what's on the display, not the raw video. Any ideas?
-
There is a setting in the config file to enable radiometric video. If you know how to do that you can check if it is present and add it if it is not there. Not sure if it also can be set manually using rset command in Telnet, but you can try.
-
I do have that enabled. I noticed in Tomas's log from his E40 he had this
Streaming...
got frame 1439394751:346415, 157440 bytes
got frame 1439394751:411283, 157440 bytes
got frame 1439394751:479331, 157440 bytes
whereas I have this
Streaming...
got frame 1509121947:109894, 153600 bytes
got frame 1509121947:176200, 153600 bytes
got frame 1509121947:241827, 153600 bytes
got frame 1509121947:307965, 153600 bytes
The frames are too small to be radiometric.
When it's setting up the stream I see that the radiometric option is there...
UncompressedFormat(2)
bits per pixel: 16
GUID: 46374d3000001000800000aa00389b71 (F7M0)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 03
size: 320x246
bit rate: 912384-912384
max frame size: 157440
default interval: 1/7
interval[0]: 1/30
interval[1]: 1/15
interval[2]: 1/7
but it doesn't seem to grab that stream.
-
Making some progress. I had to checkout version 6ff39f17e3f3a26f437bc6b3c09cc6c289db7891 of libuvc. Now my frames are the correct size, but when I split them neither Imagemagick nor exiftool recognize them as being images.
-
More progress. I used Tomas's split.pl script. I had to modify the magic bytes line to match mine
#!/usr/bin/perl
undef $/;
$_ = <>;
$n = 100;
# FLIR camera E40
$pat="\x46\x46\x46\x00\x55\x56";
for $content (split(/(?=$pat)/)) {
open(OUT, ">seq" . ++$n . ".fff");
binmode OUT;
print OUT $content;
close(OUT);
}
From my raw data file:
00025800 46 46 46 00 55 56 43 00 00 00 00 00 00 00 00 00 |FFF.UVC.........|
00025810 00 00 00 00 64 00 00 00 40 00 00 00 02 00 00 00 |....d...@.......|
00025820 03 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |................|
00025830 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00025840 20 00 00 00 71 00 00 00 01 00 00 00 80 00 00 00 | ...q...........|
now exiftool "sort of" recognizes that the frames are thermal images
ExifTool Version Number : 10.55
File Name : seq102.fff
Directory : .
File Size : 154 kB
File Modification Date/Time : 2017:10:27 13:32:26-05:00
File Access Date/Time : 2017:10:27 13:40:12-05:00
File Inode Change Date/Time : 2017:10:27 13:32:26-05:00
File Permissions : rw-r--r--
File Type : FLIR
File Type Extension : fff
MIME Type : application/unknown
Creator Software : UVC
Emissivity : 0.95
Object Distance : 1.00 m
Reflected Apparent Temperature : 20.0 C
Atmospheric Temperature : 20.0 C
IR Window Temperature : 20.0 C
IR Window Transmission : 1.00
Relative Humidity : 50.0 %
Planck R1 : 14659.802
Planck B : 1391.5
Planck F : 1
Atmospheric Trans Alpha 1 : 0.006569
Atmospheric Trans Alpha 2 : 0.012620
Atmospheric Trans Beta 1 : -0.002276
Atmospheric Trans Beta 2 : -0.006670
Atmospheric Trans X : 1.900000
Camera Temperature Range Max : 120.0 C
Camera Temperature Range Min : -20.0 C
Camera Temperature Max Clip : 670.0 C
Camera Temperature Min Clip : -40.0 C
Camera Temperature Max Warn : 720.0 C
Camera Temperature Min Warn : -20.0 C
Camera Temperature Max Saturated: 150.0 C
Camera Temperature Min Saturated: -60.0 C
Camera Model : FLIR E30bx
Camera Part Number : 49001-1901
Camera Serial Number : 49023364
Camera Software : 20.0.0
Lens Model : FOL18
Lens Part Number :
Lens Serial Number :
Field Of View : 25.0 deg
Filter Model :
Filter Part Number :
Filter Serial Number :
Planck O : -6278
Planck R2 : 0.011133576
Raw Value Median : 18122
Raw Value Range : 1124
Date/Time Original : 2017:10:27 13:07:46.241-06:00
Focus Step Count : 1522
Focus Distance : 17.2 m
Raw Thermal Image Width : 17920
Raw Thermal Image Height : 21846
Warning : Unrecognized FLIR RawThermalImage data format
Raw Thermal Image Type : DAT
Raw Thermal Image : (Binary data 153568 bytes, use -b option to extract)
Peak Spectral Sensitivity : 10.3 um
this part is obviously an issue
Raw Thermal Image Width : 17920
Raw Thermal Image Height : 21846
Warning : Unrecognized FLIR RawThermalImage data format
Raw Thermal Image Type : DAT
Raw Thermal Image : (Binary data 153568 bytes, use -b option to extract)
Peak Spectral Sensitivity : 10.3 um
-
Closer.... need to figure out a way to deal with the 3840 bytes of crap at the end of each frame
-
I read this post, but there is still a little understanding. Can someone help me?
1.When connecting e4 wifi via usb, flir tools can choose whether to signal or video, and flir tools+ can record video directly. Does the method of modifying libuvc not apply to e4 wifi?
2.After getting the raw data stream through uvc, how can I convert it into Seq file?
-
I don't have a Flir E4 Wifi. Can the Flir E4 Wifi record 14-bit raw data, as opposed to remapped (colorized) video as visible on the screen?
-
Yes, if you connect e4 wifi via usb, you can choose to display 14-bit raw data or video as visible on the screen in flir tools(ver 5.0 or higher), which looks like the same as E40 (I don't have E40), but if you want to record it as seq file, You can only use flir tools+. The seq files recorded with flir tools+ can be opened in flir tools and edited. Flir tools+ is a paid software that can be used for free for one month.
-
But this topic does not have to do with flirtools. Not sure why you are asking this question here.
-
I mean that flir tools can read the 14-bit raw data from E4 WIFI. Does the method of patch libuvc have been invalidated? Can we use the driver of flir directly to get the 14-bit raw data in e4 wifi without patch for libuvc? .
-
Sorry i still do not understand why you bringing flir tools to this thread. The work done in this thread did not involve flir tools. Perhaps you are asking a generic question "if flir tools can do this or that", in which case you better open a separate topic in Thermal Imaging subforum.
-
I think it's a valid question - the original "idea" of this thread was that Flir tools didn't offer to capture raw data, but yet we found a method to do so.
If FLIR tools can now (I assume with a newer device firmware) do this, then yes, this deprecates this "hack". I would be curious how the FLIR tools to it - maybe they fixed the UVC descriptor?
-
I see. I did not try fiir tools with E4 wifi. Still, this solution in this topic is much more portable, if to run on a Pi.
-
Does it work with flir i5 cam ?
-
For other model with radiometric IR video streaming over WiFi (such as T640), is this approach also works for RTSP connection? e.g contained raw data in each frame?
-
RTSP is a compressed MJPEG stream, so no, it wouldn't work.
You need USB.