Author Topic: Flir Ex: Realtime raw radiometric data streaming via UVC (Read 77915 times)

tomas123 · « **Reply #75 on:** September 08, 2015, 01:56:49 pm »

a great tutorial

I edited my entry post on the first page and added a link list for new readers:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604

tomas123 · « **Reply #76 on:** September 16, 2015, 10:20:47 pm »

some results from playing around with the code from tmbinc

:

hardware: a raspberry pi 2 with a 5 inch hdmi display from waveshare
http://www.waveshare.com/wiki/5inch_HDMI_LCD

raspberry image: 2015-05-05-raspbian-wheezy.img

first install patched uvclib to get the raw video stream from a Flir Ex
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg623251/#msg623251

I changed the source from tmbinc for a 32 bit framebuffer...
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344
...for my display with 800x480 in 16 Bit depth (RGB565) and added some comments for using the framebuffer.

For better viewing I rescaled the video to 640x480.

source: flirfox2x.c
compile with

Code: [Select]

/usr/bin/cc -o flirfox2x ./flirfox2x.c -luvc -lm
using

Code: [Select]

$ sudo ./flirfox2x out.raw 20000
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
	bcdUVC: 0x0100
VideoStreaming(1):
	bEndpointAddress: 130
	Formats:
	UncompressedFormat(1)
		  bits per pixel: 16
		  GUID: 5955593200001000800000aa00389b71 (YUY2)
		  default frame: 1
		  aspect ratio: 0x0
		  interlace flags: 00
		  copy protect: 00
			FrameDescriptor(1)
			  capabilities: 03
			  size: 320x240
			  bit rate: 912384-912384
			  max frame size: 153600
			  default interval: 1/3
			  interval[0]: 1/15
			  interval[1]: 1/7
			  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
[0b397000] got frame 1442439612:475769, 153600 bytes, [12419..14340 rms 12865.158203 rmax 143]
[0b397000] got frame 1442439612:564221, 153600 bytes, [12415..14340 rms 12864.794922 rmax 284]
[0b397000] got frame 1442439612:666238, 153600 bytes, [12416..14339 rms 12864.424805 rmax 424]
[0b397000] got frame 1442439612:764188, 153600 bytes, [12417..14342 rms 12864.861328 rmax 563]
[0b397000] got frame 1442439612:866825, 153600 bytes, [12417..14339 rms 12864.305664 rmax 700]

you can clean your screen from frame buffer output with

Code: [Select]

xrefresh -d :0.0

next step is writing the video to a V4L2 loopback devices ( /dev/video0 )
https://github.com/umlaeute/v4l2loopback

The installation of v4l2loopback on a raspberry 2 with 2015-05-05-raspbian-wheezy.img was tricky, because there are no sources included for building kernel modules.
(no /lib/modules/$(uname -r)/build )
All downloads to get the kernel sources for 3.18.7-v7+ fails

I found here a working solution with roll back from 3.18.7-v7+ to the older kernel 3.18.0
https://github.com/umlaeute/v4l2loopback/issues/84

Code: [Select]

sudo aptitude install -y linux-image-rpi2-rpfv linux-headers-rpi2-rpfv
echo -e "kernel=vmlinuz-3.18.0-trunk-rpi2\ninitramfs=initrd.img-3.18.0-trunk-rpi followkernel" | sudo tee -a /boot/config.txt
sudo reboot

the next steps for installing v4l2loopback are simple

Code: [Select]

$ uname -a
Linux raspberryLCD 3.18.0-trunk-rpi2 #1 SMP PREEMPT Debian 3.18.5-1~exp1+rpi19 (2015-08-08) armv7l GNU/Linux

$ git clone [url]https://github.com/umlaeute/v4l2loopback[/url]

$ cd v4l2loopback

$ make 
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
  CC [M]  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.mod.o
  LD [M]  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'

$ sudo su

# make install
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules_install
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
  INSTALL /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
  DEPMOD  3.18.0-trunk-rpi2
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
depmod -a  `uname -r`

$ sudo modprobe v4l2loopback

$ lsmod
Module                  Size  Used by
v4l2loopback           23372  0 

...

source: flirfox-loop.c
compile

Code: [Select]

/usr/bin/cc  -o flirfox-loop ./flirfox-loop.c -luvc -lm

I simple mixed tmbinc source with the v4l2loopback test.c file

source: https://github.com/umlaeute/v4l2loopback/blob/master/examples/test.c

using

Code: [Select]

$ sudo ./flirfox-loop out.raw 20000
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
	bcdUVC: 0x0100
VideoStreaming(1):
	bEndpointAddress: 130
	Formats:
	UncompressedFormat(1)
		  bits per pixel: 16
		  GUID: 5955593200001000800000aa00389b71 (YUY2)
		  default frame: 1
		  aspect ratio: 0x0
		  interlace flags: 00
		  copy protect: 00
			FrameDescriptor(1)
			  capabilities: 03
			  size: 320x240
			  bit rate: 912384-912384
			  max frame size: 153600
			  default interval: 1/3
			  interval[0]: 1/15
			  interval[1]: 1/7
			  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
using output device: /dev/video0
     vid_format->type                =2
     vid_format->fmt.pix.width       =320
     vid_format->fmt.pix.height      =240
     vid_format->fmt.pix.pixelformat =1497715271
     vid_format->fmt.pix.sizeimage   =76800
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=320
     vid_format->fmt.pix.colorspace  =8
Streaming...
[0b34c000] got frame 1442439057:790436, 153600 bytes, [12283..12491 rms 12423.203125 rmax 124]
[0b34c000] got frame 1442439057:887940, 153600 bytes, [12284..12485 rms 12422.550781 rmax 247]
[0b34c000] got frame 1442439057:988181, 153600 bytes, [12280..12489 rms 12420.328125 rmax 369]
[0b34c000] got frame 1442439058:88068, 153600 bytes, [12279..12486 rms 12418.430664 rmax 490]
[0b34c000] got frame 1442439058:187958, 153600 bytes, [12281..12484 rms 12418.650391 rmax 609]
[0b34c000] got frame 1442439058:297959, 153600 bytes, [12281..12481 rms 12419.558594 rmax 727]

You can start flirfox-loop in the background (as sample a screen session) and log out.

The program writes the screen video to /dev/video0 device.
From this point you can use all video players or webcam programs to pass the video from /dev/video0 to a html page etc...

As sample screenshot from a ssh session with x11 forwarding the mplayer

Code: [Select]

$ ssh -X pi@192.168.1.70
$ mplayer tv:// -tv driver=v4l2:device=/dev/video0
MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing tv://.
TV file format detected.
Selected driver: v4l2
 name: Video 4 Linux 2 input
 author: Martin Olschewski <olschewski@zpr.uni-koeln.de>
 comment: first try, more to come ;-)
v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.
Selected device: Dummy video device (0x0000)
 Capabilities:  video capture  read/write  streaming
 supported norms:
 inputs: 0 = loopback;
 Current input: 0
 Current format: GREY

tomas123 · « **Reply #77 on:** September 16, 2015, 10:35:15 pm »

unfortunately I have some trouble with charging the Ex with a raspberry 2

the max_usb_current switch doesn't work:

Code: [Select]

$ sudo nano /boot/config.txt
safe_mode_gpio=4
max_usb_current=1

the only solution for me was using a USB hub with a power supply

Bud · « **Reply #78 on:** September 17, 2015, 03:15:36 am »

Wow, very nice!

tomas123 · « **Reply #79 on:** September 17, 2015, 07:55:13 am »

try the V4L2 loopback devices under Ubuntu ( /dev/video0 )

it's simple

Code: [Select]

$ sudo apt-get install v4l2loopback-dkms
Setting up v4l2loopback-dkms (0.8.0-1) ...

$ sudo modprobe v4l2loopback

// check your video devices video0 / video1
$ ls /dev/video*

- edit flirfox-loop.c for your loopback device video0 / video1
- compile flirfox-loop.c with

Code: [Select]

/usr/bin/cc  -o flirfox-loop ./flirfox-loop.c -luvc -lm

read the v4l2loopback wiki for nice samples
https://github.com/umlaeute/v4l2loopback/wiki

GStreamer
FFmpeg
MPlayer
Skype

ffserver is a nice tool for a webcam server
google for:
ffserver /dev/video0

Bud · « **Reply #80 on:** September 18, 2015, 02:09:06 am »

Quote from: tomas123 on September 16, 2015, 10:35:15 pm

unfortunately I have some trouble with charging the Ex with a raspberry 2

It is not always working even with a conventional computer. I think FLIR did a poor job with implementation of the E4 USB charging circuit.

tomas123 · « **Reply #81 on:** January 10, 2016, 10:53:44 pm »

@admin

Please move this thread with essential informations about Flir E4 RAW streaming to the new subforum "Thermal Imaging"

Thank You

eg14 · « **Reply #82 on:** October 27, 2017, 02:34:25 am »

Can someone please help me? I'm on Mac. Trying to stream raw data from my E30bx. the frame size is 153600 and I'm just getting a stream of what's on the display, not the raw video. Any ideas?

Bud · « **Reply #83 on:** October 27, 2017, 02:29:00 pm »

There is a setting in the config file to enable radiometric video. If you know how to do that you can check if it is present and add it if it is not there. Not sure if it also can be set manually using rset command in Telnet, but you can try.

eg14 · « **Reply #84 on:** October 27, 2017, 04:42:40 pm »

I do have that enabled. I noticed in Tomas's log from his E40 he had this

Code: [Select]

Streaming...
got frame 1439394751:346415, 157440 bytes
got frame 1439394751:411283, 157440 bytes
got frame 1439394751:479331, 157440 bytes

whereas I have this

Code: [Select]

Streaming...
got frame 1509121947:109894, 153600 bytes
got frame 1509121947:176200, 153600 bytes
got frame 1509121947:241827, 153600 bytes
got frame 1509121947:307965, 153600 bytes

The frames are too small to be radiometric.

When it's setting up the stream I see that the radiometric option is there...

Code: [Select]

UncompressedFormat(2)
		  bits per pixel: 16
		  GUID: 46374d3000001000800000aa00389b71 (F7M0)
		  default frame: 1
		  aspect ratio: 0x0
		  interlace flags: 00
		  copy protect: 00
			FrameDescriptor(1)
			  capabilities: 03
			  size: 320x246
			  bit rate: 912384-912384
			  max frame size: 157440
			  default interval: 1/7
			  interval[0]: 1/30
			  interval[1]: 1/15
			  interval[2]: 1/7

but it doesn't seem to grab that stream.

eg14 · « **Reply #85 on:** October 27, 2017, 06:16:37 pm »

Making some progress. I had to checkout version 6ff39f17e3f3a26f437bc6b3c09cc6c289db7891 of libuvc. Now my frames are the correct size, but when I split them neither Imagemagick nor exiftool recognize them as being images.

eg14 · « **Reply #86 on:** October 27, 2017, 06:43:21 pm »

More progress. I used Tomas's split.pl script. I had to modify the magic bytes line to match mine

Code: [Select]

#!/usr/bin/perl
undef $/;
$_ = <>;
$n = 100;
# FLIR camera E40         
$pat="\x46\x46\x46\x00\x55\x56";
for $content (split(/(?=$pat)/)) {
        open(OUT, ">seq" . ++$n . ".fff");
        binmode OUT;
        print OUT $content;
        close(OUT);
}

From my raw data file:

Code: [Select]

00025800  46 46 46 00 55 56 43 00  00 00 00 00 00 00 00 00  |FFF.UVC.........|
00025810  00 00 00 00 64 00 00 00  40 00 00 00 02 00 00 00  |....d...@.......|
00025820  03 00 00 00 01 00 00 00  00 00 00 00 00 00 00 00  |................|
00025830  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00025840  20 00 00 00 71 00 00 00  01 00 00 00 80 00 00 00  | ...q...........|

now exiftool "sort of" recognizes that the frames are thermal images

Code: [Select]

ExifTool Version Number         : 10.55
File Name                       : seq102.fff
Directory                       : .
File Size                       : 154 kB
File Modification Date/Time     : 2017:10:27 13:32:26-05:00
File Access Date/Time           : 2017:10:27 13:40:12-05:00
File Inode Change Date/Time     : 2017:10:27 13:32:26-05:00
File Permissions                : rw-r--r--
File Type                       : FLIR
File Type Extension             : fff
MIME Type                       : application/unknown
Creator Software                : UVC
Emissivity                      : 0.95
Object Distance                 : 1.00 m
Reflected Apparent Temperature  : 20.0 C
Atmospheric Temperature         : 20.0 C
IR Window Temperature           : 20.0 C
IR Window Transmission          : 1.00
Relative Humidity               : 50.0 %
Planck R1                       : 14659.802
Planck B                        : 1391.5
Planck F                        : 1
Atmospheric Trans Alpha 1       : 0.006569
Atmospheric Trans Alpha 2       : 0.012620
Atmospheric Trans Beta 1        : -0.002276
Atmospheric Trans Beta 2        : -0.006670
Atmospheric Trans X             : 1.900000
Camera Temperature Range Max    : 120.0 C
Camera Temperature Range Min    : -20.0 C
Camera Temperature Max Clip     : 670.0 C
Camera Temperature Min Clip     : -40.0 C
Camera Temperature Max Warn     : 720.0 C
Camera Temperature Min Warn     : -20.0 C
Camera Temperature Max Saturated: 150.0 C
Camera Temperature Min Saturated: -60.0 C
Camera Model                    : FLIR E30bx
Camera Part Number              : 49001-1901
Camera Serial Number            : 49023364
Camera Software                 : 20.0.0
Lens Model                      : FOL18
Lens Part Number                : 
Lens Serial Number              : 
Field Of View                   : 25.0 deg
Filter Model                    : 
Filter Part Number              : 
Filter Serial Number            : 
Planck O                        : -6278
Planck R2                       : 0.011133576
Raw Value Median                : 18122
Raw Value Range                 : 1124
Date/Time Original              : 2017:10:27 13:07:46.241-06:00
Focus Step Count                : 1522
Focus Distance                  : 17.2 m
Raw Thermal Image Width         : 17920
Raw Thermal Image Height        : 21846
Warning                         : Unrecognized FLIR RawThermalImage data format
Raw Thermal Image Type          : DAT
Raw Thermal Image               : (Binary data 153568 bytes, use -b option to extract)
Peak Spectral Sensitivity       : 10.3 um

this part is obviously an issue

Code: [Select]

Raw Thermal Image Width         : 17920
Raw Thermal Image Height        : 21846
Warning                         : Unrecognized FLIR RawThermalImage data format
Raw Thermal Image Type          : DAT
Raw Thermal Image               : (Binary data 153568 bytes, use -b option to extract)
Peak Spectral Sensitivity       : 10.3 um

eg14 · « **Reply #87 on:** October 27, 2017, 08:46:25 pm »

Closer.... need to figure out a way to deal with the 3840 bytes of crap at the end of each frame

cq-317 · « **Reply #88 on:** September 16, 2018, 07:50:44 am »

I read this post, but there is still a little understanding. Can someone help me?

1.When connecting e4 wifi via usb, flir tools can choose whether to signal or video, and flir tools+ can record video directly. Does the method of modifying libuvc not apply to e4 wifi?

2.After getting the raw data stream through uvc, how can I convert it into Seq file?

tmbinc · « **Reply #89 on:** September 17, 2018, 12:44:39 am »

I don't have a Flir E4 Wifi. Can the Flir E4 Wifi record 14-bit raw data, as opposed to remapped (colorized) video as visible on the screen?

cq-317 · « **Reply #90 on:** September 17, 2018, 06:56:24 am »

Yes, if you connect e4 wifi via usb, you can choose to display 14-bit raw data or video as visible on the screen in flir tools（ver 5.0 or higher）, which looks like the same as E40 (I don't have E40), but if you want to record it as seq file, You can only use flir tools+. The seq files recorded with flir tools+ can be opened in flir tools and edited. Flir tools+ is a paid software that can be used for free for one month.

Bud · « **Reply #91 on:** September 17, 2018, 11:54:39 am »

But this topic does not have to do with flirtools. Not sure why you are asking this question here.

cq-317 · « **Reply #92 on:** September 17, 2018, 01:51:36 pm »

I mean that flir tools can read the 14-bit raw data from E4 WIFI. Does the method of patch libuvc have been invalidated? Can we use the driver of flir directly to get the 14-bit raw data in e4 wifi without patch for libuvc? .

Bud · « **Reply #93 on:** September 17, 2018, 08:20:30 pm »

Sorry i still do not understand why you bringing flir tools to this thread. The work done in this thread did not involve flir tools. Perhaps you are asking a generic question "if flir tools can do this or that", in which case you better open a separate topic in Thermal Imaging subforum.

tmbinc · « **Reply #94 on:** September 18, 2018, 09:46:30 pm »

I think it's a valid question - the original "idea" of this thread was that Flir tools didn't offer to capture raw data, but yet we found a method to do so.

If FLIR tools can now (I assume with a newer device firmware) do this, then yes, this deprecates this "hack". I would be curious how the FLIR tools to it - maybe they fixed the UVC descriptor?

Bud · « **Reply #95 on:** September 18, 2018, 10:45:00 pm »

I see. I did not try fiir tools with E4 wifi. Still, this solution in this topic is much more portable, if to run on a Pi.

Uno winner · « **Reply #96 on:** October 16, 2018, 09:37:15 am »

Does it work with flir i5 cam ?

asmart · « **Reply #97 on:** November 30, 2018, 06:49:29 am »

For other model with radiometric IR video streaming over WiFi (such as T640), is this approach also works for RTSP connection? e.g contained raw data in each frame?

Spirit532 · « **Reply #98 on:** December 06, 2018, 02:29:56 am »

RTSP is a compressed MJPEG stream, so no, it wouldn't work.
You need USB.


EEVblog Main Site	EEVblog on Youtube	EEVblog on Twitter	EEVblog on Facebook	EEVblog on Odysee

Author Topic: Flir Ex: Realtime raw radiometric data streaming via UVC (Read 77915 times)

Share me