Question about FLIR One for Android

[tomas123]

I can't confirm that.
On my raspberry pi guvcview color controls is automatic set to MJPEG for /dev/video2 and /dev/video3 and all works fine 

flir8i2.c (see attachment) writes all jpg frames for /dev/video2 and /dev/video3 to file

the only ones differences I found in the .jpg files was the subsampling (v2=/dev/video2 and v3=/dev/video3)

Code: [Select]

$ exiftool -YCb* \#0000010-v*.jpg
======== #0000010-v2.jpg
Y Cb Cr Sub Sampling            : YCbCr4:2:2 (2 1)
======== #0000010-v3.jpg
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
    2 image files read
both subsamplings are valid for MJPEG
but nevertheless I changed the code for libjpeg at line 353 for YCbCr4:2:2 subsampling   

Code: [Select]

    // 4:2:2 (2x1 1x1 1x1) - CrH 50% - CbH 50% - CrV 100% - CbV 100%
    cinfo.comp_info[0].h_samp_factor = 2; // Y
    cinfo.comp_info[0].v_samp_factor = 1;
    cinfo.comp_info[1].h_samp_factor = 1; // Cb
    cinfo.comp_info[1].v_samp_factor = 1;
    cinfo.comp_info[2].h_samp_factor = 1; // Cr
    cinfo.comp_info[2].v_samp_factor = 1;
Now I can't see a difference.

$ exiftool  \#0000010-v*.jpg
======== #0000010-v2.jpg
ExifTool Version Number         : 10.02
File Name                       : #0000010-v2.jpg
File Size                       : 7.7 kB
Resource Fork Size              : 286 bytes
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Image Width                     : 640
Image Height                    : 480
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:2 (2 1)
Image Size                      : 640x480
Megapixels                      : 0.307

======== #0000010-v3.jpg
ExifTool Version Number         : 10.02
File Name                       : #0000010-v3.jpg
File Size                       : 11 kB
Resource Fork Size              : 286 bytes
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Image Width                     : 160
Image Height                    : 120
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:2 (2 1)
Image Size                      : 160x120
Megapixels                      : 0.019
    2 image files read

... except the missing JFIF in /video2

Code: [Select]

$ hexdump -n32 -C #0000010-v2.jpg
00000000  ff d8 ff c0 00 11 08 01  e0 02 80 03 01 21 00 02  |.............!..|
00000010  11 01 03 11 01 ff dd 00  04 00 28 ff db 00 84 00  |..........(.....|

$ hexdump -n32 -C  #0000010-v3.jpg
00000000  ff d8 ff e0 00 10 4a 46  49 46 00 01 01 00 00 01  |......JFIF......|
00000010  00 01 00 00 ff db 00 43  00 03 02 02 03 02 02 03  |.......C........|

no differences in the announcements of the v4l2 loopback devices

Code: [Select]

$ v4l2-ctl --all -d /dev/video2
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0002)
Bus info      : platform:v4l2loopback-002
Driver version: 3.18.13
Capabilities  : 0x85208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x05208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Video input : 0 (loopback: ok)
Format Video Capture:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Format Video Output:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Streaming Parameters Video Capture:
Frames per second: 5.000 (5/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 5.000 (5/1)
Write buffers    : 8

User Controls

                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0

$ v4l2-ctl --all -d /dev/video3
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0003)
Bus info      : platform:v4l2loopback-003
Driver version: 3.18.13
Capabilities  : 0x85208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x05208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Video input : 0 (loopback: ok)
Format Video Capture:
Width/Height  : 160/120
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 76800
Colorspace    : SRGB
Custom Info   : feedcafe
Format Video Output:
Width/Height  : 160/120
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 76800
Colorspace    : SRGB
Custom Info   : feedcafe
Streaming Parameters Video Capture:
Frames per second: 5.000 (5/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 5.000 (5/1)
Write buffers    : 8

User Controls

                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0
But still mjpg_streamer has the same problem with the mjpeg stream 160x120.

Code: [Select]

$ export LD_LIBRARY_PATH=/usr/local/lib

$ mjpg_streamer -i "input_uvc.so -d /dev/video3 -r 160x120" -o "output_http.so -w /usr/local/www"
MJPG Streamer Version: svn rev: Unversioned directory
 i: Using V4L2 device.: /dev/video3
 i: Desired Resolution: 160 x 120
 i: Frames Per Second.: 5
i: Format............: MJPEG
...
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
*** glibc detected *** mjpg_streamer: free(): invalid next size (normal): 0x01258910 ***
Aborted
Is 160x120 to small for mjpg_streamer?

[cynfab]

tomas123,
Maybe we should expanding the 160 x 120 image to 320 x 240. just a quick scaling by 2 should be enough to see if there is a minimum size issue.

   ...ken...

[tomas123]

Yes, with an upscaled image to 320 x 240 mjpg_streamer works fine without a buffer/pointer error.

Code: [Select]

./mjpg_streamer -i "./input_uvc.so -d /dev/video3" -o "output_http.so -w /usr/local/www"
I wasted many time for this bug 

Open and edit the apporitate plugins (input_uvc) and the programs Makefile, and remove the hashmark befor the following row:
#CFLAGS +=  -DDEBUG

After debugging the mjpg_streamer source code  I know, that the error comes from the second (!) grabbed frame

Code: [Select]

$ ./mjpg_streamer -i "./input_uvc.so -d /dev/video3" -o "output_http.so -w /usr/local/www"
MJPG Streamer Version: svn rev: 3:172M
 DBG(input_uvc.c, input_init(), 107): argv[0]=UVC webcam grabber
 DBG(input_uvc.c, input_init(), 107): argv[1]=-d
 DBG(input_uvc.c, input_init(), 107): argv[2]=/dev/video3
 DBG(input_uvc.c, input_init(), 162): case 2,3
 DBG(input_uvc.c, input_init(), 245): input id: 0
 i: Using V4L2 device.: /dev/video3
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
 DBG(input_uvc.c, input_init(), 265): vdIn pn: 0
i: The format asked unavailable, so the width 160 height 120
 DBG(v4l2uvc.c, init_videoIn(), 94): Current size: 160x120
 DBG(v4l2uvc.c, init_videoIn(), 123): Supported format: [MJPG]
 DBG(v4l2uvc.c, init_videoIn(), 153): Supported size with the current format: 160x120
 DBG(v4l2uvc.c, init_videoIn(), 123): Supported format: RGB3
 DBG(v4l2uvc.c, init_videoIn(), 155): Supported size: 160x120
 DBG(v4l2uvc.c, init_videoIn(), 123): Supported format: BGR3
 DBG(v4l2uvc.c, init_videoIn(), 155): Supported size: 160x120
 DBG(v4l2uvc.c, init_videoIn(), 123): Supported format: YU12
 DBG(v4l2uvc.c, init_videoIn(), 155): Supported size: 160x120
 DBG(v4l2uvc.c, init_videoIn(), 123): Supported format: YV12
 DBG(v4l2uvc.c, init_videoIn(), 155): Supported size: 160x120
...
 DBG(v4l2uvc.c, enumerateControls(), 801): V4L2 API's V4L2_CTRL_FLAG_NEXT_CTRL is supported
 DBG(v4l2uvc.c, control_readed(), 705): V4L2 parameter found: User Controls value 2123345140 Class: USER
 DBG(v4l2uvc.c, control_readed(), 710): Unable to get the value of User Controls retcode: -1  Permission denied
 DBG(v4l2uvc.c, control_readed(), 705): V4L2 parameter found: keep_format value 0 Class: USER
 DBG(v4l2uvc.c, control_readed(), 705): V4L2 parameter found: sustain_framerate value 0 Class: USER
 DBG(v4l2uvc.c, control_readed(), 705): V4L2 parameter found: timeout value 0 Class: USER
 DBG(v4l2uvc.c, control_readed(), 705): V4L2 parameter found: timeout_image_io value 0 Class: USER
 DBG(v4l2uvc.c, enumerateControls(), 836): JPEG compression details:
 DBG(v4l2uvc.c, enumerateControls(), 837): Quality: 0
 DBG(v4l2uvc.c, enumerateControls(), 838): APPn: 0
 DBG(v4l2uvc.c, enumerateControls(), 839): APP length: 0
 DBG(v4l2uvc.c, enumerateControls(), 840): APP data:
 DBG(v4l2uvc.c, enumerateControls(), 841): COM length: 0
 DBG(v4l2uvc.c, enumerateControls(), 842): COM data:
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
 DBG(input_uvc.c, input_run(), 310): launching camera thread #00
,sizeof(struct v4l2_buffer) DBG(v4l2uvc.c, uvcGrab(), 431): X3
 DBG(input_uvc.c, cam_thread(), 382): received frame of size: 76800 from plugin: 0
 DBG(input_uvc.c, cam_thread(), 411): copying frame from input: 0
 DBG(input_uvc.c, cam_thread(), 436): waiting for next frame
*** glibc detected *** ./mjpg_streamer: free(): invalid next size (normal): 0x000ee9d8 ***
Aborted
the errors comes from this line:

input_uvc.c

Code: [Select]

             /* grab a frame */
377:        if(uvcGrab(pcontext->videoIn) < 0) {

this goes to V4l2 command VIDIOC_DQBUF
v4l2uvc.c

Code: [Select]

int uvcGrab(struct vdIn *vd)
{
430:     vd->buf.memory = V4L2_MEMORY_MMAP;
431:     ret = xioctl(vd->fd, VIDIOC_DQBUF, &vd->buf);

with

Code: [Select]

static int xioctl(int fh, int request, void *arg)
{
        int r;
        do
        {
            r = ioctl(fh, request, arg);
        } while (-1 == r && EINTR == errno);
        return r;
}

but I found no solution


only for information:

... except the missing JFIF in /video2

Code: [Select]

$ hexdump -n32 -C #0000010-v2.jpg
00000000  ff d8 ff c0 00 11 08 01  e0 02 80 03 01 21 00 02  |.............!..|
00000010  11 01 03 11 01 ff dd 00  04 00 28 ff db 00 84 00  |..........(.....|

$ hexdump -n32 -C  #0000010-v3.jpg
00000000  ff d8 ff e0 00 10 4a 46  49 46 00 01 01 00 00 01  |......JFIF......|
00000010  00 01 00 00 ff db 00 43  00 03 02 02 03 02 02 03  |.......C........|


Motion JPEG AVI files typically have an APP0 header (FF E0), with the tag 'AVI1' and not 'JFIF'.

remove a JFIF header from jpeg with

Code: [Select]

  cinfo.write_JFIF_header = FALSE;
  jpeg_start_compress(&cinfo, TRUE);

[tomas123]

Hi cynfab,

have you ever tried mjpg_streamer on your Linux Mint?

Today I tested it the first time on a Linux Mint Notebook with our old code flir8i.c
Reverse:
- real video stream 640x480 on /dev/video2 goes to a memory error
- thermal video stream 160x120 on /dev/video3 works fine

In the german ubuntu forum
https://wiki.ubuntuusers.de/MJPG-Streamer/
they describe, that the old Rev. 182 (Date 2008-06-21) of /mjpg_streamer is buggy while reading MJPEG.

I can confirm it.

[cynfab]

Hi tomas123,
No I haven't tried mjpeg-streamer on Mint. been busy & now snowed in again
Trying to get a new kernel built for my pcduino, and having some toolchain problems. Once I get that working I'll be trying the various flir8x, v4l2 & loopback code on it.

   ...ken...

[Ben321]

When is FLIR going to start selling 160x120 Lepton chips as standalone components. I just recently checked their website and they link to Digikey which is selling the old 80x60 chips still. How long until the old ones sell out, so they can start selling the new version? And when (if ever) is FLIR going to start making true 320x240 or 640x480 Lepton chips? It seems they must have in the past couple years discovered a better manufacturing process for making vanadium oxide microbolometer chips, because 160x120 thermal imagers used to sell for about $2500, but the FLIR One costs only $250. If this is how dramatic they have been able to reduce the cost on 160x120 thermal images, I'm expecting them to eventually expand their range of super-cheap thermal imagers to eventually include ones with super-cheap 320x240, or even 640x480, VOX microbolometer chips (maybe a Lepton v4 or v5). Have you guys heard any rumors about any plans from FLIR for future versions of the Lepton chip that have a higher resolution than 160x120?

[Lord of nothing]

are there any other company who make thermal sensors?

[frenky]

Check out this two:
http://www.i3system.com/eng/
http://www.heimannsensor.com/products_imaging.php

Sent from my LG-D855 using Tapatalk

[tomas123]

a subforum for "Thermal Imaging" 


Hi cynfab,

after terrible hours with a self compiled kernel driver (v4l2loopback.so) for ARM systems

Code: [Select]

$ find / -name v4l2loopback.ko 2>/dev/zero
/lib/modules/3.18.0-trunk-rpi2/extra/v4l2loopback.ko
/lib/modules/3.18.13-v7+/extra/v4l2loopback.ko
... I suggest new concept for using MJPEG Streamer with our Flir One:
Create a RAMDisk and use the input_file.so module of mjpg_streamer.
It works great with the low powered raspberry pi and you don't need the loopback devices /dev/videoX


(1) create a small ramdisk of 20MB

Code: [Select]

$ sudo mkdir /mnt/RAMDisk
$ sudo chmod 777 /mnt/RAMDisk

$ sudo mount -t tmpfs  -o size=20m none  /mnt/RAMDisk

$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
none               20480   20480         0 100% /mnt/RAMDisk

(2) compile the code flir8k.c from attachment


check your packages

Code: [Select]

$ sudo apt-get install libusb-1.0-0
$ sudo apt-get install libusb-1.0-0-dev
$ sudo apt-get install libjpeg-dev

Code: [Select]

$ make
gcc -I/usr/include/libusb-1.0    -c -o flir8k.o flir8k.c
gcc -I/usr/include/libusb-1.0 -o flir8k Palettes.o flir8k.o -lusb-1.0 -ljpeg -lm -Wall

(3) start the binary

now the binary writes the two images real.jpg and thermal.jpg to /mnt/RAMDisk/ with 8 fps

Code: [Select]

$ sudo ./flir8k
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO

:xx Sun Jan 10 16:30:27 2016

: Sun Jan 10 16:30:27 2016
 bulk read EP 0x81, actual length 101
HEX:
 cc 01 00 00 01 00 00 00 55 00 00 00 30 73 8c df 7b 22 74 79 70 65 22 3a 22 62 61 74 74 65 72 79 56 6f 6c 74 61 67 65 55 70 64 61 74 65 22 2c 22 64 61 74 61 22 3a 7b 22 76 6f 6c 74 61 67 65 22 3a 34 2e 31 37 30 30 30 30 30 37 36 32 39 33 39 35 2c 22 70 65 72 63 65 6e 74 61 67 65 22 3a 31 30 30 7d 7d 00
STRING:
?U0s??{"type":"batteryVoltageUpdate","data":{"voltage":4.17000007629395,"percentage":100}}

: Sun Jan 10 16:30:27 2016
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x83:-7 LIBUSB_ERROR_TIMEOUT

ask for CameraFiles.zip on EP 0x83:

: Sun Jan 10 16:30:28 2016

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 41 00 00 00 f8 b3 f7 00 ]

Write successful!
EP 0x02 to be sent: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}
Write successful!
Sent 65 bytes with string: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 33 00 00 00 ef db c1 c1 ]

Write successful!
EP 0x02 to be sent 51 Bytes: {"type":"readFile","data":{"streamIdentifier":10}}
Write successful!
Sent 51 bytes with string: {"type":"readFile","data":{"streamIdentifier":10}}

: Sun Jan 10 16:30:28 2016


Ask for video stream, start EP 0x85:

: Sun Jan 10 16:30:28 2016
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x81:-7 LIBUSB_ERROR_TIMEOUT
#000001 0/10 fps:{"shutterState":"ON","shutterTemperature":310.179992675781,"usbNotifiedTimestamp":1177004833.33163,"usbEnqueuedTimestamp":1177004833.33369,"ffcState":"FFC_VALID_RAD"}
#000002 0/10 fps:{"shutterState":"FFC","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004852.00896,"usbEnqueuedTimestamp":1177004852.01061,"ffcState":"FFC_RAD_APPROX"}

: Sun Jan 10 16:30:47 2016
 bulk read EP 0x81, actual length 206
HEX:
 cc 01 00 00 01 00 00 00 55 00 00 00 30 73 8c df 7b 22 74 79 70 65 22 3a 22 62 61 74 74 65 72 79 56 6f 6c 74 61 67 65 55 70 64 61 74 65 22 2c 22 64 61 74 61 22 3a 7b 22 76 6f 6c 74 61 67 65 22 3a 34 2e 31 37 30 30 30 30 30 37 36 32 39 33 39 35 2c 22 70 65 72 63 65 6e 74 61 67 65 22 3a 31 30 30 7d 7d 00 cc 01 00 00 01 00 00 00 59 00 00 00 88 cc 5a 95 7b 22 74 79 70 65 22 3a 22 62 61 74 74 65 72 79 43 68 61 72 67 69 6e 67 53 74 61 74 65 55 70 64 61 74 65 22 2c 22 64 61 74 61 22 3a 7b 22 63 68 61 72 67 69 6e 67 53 74 61 74 65 22 3a 22 73 74 61 74 65 43 68 61 72 67 69 6e 67 53 6d 61 72 74 50 68 6f
STRING:
?U0s??{"type":"batteryVoltageUpdate","data":{"voltage":4.17000007629395,"percentage":100}}?Y??Z?{"type":"batteryChargingStateUpdate","data":{"chargingState":"stateChargingSmartPho
#000003 0/10 fps:{"shutterState":"FFC","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004852.57983,"usbEnqueuedTimestamp":1177004852.58155,"ffcState":"FFC_RAD_APPROX"}
#000004 3/10 fps:{"shutterState":"FFC","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004852.69227,"usbEnqueuedTimestamp":1177004852.69411,"ffcState":"FFC_RAD_APPROX"}
#000005 8/10 fps:{"shutterState":"ON","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004852.80562,"usbEnqueuedTimestamp":1177004852.80794,"ffcState":"FFC_RAD_APPROX"}
#000006 11/10 fps:{"shutterState":"ON","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004852.91994,"usbEnqueuedTimestamp":1177004852.92159,"ffcState":"FFC_RAD_APPROX"}
#000007 14/10 fps:{"shutterState":"ON","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004853.03383,"usbEnqueuedTimestamp":1177004853.03595,"ffcState":"FFC_RAD_APPROX"}
#000008 18/10 fps:{"shutterState":"ON","shutterTemperature":308.640014648438,"usbNotifiedTimestamp":1177004853.14999,"usbEnqueuedTimestamp":1177004853.15202,"ffcState":"FFC_RAD_APPROX"}
#000009 20/10 fps:{"shutterState":"ON","shutterTemperature":308.75,"usbNotifiedTimestamp":1177004853.26203,"usbEnqueuedTimestamp":1177004853.26407,"ffcState":"FFC_RAD_APPROX"}
#000010 23/10 fps:{"shutterState":"ON","shutterTemperature":308.75,"usbNotifiedTimestamp":1177004853.37733,"usbEnqueuedTimestamp":1177004853.37923,"ffcState":"FFC_RAD_APPROX"}
#000011 26/10 fps:{"shutterState":"ON","shutterTemperature":308.75,"usbNotifiedTimestamp":1177004853.49003,"usbEnqueuedTimestamp":1177004853.49223,"ffcState":"FFC_RAD_APPROX"}
#000012 28/10 fps:{"shutterState":"ON","shutterTemperature":308.769989013672,"usbNotifiedTimestamp":1177004853.60389,"usbEnqueuedTimestamp":1177004853.60574,"ffcState":"FFC_RAD_APPROX"}
...
#003170 86/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004033.54082,"usbEnqueuedTimestamp":1177004033.542,"ffcState":"FFC_VALID_RAD"}
#003171 84/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004033.65724,"usbEnqueuedTimestamp":1177004033.6585,"ffcState":"FFC_VALID_RAD"}
#003172 83/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004033.76884,"usbEnqueuedTimestamp":1177004033.77,"ffcState":"FFC_VALID_RAD"}
#003173 83/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004033.88303,"usbEnqueuedTimestamp":1177004033.89359,"ffcState":"FFC_VALID_RAD"}
#003174 84/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004034.00084,"usbEnqueuedTimestamp":1177004034.00196,"ffcState":"FFC_VALID_RAD"}
#003175 84/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004034.11106,"usbEnqueuedTimestamp":1177004034.11261,"ffcState":"FFC_VALID_RAD"}
#003176 84/10 fps:{"shutterState":"ON","shutterTemperature":310.679992675781,"usbNotifiedTimestamp":1177004034.2256,"usbEnqueuedTimestamp":1177004034.22683,"ffcState":"FFC_VALID_RAD"}

(4) run a web server on /var/www/

- like apache

Code: [Select]

$ sudo apt-get install apache2
- link the two files real.jpg and thermal.jpg from RAMDISK to web folder
- check your web folder (current apache on raspi is /var/www/html/thermal.jpg

Code: [Select]

$ sudo ln -s /mnt/RAMDisk/thermal.jpg /var/www/thermal.jpg
$ sudo ln -s /mnt/RAMDisk/real.jpg /var/www/real.jpg// don't forget the "sudo chmod 777 /mnt/RAMDisk" from (1) 

now you can see the two images in your web browser

Code: [Select]

http://192.168.1.70/real.jpg
http://192.168.1.70/thermal.jpg
my raspberry pi has the address 192.168.1.70


(5) View video stream on an iDevice / Smartphone

 load from apple app store ore google play store the free app IP Cam Viewer Lite from NibblesnBits

http://elinux.org/RPi-Cam-Web-Interface#View_video_stream_on_an_iDevice_.2F_Smartphone

Select the menu icon
Press Manage Cameras
Select Add Camera then Generic URL
Give your cam a name
For Type choose Generic Video URL
Enter your URL, e.g. http://192.168.0.1:80/thermal.jpg
Press Test
If it works, press Save

You can to it twice for thermal.jpg and real.jpg and then you see the two streams parallel.
It's a little bit rough for the large image real.jpg with 9fps over WLAN, but thermal.jpg works fine


(6) create two MJPEG streams for real and thermal image

compile mjpg-streamer from source
mjpg-streamer needs imagemagick

Code: [Select]

$ sudo apt-get install imagemagick

Code: [Select]

$ sudo apt-get install subversion
$ svn co svn://svn.code.sf.net/p/mjpg-streamer/code/ mjpg-streamer
...
Checked out revision 182.
$ cd mjpg-streamer/mjpg-streamer
$ make
gcc -D'SVN_REV="3:172"' -O2 -DLINUX -D_GNU_SOURCE -Wall    -c -o mjpg_streamer.o mjpg_streamer.c
gcc -D'SVN_REV="3:172"' -O2 -DLINUX -D_GNU_SOURCE -Wall    -c -o utils.o utils.c
gcc -D'SVN_REV="3:172"' -O2 -DLINUX -D_GNU_SOURCE -Wall  mjpg_streamer.o utils.o -lpthread -ldl -o mjpg_streamer
chmod 755 mjpg_streamer
make -C plugins/input_uvc all
...
cp plugins/input_file/input_file.so

$ ls -al
total 412
drwxr-xr-x  5 pi pi   4096 Jan 10 17:24 .
drwxr-xr-x 10 pi pi   4096 Jan 10 17:22 ..
-rw-r--r--  1 pi pi    102 Jan 10 17:22 CHANGELOG
-rwxr-xr-x  1 pi pi  14331 Jan 10 17:24 input_file.so
-rwxr-xr-x  1 pi pi 168606 Jan 10 17:24 input_testpicture.so
-rwxr-xr-x  1 pi pi  30905 Jan 10 17:24 input_uvc.so
-rw-r--r--  1 pi pi  17987 Jan 10 17:22 LICENSE
-rw-r--r--  1 pi pi   5107 Jan 10 17:22 Makefile
-rwxr-xr-x  1 pi pi  16104 Jan 10 17:24 mjpg_streamer
-rw-r--r--  1 pi pi  15738 Jan 10 17:22 mjpg_streamer.c
-rw-r--r--  1 pi pi   3617 Jan 10 17:22 mjpg_streamer.h
-rw-r--r--  1 pi pi   9800 Jan 10 17:24 mjpg_streamer.o
-rwxr-xr-x  1 pi pi  20284 Jan 10 17:24 output_file.so
-rwxr-xr-x  1 pi pi  32436 Jan 10 17:24 output_http.so
-rwxr-xr-x  1 pi pi  14891 Jan 10 17:24 output_udp.so
drwxr-xr-x 13 pi pi   4096 Jan 10 17:22 plugins
-rw-r--r--  1 pi pi   2631 Jan 10 17:22 README
drwxr-xr-x  2 pi pi   4096 Jan 10 17:22 scripts
-rwxr-xr-x  1 pi pi   4630 Jan 10 17:22 start.sh
-rw-r--r--  1 pi pi    285 Jan 10 17:22 TODO
-rw-r--r--  1 pi pi   2866 Jan 10 17:22 utils.c
-rw-r--r--  1 pi pi   2399 Jan 10 17:22 utils.h
-rw-r--r--  1 pi pi   1804 Jan 10 17:24 utils.o
drwxr-xr-x  2 pi pi   4096 Jan 10 17:22 www

You don't need a "sudo make install". Stay in current folder, the are all required binaries included.

test the mjpg_streamer and get the help

Code: [Select]

$ LD_LIBRARY_PATH=./ ./mjpg_streamer -i "input_file.so -h"
MJPG Streamer Version: svn rev: 3:172
 ---------------------------------------------------------------
 Help for input plugin..: FILE input plugin
 ---------------------------------------------------------------
 The following parameters can be passed to this plugin:

 [-d | --delay ]........: delay to pause between frames
 [-f | --folder ].......: folder to watch for new JPEG files
 [-r | --remove ].......: remove/delete JPEG file after reading
 [-n | --name ].........: ignore changes unless filename matches
 ---------------------------------------------------------------

$ LD_LIBRARY_PATH=./ ./mjpg_streamer -o "output_http.so -h"
MJPG Streamer Version: svn rev: 3:172
 ---------------------------------------------------------------
 Help for output plugin..: HTTP output plugin
 ---------------------------------------------------------------
 The following parameters can be passed to this plugin:

 [-w | --www ]...........: folder that contains webpages in
                           flat hierarchy (no subfolders)
 [-p | --port ]..........: TCP port for this HTTP server
 [-c | --credentials ]...: ask for "username:password" on connect
 [-n | --nocommands ]....: disable execution of commands
 ---------------------------------------------------------------
write a start script start1.sh (there is already a start.sh)

Code: [Select]

$ cat ./start1.sh
export LD_LIBRARY_PATH=./
./mjpg_streamer -i "input_file.so -f /mnt/RAMDisk/ -n thermal.jpg" -o "output_http.so -p 8080 -w /usr/local/www" &
./mjpg_streamer -i "input_file.so -d 250 -f /mnt/RAMDisk/ -n real.jpg"    -o "output_http.so -p 8081 -w /usr/local/www"

Please note, that I reduced with "-d 250"ms the frame rate of the visible image to 4 fps, to reduce the WLAN bandwidth

and start the script

Code: [Select]

$ ./start1.sh
MJPG Streamer Version: svn rev: 3:172M
 i: folder to watch...: /mnt/RAMDisk/
 i: forced delay......: 250
 i: delete file.......: no, do not delete
 i: filename must be..: real.jpg
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8081

MJPG Streamer Version: svn rev: 3:172M
 o: username:password.: disabled
 o: commands..........: enabled
 i: folder to watch...: /mnt/RAMDisk/
 i: forced delay......: 0
 i: delete file.......: no, do not delete
 i: filename must be..: thermal.jpg
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled

now we have two MJPEG live streams

see the two web front ends for informations

http://192.168.1.70:8080
http://192.168.1.70:8081

test it with VLC or mplayer etc.

Code: [Select]

http://192.168.1.70:8080/?action=stream
http://192.168.1.70:8081/?action=stream
(7) a simple html web front end

create a index.html in your web folder /var/www

Code: [Select]

$ cat /var/www/index.html
<!DOCTYPE html>
<html>
<head>
<style type="text/css">
body {
text-align:center;
background-color:#000000;
}
.container {
width:100%;
}
.motion1 {
border: 0;
width: 40%;
height: auto;
}
.motion2 {
border: 0;
width: 40%;
height: auto;
}
.clear {
clear:both;
}
</style>
</head>
<body>
<a href="http://192.168.1.70:8080"><img class="motion1" src="http://192.168.1.70:8080/?action=stream" /></a>

<a href="http://192.168.1.70:8081"><img class="motion2" src="http://192.168.1.70:8081/?action=stream" /></a>
</body>
</html>

load the site:

Code: [Select]

http://192.168.1.70

[cynfab]

Hi tomas123,
Wow, you have been busy...
Yea... Thermal Imaging section.
Well I got my new kernel 4.3.3 for the pcduino building with buildroot, I was going to add the v4l2 kernel module, but after your recent experiments, maybe that's
not a good idea to waste any more time on.... I'd much rather a userland solution.

I'll try and replicate your system soon. Busy week ahead.

   ...ken...

[tomas123]

I'd much rather a userland solution.

I read something about the interesting UV4L "User space Video4Linux collection"
http://www.linux-projects.org/modules/sections/index.php?op=viewarticle&artid=16
but I have no idea, how to implement this in our code.

The workaround over the RAMDisk has advantages and disadvantages.
mjpg-streamer waits patiently for the next jpg image.
Therefore you can restart the flir binary without stopping the mjpeg stream!  // edit: restart works with v4l2 driver also 
Otherwise you have more random lags as with /dev/videoX.

For a surveillance camera with a low frame rate is the RAMDisk workaround acceptable.
If you want a nice live stream without lags, than is v4l2 the better solution.
 
The cutted source code flir8k contains a better error correction and a filter for FFC frames.

Code: [Select]

// freeze thermal image if "shutterState"="FFC"
  if (strncmp (&buf85[28+ThermalSize+JpgSize+17],"FFC",3)!=0)
  {
 
I added a nice restart sequence, because sometimes the code doesn't correct start the USB flir camera

$ sudo ./flir8k
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO
...
: Sun Jan 10 14:57:54 2016
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x81:-1 LIBUSB_ERROR_IO
EP 0x83 LIBUSB_ERROR_NO_DEVICE -> reset USB
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO

....
Write successful!
EP 0x02 to be sent: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}
Write successful!
Sent 65 bytes with string: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 33 00 00 00 ef db c1 c1 ]

Write successful!

[Brumby]

a subforum for "Thermal Imaging" 

+1 

[tomas123]

...
flir8i2.c (see attachment) writes all jpg frames for /dev/video2 and /dev/video3 to file
...
But still mjpg_streamer has the same problem with the mjpeg stream 160x120.

Code: [Select]

$ export LD_LIBRARY_PATH=/usr/local/lib

$ mjpg_streamer -i "input_uvc.so -d /dev/video3 -r 160x120" -o "output_http.so -w /usr/local/www"
MJPG Streamer Version: svn rev: Unversioned directory
 i: Using V4L2 device.: /dev/video3
 i: Desired Resolution: 160 x 120
 i: Frames Per Second.: 5
i: Format............: MJPEG
...
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
*** glibc detected *** mjpg_streamer: free(): invalid next size (normal): 0x01258910 ***
Aborted
Is 160x120 to small for mjpg_streamer?

I found a nice workaround for mjpg_streamer with the v4l2 input modul and 160x120 frame size. 

only enlarge the initialisation frame size of the v4l2 device from 160x120 to:

Code: [Select]

#define VIDEO_DEVICE2 "/dev/video3" // colorized thermal image
#define FRAME_WIDTH2  320
#define FRAME_HEIGHT2 240

than you can write a smaller jpeg of 160x120 to VIDEO_DEVICE2

Code: [Select]

  // write frame as jpg to fdwr2         
...
  int image_width=160; /* Number of columns in image */
  int image_height=120; /* Number of rows in image */
  int quality=90;

and mjpg_streamer works fine with v4l2 loopback device (also VLC etc.)

Code: [Select]

./mjpg_streamer -i "./input_uvc.so -d /dev/video2" -o "./output_http.so -p 8080 -w /usr/local/www" &
./mjpg_streamer -i "./input_uvc.so -d /dev/video3" -o "./output_http.so -p 8081 -w /usr/local/www"
 

Trying to get a new kernel built for my pcduino, and having some toolchain problems.

I know this point from my early cross compile experiences for embedded devices like router, NAS etc.
Last years I'm lazy and I mostly load a ready image for the raspberry pi.
Unfortunately currently there are no 100% corresponding kernel sources for the last raspberry pi image. It's a pain for compiling kernel drivers.   

[cynfab]

tomas123,
Been there done that.... Cross compiling kernel drivers is a PITA.....
I used to be better at it than I am now... been lazy lately too, 
Fixed the toolchain problems, cockpit problem again, kernel now compiling and running...
Then Linus releases 4.4 LTS so I need to repeat it all again with the new kernel code.

   ...ken...

[tomas123]

http://www.linksprite.com/linksprite-pcduino/

What are the advantages of the pcduino, which do you need versus a raspberry pi?
Flash?

for other readers: on the right side is a pcDuino3

[cynfab]

tomas123,
A bit off topic for the FlirOne Android thread but not tooooo far off.
I have several sbc's, and am always looking for a better "bang for the buck".
Prices keep falling and capabilities keep getting better.
I bought the pcduino3 nano lite just before Xmas, at $US15 delivered from Amazon.
Unfortunately that was some sort of promotion, and the real price is more like $US40.
So it is not as good a deal as first thought. Oh well... I'll buy one of almost any SBC if I think it's worth evaluating.
My collection of SBC's is like Frasers collection of thermal cameras (see not quite off topic )
It is still a nice board with fairly good kernel support.
I also have 2 RPi model B boards which I got in the early days of the RPi.
IMHO their USB core in the SOC stinks. Their USB power system on the board stinks.
This may have improved a bit in later versions RPi2.

[tomas123]

Hi cynfab,

I also have 2 RPi model B boards which I got in the early days of the RPi.
IMHO their USB core in the SOC stinks. Their USB power system on the board stinks.
This may have improved a bit in later versions RPi2.

I have also a "nice" collection of RPi A, RPi B and now RPi 2.
The USB power of RPi 2 is noticeable more better. We get no more low power resets after plug in a 300mA USB device.

I found a nice workaround for mjpg_streamer with the v4l2 input modul and 160x120 frame size. 

only enlarge the initialisation frame size of the v4l2 device from 160x120 to:

Code: [Select]

#define VIDEO_DEVICE2 "/dev/video3" // colorized thermal image
#define FRAME_WIDTH2  320
#define FRAME_HEIGHT2 240


as alternative for the RAMdisk version flir8k.c https://www.eevblog.com/forum/thermal-imaging/question-about-flir-one-for-android/msg839518/#msg839518

I uploaded as attachment the last version flir8k.c for v4l2 loopback devices with the better error correction, FFC filter etc.

Code: [Select]

$ make
gcc -I/usr/include/libusb-1.0    -c -o flir8l.o flir8l.c
gcc -I/usr/include/libusb-1.0 -o flir8l Palettes.o flir8l.o -lusb-1.0 -ljpeg -lm -Wall

$ sudo ./flir8l
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
using output device: /dev/video1
     vid_format->type                =2
     vid_format->fmt.pix.width       =160
     vid_format->fmt.pix.height      =120
     vid_format->fmt.pix.pixelformat =1497715271
     vid_format->fmt.pix.sizeimage   =20480
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=160
     vid_format->fmt.pix.colorspace  =8
using output device: /dev/video2
     vid_format->type                =2
     vid_format->fmt.pix.width       =640
     vid_format->fmt.pix.height      =480
     vid_format->fmt.pix.pixelformat =1196444237
     vid_format->fmt.pix.sizeimage   =1228800
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=0
     vid_format->fmt.pix.colorspace  =8
using output device: /dev/video3
     vid_format->type                =2
     vid_format->fmt.pix.width       =320
     vid_format->fmt.pix.height      =240
     vid_format->fmt.pix.pixelformat =1196444237
     vid_format->fmt.pix.sizeimage   =307200
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=0
     vid_format->fmt.pix.colorspace  =8
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO
...

use second terminal to start twice mjpg-streamers

Code: [Select]

~/MJPG-Streamer/test/mjpg-streamer/mjpg-streamer $ cat ./start1.sh
./mjpg_streamer -i "./input_uvc.so -d /dev/video2" -o "./output_http.so -p 8080 -w /usr/local/www"&
./mjpg_streamer -i "./input_uvc.so -d /dev/video3" -o "./output_http.so -p 8081 -w /usr/local/www"

~/MJPG-Streamer/test/mjpg-streamer/mjpg-streamer $ ls
CHANGELOG      input_testpicture.so  LICENSE   mjpg_streamer    mjpg_streamer.h  output_file.so  output_udp.so  README   start1.sh  TODO     utils.h  www
input_file.so  input_uvc.so          Makefile  mjpg_streamer.c  mjpg_streamer.o  output_http.so  plugins        scripts  start.sh   utils.c  utils.o

~/MJPG-Streamer/test/mjpg-streamer/mjpg-streamer $ ./start1.sh
MJPG Streamer Version: svn rev: 3:172
 i: Using V4L2 device.: /dev/video2
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
 
MJPG Streamer Version: svn rev: 3:172
 i: Using V4L2 device.: /dev/video3
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
 i: The format asked unavailable, so the width 320 height 240
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8081
 o: username:password.: disabled
 o: commands..........: enabled

next step like here

now we have two MJPEG live streams

see the two web front ends for informations

http://192.168.1.70:8080
http://192.168.1.70:8081

test it with VLC or mplayer etc.

Code: [Select]

http://192.168.1.70:8080/?action=stream
http://192.168.1.70:8081/?action=stream
(7) a simple html web front end

create a index.html in your web folder /var/www

Code: [Select]

$ cat /var/www/index.html
<!DOCTYPE html>
<html>
<head>
<style type="text/css">
body {
text-align:center;
background-color:#000000;
}
.container {
width:100%;
}
.motion1 {
border: 0;
width: 40%;
height: auto;
}
.motion2 {
border: 0;
width: 40%;
height: auto;
}
.clear {
clear:both;
}
</style>
</head>
<body>
<a href="http://192.168.1.70:8080"><img class="motion1" src="http://192.168.1.70:8080/?action=stream" /></a>

<a href="http://192.168.1.70:8081"><img class="motion2" src="http://192.168.1.70:8081/?action=stream" /></a>
</body>
</html>

load the site:

Code: [Select]

http://192.168.1.70

[tomas123]

Hi cynfab,

the next version for v4l2:
I extend the thermal image with 8 lines to 128x160 pixel.
Now is the height divisible by 16 (avoid video error: "not multiples of 16") and we have place for an information line.

With knowledge of the formulas in my excel sheet RAW2Temp.xls
(attachment of https://www.eevblog.com/forum/thermal-imaging/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072)
I calculated the min/max temperature from the image and wrote some nice hot spot markers on the edges.
( thanks to tmbinc for the font.h the the code snippet  ,  see https://www.eevblog.com/forum/thermal-imaging/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344 )

For calculating the correct temperature you need the calibration values from your Flir One.
You can extract it from your CameraFiles.zip at EP 0x83, but more simply is to use an jpg image from the FlirOne.app and feed exiftool:

Code: [Select]

$ exiftool  -plan*  FLIROne-2015-11-30-17-26-48+0100.jpg
Planck R1                       : 16438.1
Planck B                        : 1416.5
Planck F                        : 1
Planck O                        : -1315
Planck R2                       : 0.0123569
and write it to the c source code

Code: [Select]

// -- define Flir calibration values ---------------
// exiftool -plan* FLIROne-2015-11-30-17-26-48+0100.jpg
#define  PlanckR1  16438.1
#define  PlanckB  1416.5
#define  PlanckF  1.0
#define  PlanckO  -1315.0
#define  PlanckR2  0.0123569please note that I wrote always a decimal point, to avoid some unwanted integer castings in c

I wrote here dozens of post about the flir formulas, therefore only the code without comments:

Code: [Select]

double raw2temperature(unsigned short RAW)
{
 // mystery correction factor
 RAW *=4;
 // calc amount of radiance of reflected objects ( Emissivity < 1 )
 double RAWrefl=PlanckR1/(PlanckR2*(exp(PlanckB/(TempReflected+273.15))-PlanckF))-PlanckO;
 // get displayed object temp max/min
 double RAWobj=(RAW-(1-Emissivity)*RAWrefl)/Emissivity;
 // calc object temperature
 return PlanckB/log(PlanckR1/(PlanckR2*(RAWobj+PlanckO))+PlanckF)-273.15; 
}

There is a uncleared issue. With the FlirOne.App we get the correct Flir RAW values for the Excel Sheet above.
But over USB the RAW values seems divided by factor 4.
I'm unsure and will further check it with some temperature measurements.


Currently the string with min/max temperature is only for temperature below 100°C  because I forgot to truncate the strings to 20 chars   

[cynfab]

Hi tomas123,
Nice

I moved the plank constants to a header file so that the flir8xx.c code can be clean of individual F1 G2 characteristics.

Have you been able to view the video stream with guvcview, or mplayer?  .. see attached.
mplayer looks similar.

   ...ken...

[tomas123]

this is only a side effect of the mjpeg streamer workaround

I found a nice workaround for mjpg_streamer with the v4l2 input modul and 160x120 frame size. 

only enlarge the initialisation frame size of the v4l2 device from 160x120 to:

Code: [Select]

#define VIDEO_DEVICE2 "/dev/video3" // colorized thermal image
#define FRAME_WIDTH2  320
#define FRAME_HEIGHT2 240

than you can write a smaller jpeg of 160x120 to VIDEO_DEVICE2

the solution for guvcview or mplayer:

unload and reload the v4l2loopback kernel driver

Code: [Select]

// unload
$ sudo modprobe -r v4l2loopback
// load
$ sudo modprobe v4l2loopback devices=5
change code to the exact image size

Code: [Select]

#define VIDEO_DEVICE2 "/dev/video3" // colorized thermal image
#define FRAME_WIDTH2   160 //320
#define FRAME_HEIGHT2  128 //240
compile and guvcview / mplayer works fine


in all your screenshots I see, that your frame rate is around 8.2 fps
I have a later FlirOne hardware version. My frame rate ist mostly around 7.9fps 

Code: [Select]

#00007305 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.42668,"usbEnqueuedTimestamp":1177508916.4285,"ffcState":"FFC_VALID_RAD"}
#00007306 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.5428,"usbEnqueuedTimestamp":1177508916.54466,"ffcState":"FFC_VALID_RAD"}
#00007307 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.65624,"usbEnqueuedTimestamp":1177508916.65782,"ffcState":"FFC_VALID_RAD"}
#00007308 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.76904,"usbEnqueuedTimestamp":1177508916.77088,"ffcState":"FFC_VALID_RAD"}
#00007309 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.88276,"usbEnqueuedTimestamp":1177508916.88477,"ffcState":"FFC_VALID_RAD"}
#00007310 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508916.99627,"usbEnqueuedTimestamp":1177508916.99783,"ffcState":"FFC_VALID_RAD"}
#00007311 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.11095,"usbEnqueuedTimestamp":1177508917.11306,"ffcState":"FFC_VALID_RAD"}
#00007312 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.22461,"usbEnqueuedTimestamp":1177508917.22623,"ffcState":"FFC_VALID_RAD"}
#00007313 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.34107,"usbEnqueuedTimestamp":1177508917.3429,"ffcState":"FFC_VALID_RAD"}
#00007314 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.4537,"usbEnqueuedTimestamp":1177508917.45972,"ffcState":"FFC_VALID_RAD"}
#00007315 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.56661,"usbEnqueuedTimestamp":1177508917.56837,"ffcState":"FFC_VALID_RAD"}
#00007316 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.68063,"usbEnqueuedTimestamp":1177508917.6869,"ffcState":"FFC_VALID_RAD"}
#00007317 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.79463,"usbEnqueuedTimestamp":1177508917.79671,"ffcState":"FFC_VALID_RAD"}
#00007318 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508917.91498,"usbEnqueuedTimestamp":1177508917.91678,"ffcState":"FFC_VALID_RAD"}
#00007319 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.0219,"usbEnqueuedTimestamp":1177508918.02369,"ffcState":"FFC_VALID_RAD"}
#00007320 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.13852,"usbEnqueuedTimestamp":1177508918.14039,"ffcState":"FFC_VALID_RAD"}
#00007321 81/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.25196,"usbEnqueuedTimestamp":1177508918.25356,"ffcState":"FFC_VALID_RAD"}
#00007322 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.36496,"usbEnqueuedTimestamp":1177508918.3667,"ffcState":"FFC_VALID_RAD"}
#00007323 81/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.4781,"usbEnqueuedTimestamp":1177508918.48,"ffcState":"FFC_VALID_RAD"}
#00007324 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.59206,"usbEnqueuedTimestamp":1177508918.59365,"ffcState":"FFC_VALID_RAD"}
#00007325 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.70655,"usbEnqueuedTimestamp":1177508918.70863,"ffcState":"FFC_VALID_RAD"}
#00007326 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.82,"usbEnqueuedTimestamp":1177508918.82159,"ffcState":"FFC_VALID_RAD"}
#00007327 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508918.93411,"usbEnqueuedTimestamp":1177508918.93615,"ffcState":"FFC_VALID_RAD"}
#00007328 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.04937,"usbEnqueuedTimestamp":1177508919.05093,"ffcState":"FFC_VALID_RAD"}
#00007329 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.16221,"usbEnqueuedTimestamp":1177508919.16431,"ffcState":"FFC_VALID_RAD"}
#00007330 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.27651,"usbEnqueuedTimestamp":1177508919.28135,"ffcState":"FFC_VALID_RAD"}
#00007331 81/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.38965,"usbEnqueuedTimestamp":1177508919.39117,"ffcState":"FFC_VALID_RAD"}
#00007332 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.50482,"usbEnqueuedTimestamp":1177508919.50864,"ffcState":"FFC_VALID_RAD"}
#00007333 81/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.61759,"usbEnqueuedTimestamp":1177508919.61911,"ffcState":"FFC_VALID_RAD"}
#00007334 81/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.73212,"usbEnqueuedTimestamp":1177508919.73392,"ffcState":"FFC_VALID_RAD"}
#00007335 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.8459,"usbEnqueuedTimestamp":1177508919.84773,"ffcState":"FFC_VALID_RAD"}
#00007336 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508919.96018,"usbEnqueuedTimestamp":1177508919.96197,"ffcState":"FFC_VALID_RAD"}
#00007337 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.07373,"usbEnqueuedTimestamp":1177508920.07574,"ffcState":"FFC_VALID_RAD"}
#00007338 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.18741,"usbEnqueuedTimestamp":1177508920.18898,"ffcState":"FFC_VALID_RAD"}
#00007339 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.30213,"usbEnqueuedTimestamp":1177508920.30423,"ffcState":"FFC_VALID_RAD"}
#00007340 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.41558,"usbEnqueuedTimestamp":1177508920.41705,"ffcState":"FFC_VALID_RAD"}
#00007341 80/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.52954,"usbEnqueuedTimestamp":1177508920.53154,"ffcState":"FFC_VALID_RAD"}
#00007342 79/10 fps:{"shutterState":"ON","shutterTemperature":310.799987792969,"usbNotifiedTimestamp":1177508920.64426,"usbEnqueuedTimestamp":1177508920.6458,"ffcState":"FFC_VALID_RAD"}


[cynfab]

Hi tomas123,

Your latest code is now cross compiled and running with mjpg-streamer on my pcduino3 nano lite. kernel 4.3.3
Wow, there are a lot of forks of mpg-streamer... close to 150!!
https://github.com/jacksonliam/mjpg-streamer   146 forks
https://github.com/oliv3r/mjpg-streamer    2 forks
and maybe more...

I'm getting 7.9-8.1 fps on the pcduino3.
And the occasional reset buffer because of bad Magic Byte.
The original 160 x 120 for video3 works with guvcview and mplayer.

  ...ken...

[tomas123]

 
if we debug the 160x120 resolution bug, than we can fork #151 

Code: [Select]

// mystery correction factor
 RAW *=4;
I checked it in my kitchen from -20°C to +100°C (boiling water). Tricky is to compare the temperature from our display (hottest point) with Flir crosshair (Flir One App) as medium of 4x4 pixel.
But it seems, that the correction factor works fine.
Please make own tests with your calibration values.

Maybe this is a effect of the conversion factor from 14 Bit (Sensor) to 16 Bit (2 Byte RAW)

[tomas123]

I found a simple solution to check the factor 4 between "Lepton RAW" and "Flir radiometric jpg RAW":

The tea bag simulator of Flir SDK use real Lepton sensor values. You found it in the SDK file sampleframes.zip (6MB).
I made with the SDK App a shot from the (random) tea bag frame 00051.
Thereby we got a Flir radiometric jpg with an embedded calibrated RawThermalImage with a good temperature range from min/max=34°C/65°C
(the unrealistic min temperature is a result of bad calibration values for the simulator)

first some image magick steps with the lepton sensor values of frame 00051 (file 00051-lep) from SDK sampleframes.zip
("-chop 2x0+0+0 -chop 2x0+80+0" removes the 4 additional vertical lines with extra unknown informations)

Code: [Select]

// convert sample frame 00051 from SDK
> convert -depth 16 -size 164x120 gray:00051-lep gray:- | convert -depth 16 -endian lsb -size 164x120 gray:- -chop 2x0+0+0 -chop 2x0+80+0 -rotate 90 00051-lep.png
// multiply with factor 4
> convert 00051-lep.png -fx "4*u" 00051-lep4x.png
// get channel statistics
> identify -verbose 00051-lep4x.png
  Channel statistics:
    Pixels: 19200
    Gray:
      min: 14220 (0.216983)
      max: 20840 (0.317998)
      mean: 15014.9 (0.229113)
      standard deviation: 1218.95 (0.0186)
      kurtosis: 5.54453
      skewness: 2.30479
and now compare the statistics with the "real" Flir radiometric image from the SDK app

Code: [Select]

// extract RawThermalImage from a Flir radiometric image
> exiftool -b -RawThermalImage FLIROne-2016-01-18-10-33-07+0100.jpg > 00051exif.png
// image is postprocessed and resized to 320x240
> convert 00051exif.png -resize 160x FlirAppRAW120x160.png
// get channel statistics
>identify -verbose FlirAppRAW120x160.png
...
  Channel statistics:
    Pixels: 19200
    Gray:
      min: 14266 (0.217685)
      max: 20831 (0.317861)
      mean: 15021.4 (0.229212)
      standard deviation: 1221.27 (0.0186354)
      kurtosis: 5.44004
      skewness: 2.28451
By deducting the Flir post processing steps to the original RAW values (see patterns below) we got a great result!
A sample:

max-min=65-34=31°C
max-min=20831-14266=6565 digits
-> 1 digit = 31/6565=0.0047 Kelvin
The difference between mean in this images is:
(15021.4-15014.9)*0.0047 Kelvin= 0.031 Kelvin 

You also see, that the last digit ADC "resolution" of Lepton in this range is > 0.0047*4 = 19mK (useless because of strong noise, see second image below with 4 Kelvin scale)

here is a real live sample from Flir One G2 (shot after a small warm up time of about 2 minutes):

I saved with the SDK.app simultaneously a upscaled Flir Radiometric JPG  and a real Lepton ThermalLinearFlux14BitImage.

Afterwards I rebuild with my old panorama script (see my footer) a real size 160x120 Lepton radiometric jpg (a Flir format).
You can load this sample jpg images in Flir Tools and compare the quality.

First a original image shot with the Flir App.
The App crop    the Lepton sensor to about 120x90 Pixel.
Please note the artefacts/patterns!
Flir makes a nice lens distortion correction of the Lepton sensor for best MSX overlaying 




real  Lepton sensor 160x120 (no image postprocessing and with noise/grain because the temperature spread is only 4 Kelvin)

[tomas123]

minor update:

- a smaller font 5x7 for more chars in information line
- better filtering FFC frames
- crosshair for medium temperature of 2x2 "center" pixels
- info line with temperature  min/crosshair/max

[cynfab]

tomas123,

Very nice