Question about FLIR One for Android

[tomas123]

I saw, there is a typo in line 245 of file flir8f.c.

changed

Code: [Select]

   write(fdwr1, &buf[28+ThermalSize], JpgSize);to

Code: [Select]

   write(fdwr1, &buf85[28+ThermalSize], JpgSize);
edit
... and a memory leak 

Code: [Select]

  // write video to v4l2loopback
   write(fdwr0, fb_proc, framesize0);
   write(fdwr1, &buf85[28+ThermalSize], JpgSize);
   
   free(fb_proc);

[cynfab]

Hi tomas123,
This looks really good, better than my last effort that just hoped to get a full frame.
Have you had any luck getting video1 to stream the visual jpg data?
I've poked at it and still have no clue as to what it should do. The v4l2 api is very dense reading. and I am struggling with trying to understand what I need to do.
But, it looks like things are progressing.

   ...ken...

[tomas123]

Have you had any luck getting video1 to stream the visual jpg data?

Yes, my last code (found typo buff85 and the memory leak) runs since two days without errors on /dev/video0 and /dev/video1 on my raspberry pi.

after 1.309.405 frames with 8,2 fps:
(1.309.405/8,2/3600=44 hours)

Code: [Select]

#1309405 82/10 fps:{"shutterState":"ON","shutterTemperature":311.869995117188,"usbNotifiedTimestamp":1176080489.24421,"usbEnqueuedTimestamp":1176080489.24568,"ffcState":"FFC_VALID_RAD"}
#1309406 83/10 fps:{"shutterState":"ON","shutterTemperature":311.869995117188,"usbNotifiedTimestamp":1176080489.35792,"usbEnqueuedTimestamp":1176080489.35936,"ffcState":"FFC_VALID_RAD"}
#1309407 82/10 fps:{"shutterState":"ON","shutterTemperature":311.880004882812,"usbNotifiedTimestamp":1176080489.4717,"usbEnqueuedTimestamp":1176080489.47328,"ffcState":"FFC_VALID_RAD"}
#1309408 82/10 fps:{"shutterState":"ON","shutterTemperature":311.880004882812,"usbNotifiedTimestamp":1176080489.5864,"usbEnqueuedTimestamp":1176080489.58808,"ffcState":"FFC_VALID_RAD"}
#1309409 82/10 fps:{"shutterState":"ON","shutterTemperature":311.880004882812,"usbNotifiedTimestamp":1176080489.70209,"usbEnqueuedTimestamp":1176080489.70338,"ffcState":"FFC_VALID_RAD"}

With Guvcview I can view both videos simultaneously with perfect quality (no drops and artefacts etc).
Can you exact describe your problems with /dev/video1 ?

My next step is to send both video (thermal/real) with RTSP streaming (ffmpeg) to a Synology surveillance station ...

edit:

Do you remember, how many time I wasted on the wrong system (Linux VM, OS X)?
Apparently the FlirOne G2 USB connection is tricky...
 

I played around with some time outs and found this as the best solution for my raspberry pi:

Code: [Select]

r = libusb_bulk_transfer(devh, 0x85, buf, sizeof(buf), &actual_length, 100);
...
r = libusb_bulk_transfer(devh, 0x81, buf, sizeof(buf), &actual_length, 10);
r = libusb_bulk_transfer(devh, 0x83, buf, sizeof(buf), &actual_length, 10); EP 0x81 and 0x83 are almost empty (time out of 10ms works fine) and EP 0x85 lose regularly some bytes between chunks, if the time out is shorter as 100ms.

if you see in the terminal this correct status string, then you know, that all chunks are correct stitched to a full frame without loosing bytes
(the status string is the end of a full frame)

Code: [Select]

#1309405 82/10 fps:{"shutterState":"ON","shutterTemperature":311.869995117188,"usbNotifiedTimestamp":1176080489.24421,"usbEnqueuedTimestamp":1176080489.24568,"ffcState":"FFC_VALID_RAD"}

[cynfab]

Hi tomas123,
I start the videoloopback module like this:

sudo modprobe v4l2loopback exclusive_caps=0,0 video_nr=0,1

then run flir8g, then guvcview.
THis then displays the thermal image.
I then go to video controls and select device 1.
Guvcview then gives an error that it can't start a video stream on that device... see attached screenshot.

I can certainly believe that the F1G2 USB is odd as I have continuous problems on Android with various apps not being able to claim the usb interface.
I believe all the chunks are being stiched as the screenshot shows the terminal output.

   ...ken...

[cynfab]

And, this is what I get with mplayer...

[tomas123]

hm, my mplayer also works fine

after first 400 frames (ignore the warnings):

Code: [Select]

$ mplayer tv:// -tv driver=v4l2:device=/dev/video1
MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing tv://.
TV file format detected.
Selected driver: v4l2
 name: Video 4 Linux 2 input
 author: Martin Olschewski <olschewski@zpr.uni-koeln.de>
 comment: first try, more to come ;-)
v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.
Selected device: Dummy video device (0x0001)
 Capabilities:  video capture  read/write  streaming
 supported norms:
 inputs: 0 = loopback;
 Current input: 0
 Current format: MJPEG
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
v4l2: ioctl set format failed: Invalid argument
tv.c: norm_from_string(pal): Bogus norm parameter, setting default.
v4l2: ioctl enum norm failed: Inappropriate ioctl for device
Error: Cannot set norm!
Selected input hasn't got a tuner!
v4l2: ioctl set mute failed: Invalid argument
v4l2: ioctl query control failed: Invalid argument
v4l2: ioctl query control failed: Invalid argument
v4l2: ioctl query control failed: Invalid argument
v4l2: ioctl query control failed: Invalid argument
Xlib:  extension "XFree86-VidModeExtension" missing on display "localhost:10.0".
[VO_XV] It seems there is no Xvideo support for your video card available.
[VO_XV] Run 'xvinfo' to verify its Xv support and read
[VO_XV] DOCS/HTML/en/video.html#xv!
[VO_XV] See 'mplayer -vo help' for other (non-xv) video out drivers.
[VO_XV] Try -vo x11.
[gl] using extended formats. Use -vo gl:nomanyfmts if playback fails.
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 53.35.0 (external)
Mismatching header version 53.32.2
Selected video codec: [ffmjpeg] vfm: ffmpeg (FFmpeg MJPEG)
==========================================================================
Audio: no sound
Starting playback...
Movie-Aspect is undefined - no prescaling applied.
VO: [gl_nosw] 640x480 => 640x480 Planar 422P
V:   0.0 399/399 ??% ??% ??,?% 0 0


Your screenshot shows bugless "full frames", therefore our code works fine.
It's possible, that you have trouble with your v4l2 drivers.

Please post your output from this commands:

Code: [Select]

$ ls -al /dev/video*
$ v4l2-ctl --list-devices
$ v4l2-ctl --all -d /dev/video0
$ v4l2-ctl --all -d /dev/video1

compare with my result

Code: [Select]

pi@raspberryLCD ~ $ ls -al /dev/video*
crw-rw---T+ 1 root video 81, 0 Dec 27 13:12 /dev/video0
crw-rw---T+ 1 root video 81, 1 Dec 27 13:12 /dev/video1

pi@raspberryLCD ~ $ v4l2-ctl --list-devices
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video0

Dummy video device (0x0001) (platform:v4l2loopback-001):
/dev/video1

pi@raspberryLCD ~ $ v4l2-ctl --all -d /dev/video0
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0000)
Bus info      : platform:v4l2loopback-000
Driver version: 3.18.13
Capabilities  : 0x85208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x05208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Video input : 0 (loopback: ok)
Format Video Capture:
Width/Height  : 160/120
Pixel Format  : 'GREY'
Field         : None
Bytes per Line: 160
Size Image    : 19200
Colorspace    : SRGB
Custom Info   : feedcafe
Format Video Output:
Width/Height  : 160/120
Pixel Format  : 'GREY'
Field         : None
Bytes per Line: 160
Size Image    : 19200
Colorspace    : SRGB
Custom Info   : feedcafe
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers    : 8

User Controls

                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0

pi@raspberryLCD ~ $ v4l2-ctl --all -d /dev/video1
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0001)
Bus info      : platform:v4l2loopback-001
Driver version: 3.18.13
Capabilities  : 0x85208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x05208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Video input : 0 (loopback: ok)
Format Video Capture:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Format Video Output:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers    : 8

User Controls

                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0


[tomas123]

there are some interesting differences between our mplayer output lines

my mplayer:

Code: [Select]

Current format: MJPEG
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
...
VO: [gl_nosw] 640x480 => 640x480 Planar 422P
your mplayer shows another video format

Code: [Select]

Current format: BGR32
Opening video decoder: [raw] RAW uncompressed video
...
VO: [vdpau] 640x480 => 640x480 Planar YV12
maybe, there is a another video driver on your system
reboot, check it and try to write to another port (like video2) or change in our code video0<->video1

PS: another difference:

please note, you must load v4l2loopback with /dev/video0 and /dev/video1

Code: [Select]

// unload v4l2loopback
$ sudo modprobe -r v4l2loopback

$ ls /dev/video*
ls: cannot access /dev/video*: No such file or directory

$ sudo modprobe v4l2loopback video_nr=0,1

$ ls /dev/video*
/dev/video0  /dev/video1


Why do you needs the switch "exclusive_caps"?

I start the videoloopback module like this:

sudo modprobe v4l2loopback exclusive_caps=0,0 video_nr=0,1

https://github.com/umlaeute/v4l2loopback/blob/master/NEWS

v4l2loopback-0.7.0

  - experimental'exclusive_caps' mode that only reports CAPTURE/OUTPUT
    capabilities exclusively (support for Chromium/WebRTC)

see also https://github.com/umlaeute/v4l2loopback/issues/78

[cynfab]

Hi tomas123,

Well, there are quite a few differences between our v4l2loopback stuff.

Code: [Select]

kenm@kenm-Studio-XPS-8000 ~/FLIR_ONE/thomas123/New_test/v4l $ ls -al /dev/video*
crw-rw----+ 1 root video 81, 0 Dec 29 10:44 /dev/video0
crw-rw----+ 1 root video 81, 1 Dec 29 10:44 /dev/video1
kenm@kenm-Studio-XPS-8000 ~/FLIR_ONE/thomas123/New_test/v4l $ v4l2-ctl --list-devices
Dummy video device (0x0000) (v4l2loopback:0):
/dev/video0

Dummy video device (0x0001) (v4l2loopback:1):
/dev/video1

kenm@kenm-Studio-XPS-8000 ~/FLIR_ONE/thomas123/New_test/v4l $ v4l2-ctl --all -d /dev/video0
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0000)
Bus info      : v4l2loopback:0
Driver version: 0.8.0
Capabilities  : 0x05000002
Video Output
Read/Write
Streaming
Video output: 0 (loopback in)
Format Video Output:
Width/Height  : 160/120
Pixel Format  : 'GREY'
Field         : None
Bytes per Line: 160
Size Image    : 19200
Colorspace    : SRGB
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers    : 8
                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0
kenm@kenm-Studio-XPS-8000 ~/FLIR_ONE/thomas123/New_test/v4l $  v4l2-ctl --all -d /dev/video1
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0001)
Bus info      : v4l2loopback:1
Driver version: 0.8.0
Capabilities  : 0x05000003
Video Capture
Video Output
Read/Write
Streaming
Video input : 0 (loopback: ok)
Video output: 0 (loopback in)
Format Video Capture:
Width/Height  : 640/480
Pixel Format  : 'BGR4'
Field         : None
Bytes per Line: 2560
Size Image    : 1228800
Colorspace    : SRGB
Format Video Output:
Width/Height  : 640/480
Pixel Format  : 'BGR4'
Field         : None
Bytes per Line: 2560
Size Image    : 1228800
Colorspace    : SRGB
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers    : 8
                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0

So it looks like maybe this is a versioning problem, because I am using what comes with Linux Mint17.3 and you are maybe using something compiled from source or at least a more recent version.

I think I'll try getting v4l2loopback from github and replace what comes with Mint17.3.

I've tried using /dev/video2 and it makes no difference.
I've tried with and without  "exclusive_caps", and it makes no difference.

"Driver version: 3.18.13"
vs my:
"Driver version: 0.8.0"
worries me.

As does my:
Current format: BGR32

   ...ken...

[tomas123]

my suggest:

try to write the real video to /dev/video0 and thermal to /dev/video1 and see what happens

Code: [Select]

#define VIDEO_DEVICE0 "/dev/video1"
#define FRAME_WIDTH0  160
#define FRAME_HEIGHT0 120

#define VIDEO_DEVICE1 "/dev/video0"
#define FRAME_WIDTH1  640
#define FRAME_HEIGHT1 480

because we write

Code: [Select]

#define FRAME_FORMAT1 V4L2_PIX_FMT_MJPEGand you get

Code: [Select]

Pixel Format  : 'BGR4'instead

Code: [Select]

Pixel Format  : 'MJPG'

BGR4 = V4L2_PIX_FMT_BGR32


version 3.18.13 looks like my kernel version on raspberry   

Code: [Select]

$ v4l2-ctl --info
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0000)
Bus info      : platform:v4l2loopback-000
Driver version: 3.18.13
Capabilities  : 0x85208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x05208001
Video Capture
Video Memory-to-Memory
Read/Write
Streaming


Code: [Select]

$ uname -a
Linux raspberryLCD 3.18.13-v7+ #784 SMP PREEMPT Sat May 9 15:57:36 BST 2015 armv7l GNU/Linux

I posted my installations steps here:

the next steps for installing v4l2loopback are simple

Code: [Select]

$ uname -a
Linux raspberryLCD 3.18.0-trunk-rpi2 #1 SMP PREEMPT Debian 3.18.5-1~exp1+rpi19 (2015-08-08) armv7l GNU/Linux

$ git clone https://github.com/umlaeute/v4l2loopback

$ cd v4l2loopback

$ make
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
  CC [M]  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.mod.o
  LD [M]  /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'

$ sudo su

# make install
make -C /lib/modules/`uname -r`/build M=/home/pi/v4l2loopback/v4l2loopback modules_install
make[1]: Entering directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
Makefile:10: *** mixed implicit and normal rules: deprecated syntax
  INSTALL /home/pi/v4l2loopback/v4l2loopback/v4l2loopback.ko
  DEPMOD  3.18.0-trunk-rpi2
make[1]: Leaving directory '/usr/src/linux-headers-3.18.0-trunk-rpi2'
depmod -a  `uname -r`

$ sudo modprobe v4l2loopback

$ lsmod
Module                  Size  Used by
v4l2loopback           23372  0

...

[cynfab]

Just changing

Code: [Select]

#define VIDEO_DEVICE0 "/dev/video1"
#define FRAME_WIDTH0  160
#define FRAME_HEIGHT0 120

#define VIDEO_DEVICE1 "/dev/video0"
#define FRAME_WIDTH1  640
#define FRAME_HEIGHT1 480
 video1 is recognized by Guvcview and mplayer displays the thermal image, video0 is garbage


...disregard the below screenshot...

[tomas123]

Hi cynfab,

2 hours work:
I installed a Linux Mint Image on a clean notebook and note all steps 

(for another readers: Linux Mint as a Ubuntu fork)


$ uname -a
Linux mint-Lenovo-B50-70 3.19.0-32-generic #37~14.04.1-Ubuntu SMP Thu Oct 22 09:41:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

$ inxi -S
System:    Host: mint-Lenovo-B50-70 Kernel: 3.19.0-32-generic x86_64 (64 bit) Desktop: N/A
           Distro: Linux Mint 17.3 Rosa

$ sudo apt-get install g++

// check pre-installed libusb versions
$ dpkg --get-selections | grep libusb
libusb-0.1-4:amd64install
libusb-1.0-0:amd64install
libusb-1.0-0:i386install
libusbmuxd2install

// we need the header file libusb.h
$ sudo apt-get install libusb-1.0-0-dev

// check the path of libusb.h for gcc command line below
$ find / -name libusb.h 2>/dev/zero
/usr/include/libusb-1.0/libusb.h

// I see my notebook webcam as device video0
$ ls /dev/video*
/dev/video0

// there is no v4l2loopback driver installed on clean Linux Mint
$ lsmod | grep v4l2
v4l2_common            16384  1 videobuf2_core
videodev              159744  3 uvcvideo,v4l2_common,videobuf2_core

$ sudo modprobe v4l2loopback
modprobe: FATAL: Module v4l2loopback not found.

// therefore me must compile the driver for our kernel
// first load kernel sources (try to skip this step)
$ sudo apt-get install linux-generic

// 271 MB later: I think this was to much.
// Better load only the kernel headers (as part of the packet linux-generic)
$ sudo apt-get install linux-headers-generic

// try to compile v4l2loopback with packet manager (skip this step)
$ sudo apt-get install v4l2loopback-dkms
...
Building initial module for 3.19.0-32-generic
Error! Bad return status for module build on kernel: 3.19.0-32-generic (x86_64)
Consult /var/lib/dkms/v4l2loopback/0.8.0/build/make.log for more information.

// Argh! We got an Error!!
$ cat /var/lib/dkms/v4l2loopback/0.8.0/build/make.log
..
/var/lib/dkms/v4l2loopback/0.8.0/build/v4l2loopback.c:486:2: error: implicit declaration of function ‘strict_strtoul’ [-Werror=implicit-function-declaration]
  if (strict_strtoul(buf, 0, &curr))
  ^
...
make: *** [v4l2loopback.ko] Error 2

// first remove the broken packet v4l2loopback-dkms...
$ sudo apt-get remove v4l2loopback-dkms
...
Entfernen von v4l2loopback-dkms (0.8.0-1) ...
------------------------------
Deleting module version: 0.8.0
completely from the DKMS tree.
------------------------------

// ... then do it yourself
$ sudo apt-get install git

$ git clone https://github.com/umlaeute/v4l2loopback

$ cd v4l2loopback/

$ make
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/home/mint/Desktop/flir/v4l2loopback modules
make[1]: Verzeichnis »/usr/src/linux-headers-3.19.0-32-generic« wird betreten
  CC [M]  /home/mint/Desktop/flir/v4l2loopback/v4l2loopback.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/mint/Desktop/flir/v4l2loopback/v4l2loopback.mod.o
  LD [M]  /home/mint/Desktop/flir/v4l2loopback/v4l2loopback.ko

$ sudo make install
make -C /lib/modules/`uname -r`/build M=/home/mint/Desktop/flir/v4l2loopback modules_install
  INSTALL /home/mint/Desktop/flir/v4l2loopback/v4l2loopback.ko

// now load video1 and video2 (because my notebook cam is video0)
$ sudo modprobe v4l2loopback video_nr=1,2

// check it - all works fine
$ ls /dev/video*
/dev/video0  /dev/video1  /dev/video2

// edit /dev/video? in  flir8g-mint.c
#define VIDEO_DEVICE0 "/dev/video1"
#define FRAME_WIDTH0  160
#define FRAME_HEIGHT0 120

#define VIDEO_DEVICE1 "/dev/video2"
#define FRAME_WIDTH1  640
#define FRAME_HEIGHT1 480

// compile flir8g-mint.c from attachment
$ /usr/bin/gcc '-I/usr/include/libusb-1.0'  -o flir1 flir8g-mint.c -lusb-1.0 -lm

// and run the code
$ sudo ./flir
...
#000037 68/10 fps:{"shutterState":"ON","shutterTemperature":294.489990234375,"usbNotifiedTimestamp":1176135054.68748,"usbEnqueuedTimestamp":1176135054.68923,"ffcState":"FFC_RAD_APPROX"}
#000038 71/10 fps:{"shutterState":"ON","shutterTemperature":294.489990234375,"usbNotifiedTimestamp":1176135054.8017,"usbEnqueuedTimestamp":1176135054.80321,"ffcState":"FFC_RAD_APPROX"}
#000039 70/10 fps:{"shutterState":"ON","shutterTemperature":294.489990234375,"usbNotifiedTimestamp":1176135054.91573,"usbEnqueuedTimestamp":1176135054.91749,"ffcState":"FFC_RAD_APPROX"}
...

// another session
$ sudo apt-get install v4l-utils

$ v4l2-ctl --list-devices
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video1

Dummy video device (0x0001) (platform:v4l2loopback-001):
/dev/video2

Lenovo EasyCamera (usb-0000:00:14.0-6):
/dev/video0

// check correct formats
$ v4l2-ctl --list-formats-out -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
Index       : 0
Type        : Video Output
Pixel Format: 'GREY'
Name        : 8 bpp, Greyscale

$ v4l2-ctl --list-formats-out -d /dev/video2
ioctl: VIDIOC_ENUM_FMT
Index       : 0
Type        : Video Output
Pixel Format: 'MJPG'
Name        : Motion-JPEG

// install guvcview and mplayer
$ sudo apt-get install guvcview
$ sudo apt-get install mplayer

... and all works fine:


only for comparing the "Driver version":

Code: [Select]

$ v4l2-ctl --all -d /dev/video2
Driver Info (not using libv4l2):
Driver name   : v4l2 loopback
Card type     : Dummy video device (0x0001)
Bus info      : platform:v4l2loopback-001
Driver version: 3.19.8
Capabilities  : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Device Caps   : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Device Capabilities
Video input : 0 (loopback: ok)
Video output: 0 (loopback in)
Format Video Capture:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Format Video Output:
Width/Height  : 640/480
Pixel Format  : 'MJPG'
Field         : None
Bytes per Line: 0
Size Image    : 1228800
Colorspace    : SRGB
Custom Info   : feedcafe
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers     : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers    : 8

User Controls

                    keep_format (bool)   : default=0 value=0
              sustain_framerate (bool)   : default=0 value=0
                        timeout (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io (bool)   : default=0 value=0


[cynfab]

Hi tomas123,
Nice job detailing all of that.

I believe that the issue I am having is the old version of v4l2loopback that comes with Mint17.3's v4l2loopback-dkms.
I'll get a chance to rebuild it later this morning.
I've got to deal with another 3-4" of new snow, so thermal imaging will have to wait a bit

   ...ken...

[tomas123]

I was to curios about your error and thought that it's faster to try it self as write dozen posts in my broken english 

Don't forget to clean your driver:

Code: [Select]

$ dpkg --get-selections | grep v4l2
$ sudo apt-get remove v4l2loopback-dkms

[cynfab]

Thanks,
I cleaned out the old v4l2loopback, but am having issues compiling the new one.
With just a plain make, somehow it thinks I want to cross compile for arm.
Just updated the kernel to 3.19.0-32, but that made no difference,

Code: [Select]

make
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/home/kenm/FLIR_ONE/thomas123/v4l2loopback modules
make[1]: Entering directory `/usr/src/linux-headers-3.19.0-32-generic'
  CC [M]  /home/kenm/FLIR_ONE/thomas123/v4l2loopback/v4l2loopback.o
In file included from include/asm-generic/int-ll64.h:10:0,
                 from ./arch/arm/include/asm/types.h:4,
                 from include/uapi/linux/types.h:4,
                 from include/linux/compiler.h:189,
                 from include/linux/linkage.h:4,
                 from include/linux/preempt.h:9,
                 from include/linux/spinlock.h:50,
                 from include/linux/vmalloc.h:4,
                 from /home/kenm/FLIR_ONE/thomas123/v4l2loopback/v4l2loopback.c:17:
include/uapi/asm-generic/int-ll64.h:11:29: fatal error: asm/bitsperlong.h: No such file or directory
compilation terminated.
It's just one of those days.....

   ...ken...


NOOB mistake... had something set in .profile ARCH=arm

And now flir8g works perfectly!!

[Fraser]

Nice work guys

RESPECT 👍

[cynfab]

Hi tomas123,

For your thermal imaging pleasure, a version that does 3 video streams,
one is greyscale thermal
another is visual
and the third is a colorized version of the thermal image.

   ...ken...

[tomas123]

Hi cynfab,

your lookup table works great.
But first edit your memory leak by freeing fb_proc2.
(Watch the growing virtual memory of process flir8h with top.)

flir8h.c 

Code: [Select]

...
326:   free(fb_proc);
327:   free(fb_proc2);
...


... but we cant select our frame format V4L2_PIX_FMT_GREY in motion.conf 

As nice side effect, we have now a V4L2 thermal stream, which motion can process.
http://www.lavrsen.dk/foswiki/bin/view/Motion/WebHome

$ make
gcc -I/usr/include/libusb-1.0    -c -o flir8h.o flir8h.c
gcc -I/usr/include/libusb-1.0 -o flir8h Palettes.o flir8h.o -lusb-1.0 -lm -Wall

$ sudo ./flir8h

// install motion
$ sudo apt-get install motion

// edit config file
$ sudo nano /etc/motion/motion.conf

Code: [Select]

# Maximum framerate for webcam streams (default: 1)
webcam_maxrate 10

# Restrict webcam connections to localhost only (default: on)
webcam_localhost off

# thread config files.
thread /usr/local/etc/thread1.conf
thread /usr/local/etc/thread2.conf
; thread /usr/local/etc/thread3.conf
; thread /usr/local/etc/thread4.conf
// create two additional config files for /dev/video2 and /dev/video3

Code: [Select]

$ cat /usr/local/etc/thread1.conf
# V4L2_PIX_FMT_MJPEG   : 2  'MJPEG'
# V4L2_PIX_FMT_JPEG    : 3  'JPEG'
# V4L2_PIX_FMT_RGB24   : 4  'RGB3'
v4l2_palette 2
videodevice /dev/video2
target_dir /tmp/motion2
webcam_port 8081

$ cat /usr/local/etc/thread2.conf
# V4L2_PIX_FMT_MJPEG   : 2  'MJPEG'
# V4L2_PIX_FMT_JPEG    : 3  'JPEG'
# V4L2_PIX_FMT_RGB
v4l2_palette 4
videodevice /dev/video3
target_dir /tmp/motion3
webcam_port 8082
install apache2 and create a simple html file
(192.168.1.70 is the ip address from my raspberry pi)

Code: [Select]

$ cat /var/www/index.html
<!DOCTYPE html>
<html>
<head>
<style type="text/css">
body {
text-align:center;
background-color:#000000;
}
.container {
width:100%;
}
.motion1 {
border: 0;
width: 40%;
height: auto;
}
.motion2 {
border: 0;
width: 40%;
height: auto;
}
.clear {
clear:both;
}
</style>
</head>
<body>
<a href="http://192.168.1.70:8081"><img class="motion1" src="http://192.168.1.70:8081" /></a>

<a href="http://192.168.1.70:8082"><img class="motion2" src="http://192.168.1.70:8082" /></a>
</body>
</html>

open URL of server in a browser

[cynfab]

Hi tomas123,
Happy New Year....
Yep, I found and fixed the memory leak soon after my last post.

The motion stuff is cool, gotta find some time to play with it.
But, I need a way to combine the visual and thermal images as in your attachment but I don't need motion detection, just the side by side display.

   ...ken...

[tomas123]

Hi cynfab,

A Happy New Year from non snowy Germany

I don't need motion detection, just the side by side display.

I tested last days some streaming tools for V4L2.
A good entry point are this samples:
https://github.com/umlaeute/v4l2loopback/wiki
For the side by side display you can use the html template from my last post.

Here are my raw notes. All samples for our mjpeg stream /dev/video2:


MJPG-Streamer works great on Raspberry Pi and requires only 2% of CPU, because this tool only move the jpeg image without decoding/encoding.

installation guide
http://www.steinvoorte.nl/mjpgstreamer/index.php?page=2

install

Code: [Select]

$ sudo apt-get install libjpeg8-dev imagemagick subversion libv4l-dev

$ ls /usr/include/linux/videodev*
/usr/include/linux/videodev2.h

$ sudo ln -s /usr/include/linux/videodev2.h /usr/include/linux/videodev.h

$ wget http://sourceforge.net/code-snapshots/svn/m/mj/mjpg-streamer/code/mjpg-streamer-code-182.zip

$ unzip mjpg-streamer-code-182.zip

$ cd mjpg-streamer-code-182/mjpg-streamer

// select required plugin (i.e. input_file.so)
$ make USE_LIBV4L2=true mjpg_streamer input_uvc.so  output_http.so

// install (note, that apache2 use /var/www )
$ sudo cp -R www /usr/local/www
$ sudo cp mjpg_streamer /usr/local/bin
$ sudo cp output_http.so  input_uvc.so /usr/local/lib/

start (note LD_LIBRARY_PATH to input_uvc.so)

Code: [Select]

$ export LD_LIBRARY_PATH=/usr/local/lib
$ mjpg_streamer -i "input_uvc.so -d /dev/video2" -o "output_http.so -w /usr/local/www"
 i: Using V4L2 device.: /dev/video2
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
...
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
open the stream on a second Notebook
with GUI:  http://192.168.1.70:8080
Stream only: http://192.168.1.70:8080/?action=stream

unfortunately mjpg_streamer only eats mjpeg and not RGB24 /dev/video3


fswebcam is a nice tool for create (single) images with timestamps etc. from a V4L2 device

http://manpages.ubuntu.com/manpages/lucid/man1/fswebcam.1.html

Code: [Select]

$ sudo apt-get install fswebcam

$ fswebcam --device /dev/video3 1.jpg
--- Opening /dev/video3...
Trying source module v4l2...
/dev/video3 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 160x120.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to '1.jpg'.
result:


Nice for a web site with a periodic refresh

Code: [Select]

<META HTTP-EQUIV="refresh" CONTENT="15">
more complex example:

Code: [Select]

fswebcam -D 3 -S 10 -F 10 -r 1280x720 -d /dev/video0 \  --font "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono-Bold.ttf:12" \  --title "Kamera 1" --subtitle "Hallo World" \  --timestamp "%d.%m.%Y %H:%M" --info "info" $FILE

avconv / ffmpeg

my sample is for avconv
http://libav.org/avconv.html

Code: [Select]

sudo apt-get install libav-tools

CPU consumption on raspberry pi (top)
VGA 640x480 150% CPU
gray 160x120 25% CPU

ffmpeg/avconv can't work as really server but you can stream to a device (my notebook is 192.168.1.42):

Code: [Select]

$ avconv -f video4linux2 -i /dev/video2 -vcodec mpeg2video -r 25 -pix_fmt yuv420p -me_method epzs -b 2600k -bt 256k -f rtp rtp://192.168.1.42:1234
avconv version 9.14-6:9.14-1rpi1rpi1, Copyright (c) 2000-2014 the Libav developers
  built on Jul 22 2014 15:08:12 with gcc 4.6 (Debian 4.6.3-14+rpi1)
[video4linux2 @ 0xdc3740] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2, from '/dev/video2':
  Duration: N/A, start: 1451736986.645583, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj422p, 640x480, 0 kb/s, 1000k tbn, 5 tbc
Output #0, rtp, to 'rtp://192.168.1.42:1234':
  Metadata:
    encoder         : Lavf54.20.4
    Stream #0.0: Video: mpeg2video, yuv420p, 640x480, q=2-31, 2600 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg -> mpeg2video)
SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 192.168.1.42
t=0 0
a=tool:libavformat 54.20.4
m=video 1234 RTP/AVP 32
b=AS:2600

Press ctrl-c to stop encoding
frame=  437 fps= 24 q=4.2 size=    5613kB time=17.40 bitrate=2642.5kbits/s   

on client 192.168.1.42 open the stream on localhost ;-)

Code: [Select]

vlc / open network 
     rtp://@:1234
With VLC I received not good results.

Code: [Select]

$ cvlc v4l2:// :v4l2-dev=/dev/video2 :v4l2-width=640 :v4l2-height=480 --sout '#transcode{vcodec=mp4v,vb=128}:rtp{sdp=rtsp://:8554/live.ts}"' -I dummy
VLC media player 2.0.3 Twoflower (revision 2.0.2-93-g77aa89e)
[0x845c40] inhibit interface error: Failed to connect to the D-Bus session daemon: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
[0x845c40] main interface error: no suitable interface module
[0x74e00a28] main interface error: no suitable interface module
[0x6d98f0] main libvlc error: interface "globalhotkeys,none" initialization failed
[0x8495f0] dummy interface: using the dummy interface module...
[mpeg4 @ 0x730f80] removing common factors from framerate
[mjpeg @ 0x71ba20] error count: 65
[mjpeg @ 0x71ba20] error y=8 x=4
[mjpeg @ 0x71ba20] error count: 64
[mjpeg @ 0x71ba20] error y=10 x=29
[mjpeg @ 0x71ba20] mjpeg: unsupported coding type (c9)
[mjpeg @ 0x71ba20] error count: 66
[mjpeg @ 0x71ba20] error y=19 x=32
[mjpeg @ 0x71ba20] error count: 65
[mjpeg @ 0x71ba20] error y=20 x=31
[mjpeg @ 0x71ba20] only 8 bits/component accepted
[mpeg4 @ 0x7095a0] removing common factors from framerate
[mjpeg @ 0x71ba20] No JPEG data found in image
[mjpeg @ 0x71ba20] No JPEG data found in image
There is also the known feature/error

Code: [Select]

VLC is not supposed to be run as root. Sorry.

[cynfab]

Hi tomas123,
Thanks for the pointers,
I've used MJPG-Streamer on an NSLU2 for a long time  thanks to this site:

https://johnarthur.wordpress.com/2008/03/25/a-high-resolution-ip-webcam/

VLC is somewhat overkill as is ffmpeg, as they are both doing decode --> encode which gobbles up CPU cycles.
I have a pcduino nano lite which uses an Allwinner A20 as my current test system, but a single core iMX6 is the actual target. No local display though.
I guess I really need to get an iMX6 board to test with

   ...ken...

[cynfab]

Hi tomasw123,
Check this out:

https://github.com/ccrisan/motioneyeos/wiki

It is motion based, I have it running on a RPi orig B with a UVC webcam, it seems a bit slow, but the RPi is a real dog....

I'd bet they would be interested in integrating the flir8h driver...

   ...ken...

[tomas123]

I know this project ;-)
and I hope that Flir sells more F1 to the community.

I've used MJPG-Streamer on an NSLU2 for a long time

My concept was, to start the mjpg_streamer for /dev/video2 and /dev/video3 and place the two streams side by side on a web front end.
As I wrote before, mjpg_streamer only accepts mjpeg and YUV streams as input.

For inspiration as attachment a new version which writes the colorized RGB Thermal Image as MJPEG to /dev/video3
It needs the libjpeg library with libjpeg-dev (my path is /usr/include/jpeglib.h).

Code: [Select]

$ sudo apt-get install  libjpeg-dev
The code works fine...

Code: [Select]

$ make
gcc -I/usr/include/libusb-1.0 -o flir8i Palettes.o flir8i.o -lusb-1.0 -ljpeg -lm -Wall

$ sudo ./flir8i
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
using output device: /dev/video1
     vid_format->type                =2
     vid_format->fmt.pix.width       =160
     vid_format->fmt.pix.height      =120
     vid_format->fmt.pix.pixelformat =1497715271
     vid_format->fmt.pix.sizeimage   =20480
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=160
     vid_format->fmt.pix.colorspace  =8
using output device: /dev/video2
     vid_format->type                =2
     vid_format->fmt.pix.width       =640
     vid_format->fmt.pix.height      =480
     vid_format->fmt.pix.pixelformat =1196444237
     vid_format->fmt.pix.sizeimage   =1228800
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=0
     vid_format->fmt.pix.colorspace  =8
using output device: /dev/video3
     vid_format->type                =2
     vid_format->fmt.pix.width       =160
     vid_format->fmt.pix.height      =120
     vid_format->fmt.pix.pixelformat =1196444237
     vid_format->fmt.pix.sizeimage   =77824
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=0
     vid_format->fmt.pix.colorspace  =8

... with gucview and mplayer:

$ mplayer tv:// -tv driver=v4l2:device=/dev/video3
MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Team
Playing tv://.
TV file format detected.
Selected driver: v4l2
 name: Video 4 Linux 2 input
 author: Martin Olschewski <olschewski@zpr.uni-koeln.de>
 comment: first try, more to come ;-)
v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.
Selected device: Dummy video device (0x0003)
 Capabilities:  video capture  read/write  streaming
 supported norms:
 inputs: 0 = loopback;
 Current input: 0
Current format: MJPEG
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 53.35.0 (external)
Mismatching header version 53.32.2
Selected video codec: [ffmjpeg] vfm: ffmpeg (FFmpeg MJPEG)
==========================================================================
Audio: no sound
Starting playback...
Movie-Aspect is 1.33:1 - prescaling to correct movie aspect.
VO: [gl_nosw] 160x120 => 160x120 Planar YV12
V:   0.0 278/278 ??% ??% ??,?% 0 0

But unfortunately mjpg_streamer has a problem with a mjpeg stream 160x120.
I don't know why.

$ export LD_LIBRARY_PATH=/usr/local/lib

$ mjpg_streamer -i "input_uvc.so -d /dev/video3 -r 160x120" -o "output_http.so -w /usr/local/www"
MJPG Streamer Version: svn rev: Unversioned directory
 i: Using V4L2 device.: /dev/video3
 i: Desired Resolution: 160 x 120
 i: Frames Per Second.: 5
i: Format............: MJPEG
...
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled
*** glibc detected *** mjpg_streamer: free(): invalid next size (normal): 0x01258910 ***
Aborted

with MJPEG on /dev/video2 it works fine

Code: [Select]

$ mjpg_streamer -i "input_uvc.so -d /dev/video2" -o "output_http.so -w /usr/local/www"
MJPG Streamer Version: svn rev: Unversioned directory
 i: Using V4L2 device.: /dev/video2
 i: Desired Resolution: 640 x 480
 i: Frames Per Second.: 5
 i: Format............: MJPEG
...
 o: www-folder-path...: /usr/local/www/
 o: HTTP TCP port.....: 8080
 o: username:password.: disabled
 o: commands..........: enabled


PS: flir8i1.c is a working copy, which writes all colorized thermal frames as JPEGs to file with format #012345.jpg

[cynfab]

tomas123,
When I display the colorized jpeg stream with guvcview, the color palette is messed up. could this be part of the problem with mpeg-streamer?

But when I change the guvcview vid controls to RGB3 the colors are ok. Hmmmm..

   ...ken...

[tomas123]

When I display the colorized jpeg stream with guvcview, the color palette is messed up. could this be part of the problem with mpeg-streamer?

do you mean with guvcview on /dev/video3
-  flir8h with RGB24 (=RGB3)
- or flir8i with MJPEG ?

[cynfab]

I believe it was a cockpit problem, I had the guvcview color controls set to MJPEG for /dev/video3 and the resulting stream was green. I changed it to RGB3 and it was ok.
This was flir8i streaming a mjpeg stream on /dev/video3.