Question about FLIR One for Android

[cynfab]

tomas123,
Very nice.
I ran flir7i and it dumped many files before I realized it. Looking at the EP85 ones, I see ones that are pure jpg, some are pure thermal, some are combined, some are shuter state and some are 28 bytes of who knows what.

Code: [Select]

-rw-r--r-- 1 kenm kenm 107921 Dec 12 09:11 EP0x85#00023.bin
-rw-r--r-- 1 kenm kenm  26896 Dec 12 09:11 EP0x85#00024.bin
-rw-r--r-- 1 kenm kenm    168 Dec 12 09:11 EP0x85#00025.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00026.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00027.bin
-rw-r--r-- 1 kenm kenm  63289 Dec 12 09:11 EP0x85#00028.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00029.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00030.bin
-rw-r--r-- 1 kenm kenm  93785 Dec 12 09:11 EP0x85#00031.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00032.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00033.bin
-rw-r--r-- 1 kenm kenm  93385 Dec 12 09:11 EP0x85#00034.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00035.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00036.bin
-rw-r--r-- 1 kenm kenm  93201 Dec 12 09:11 EP0x85#00037.bin
-rw-r--r-- 1 kenm kenm  77944 Dec 12 09:11 EP0x85#00038.bin
-rw-r--r-- 1 kenm kenm    169 Dec 12 09:11 EP0x85#00039.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00040.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00041.bin
-rw-r--r-- 1 kenm kenm 103896 Dec 12 09:11 EP0x85#00042.bin
-rw-r--r-- 1 kenm kenm 124008 Dec 12 09:11 EP0x85#00043.bin
-rw-r--r-- 1 kenm kenm    169 Dec 12 09:11 EP0x85#00044.bin
-rw-r--r-- 1 kenm kenm 132952 Dec 12 09:11 EP0x85#00045.bin
-rw-r--r-- 1 kenm kenm    169 Dec 12 09:11 EP0x85#00046.bin
-rw-r--r-- 1 kenm kenm 116440 Dec 12 09:11 EP0x85#00047.bin
-rw-r--r-- 1 kenm kenm    169 Dec 12 09:11 EP0x85#00048.bin
-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00049.bin

EP0x85#00036 thermal
EP0x85#00037 jpg
EP0x85#00038
EP0x85#00039 shutter
EP0x85#00040 28 byte
EP0x85#00043 combined

Or maybe I'm missing something

[sonic]

[...]it was usable on the PC as a webcam.

I did this with the help of SmartCam. I modified the Android SDK example to send a MJPEG stream to it via WiFi. Details on my website.

[tomas123]

Hi sonic,

good work 

Do you know my posts about the SDK?
You can edit the sdk java files and also switch from cropped to full lepton resolution. Read this thread for more informations...

I think you are the only one, who testing the Flir SDK.
Here you can download a working SDK sample with the extracted java.class (see my post above)
 see path FLIROneSDKBundle\FLIRONEExampleApplication\app\src\main\java\com\flir\flironesdk\usb

have fun
http://www.file-upload.net/download-11040066/FLIROneSDKBundle04-Basis.zip.html

background:
 http://stackoverflow.com/questions/21417419/how-to-convert-aar-to-jar

Android Studio (version: 1.3.2) allows you to seamlessly access the .jar inside a .aar.
Bonus: it automatically decompiles the classes!
Simply follow these steps:
1. File > New > New Module > Import .JAR/.AAR Package to import you .aar as a module
2. Add the newly created module as a dependency to your main project (not sure if needed)
3. Right click on "classes.jar" as shown in the capture below, and click "Show in explorer". Here is your .jar.

next step:
Android Studio / File / Project Structur
     Modules "app" / Dependencies / + "Module Dependency" and select FlirSDK


SDK without cropped images (full resolution 160x120):

change SDK path for CameraFiles from intern
/data/data/com.flir.flironeexampleapplication/files/CameraFiles
to extern
/sdcard/Android/data/com.flir.flironeexampleapplication/files/CameraFiles

by editing two code lines

Code: [Select]

\FLIROneSDKBundle\FLIRONEExampleApplication\app\src\main\java\com\flir>grep -r -n -A1  getFilesDir *
flironesdk/Device.java:213:        //FrameProcessor.updateCameraFiles(zis, cachedContext.getFilesDir().getAbsolutePath() + File.separator);
flironesdk/Device.java-214-        FrameProcessor.updateCameraFiles(zis, cachedContext.getExternalFilesDir(null).getAbsolutePath() + File.separator);
--
flironesdk/FrameProcessor.java:101:        // String storagePath = context.getFilesDir().getAbsolutePath() + File.separator;
flironesdk/FrameProcessor.java-102-        String storagePath = context.getExternalFilesDir(null).getAbsolutePath() + File.separator;

next step is avoid overwriting your file calib.rsc after starting the apk
edit the function updateCameraFiles() in file FrameProcessor.java

Code: [Select]

static void updateCameraFiles(ZipInputStream zis, String storagePath)
{
    try
    {
        ZipEntry ze;
        while ((ze = zis.getNextEntry()) != null) {
            File f = new File(storagePath + ze.getName());
            if ((!f.exists())||(!ze.getName().endsWith( "calib.rsc")))
            {
                if (ze.isDirectory()) {
                    //File f = new File(storagePath + ze.getName());
                    if (!f.isDirectory()) {
                        f.mkdirs();
                    }
                } else {
                    File parentDir = new File(storagePath + ze.getName()).getParentFile();
                    if (( null != parentDir) &&
                            (!parentDir.isDirectory())) {
                        parentDir.mkdirs();
                    }
                    BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(storagePath + ze.getName()));
                    byte[] bytesIn = new byte['?' ];
                    int read;
                    while ((read = zis.read(bytesIn)) != - 1) {
                        bos.write(bytesIn, 0, read);
                    }
                    zis.closeEntry();
                    bos.flush();
                    bos.close();
                }
            }
        }
        zis.close();
    }
    catch (IOException e)
    {
        e.printStackTrace();
    }
}
simple delete
/sdcard/Android/data/com.flir.flironeexampleapplication/files/CameraFiles/system/calib.rsc
for getting a fresh version of the file from the FlirOne Flash

now you can edit your calib.rsc (without always overwriting with original file from your F1)

edit your calib.rsc to remove the crop factor by removing the .calib.lensCorr part
the crop factor is inside *.fff files, this files are not bad pixel map files (like Ex):

Code: [Select]

.calib.lensCorr entry
.calib.lensCorr.ds_we_ap_fi_le index
.calib.lensCorr.ds_we_ap_fi_le.enabled bool true
.calib.lensCorr.ds_we_ap_fi_le.fileName text "ds_we_ap_fi_le_LCFMap.fff"
.calib.lensCorr.ds_we_ap_fi_le.rotation double 0.345
.calib.lensCorr.ds_we_ap_fi_le.type int32 100
.calib.lensCorr.ds_we_ap_fi_le.zoom double 7.69
.calib.lensCorr.ds_we_ap_fi_leExFOL2 index
.calib.lensCorr.ds_we_ap_fi_leExFOL2.enabled bool true
.calib.lensCorr.ds_we_ap_fi_leExFOL2.fileName text "ds_we_ap_fi_leExFOL2_LCFMap.fff"
.calib.lensCorr.ds_we_ap_fi_leExFOL2.rotation double 0.345
.calib.lensCorr.ds_we_ap_fi_leExFOL2.type int32 100
.calib.lensCorr.ds_we_ap_fi_leExFOL2.zoom double 7.69
calc a new checksum crc

Code: [Select]

// crc32 sample on bash command line

// count lines
$ wc -l calib.rsc
     646 calib.rsc

//print last line
$ tail -n1 calib.rsc
# CRC32 4683a151

//calc a new crc32 for (n-1) lines
$ crc32 <(head -n 645 calib.rsc)
4683a151first try this steps with an original file
inside the large E4 thread in this forum you find some more methods for calculating the Flir CRC


a hint: the FlirOne App from Play Store save the original CameraFiles also to the visible path:
/sdcard/Android/data/com.flir.flirone/files/
CameraFiles.zip
CameraFilesSimu.zip
Here you can always get an original (unpatched) CameraFiles.zip.

view my thermal gallery 
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg802245/#msg802245

Flir One G2 Lepton 160x120 with hacked config file calib.rsc

[tomas123]

Hi sonic,

I saw in your code, that you modified the function onFrameProcessed().

Here is my code modification for onFrameProcessed() for saving full size raw images to *.raw

Another quick n' dirty solution works fine:

Save the ThermalLinearFlux14BitImage RAW image:

FLIRONEExampleApplication/app/src/main/java/com/flir/flironeexampleapplication/PreviewActivity.java

Code: [Select]


// for raw images
import java.io.FileOutputStream;

   public void onFrameProcessed(final RenderedImage renderedImage){
        thermalBitmap = renderedImage.getBitmap();
        updateThermalImageView(thermalBitmap);


        /*
        Capture this image if requested.
        */
        if (this.imageCaptureRequested) {
            imageCaptureRequested = false;
            final Context context = this;
            new Thread(new Runnable() {
                public void run() {
                    String path = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
                    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd-HH-mm-ssZ", Locale.getDefault());
                    String formatedDate = sdf.format(new Date());
                    String fileName = "FLIROne-" + formatedDate + ".jpg";
                    try{
                        lastSavedPath = path+ "/" + fileName;
                        renderedImage.getFrame().save(new File(lastSavedPath), RenderedImage.Palette.Iron, RenderedImage.ImageType.BlendedMSXRGBA8888Image);

                        MediaScannerConnection.scanFile(context,
                                new String[]{path + "/" + fileName}, null,
                                new MediaScannerConnection.OnScanCompletedListener() {
                                    @Override
                                    public void onScanCompleted(String path, Uri uri) {
                                        Log.i("ExternalStorage", "Scanned " + path + ":");
                                        Log.i("ExternalStorage", "-> uri=" + uri);
                                    }

                                });
                        if (renderedImage.imageType() == RenderedImage.ImageType.ThermalLinearFlux14BitImage){
                            // save the Lepton RAW file
                            String rawName = "FLIROne-" + formatedDate + ".raw";
                            lastSavedPath = path+ "/" + rawName;
                            File file = new File(lastSavedPath);
                            FileOutputStream fop = null;
                            fop = new FileOutputStream(file);
                            if (!file.exists()) {
                                file.createNewFile();
                            }
                            //int rawLength = renderedImage.pixelData().length;
                            byte[] LeptonRAWBytes = renderedImage.pixelData();
                            fop.write(LeptonRAWBytes);
                            fop.flush();
                            fop.close();
                        }

                    }catch (Exception e){
                        e.printStackTrace();
                    }
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {

                            thermalImageView.animate().setDuration(50).scaleY(0).withEndAction((new Runnable() {
                                public void run() {
                                    thermalImageView.animate().setDuration(50).scaleY(1);
                                }
                            }));
                        }
                    });
                }
            }).start();
        }
...


If the image type is set to "ThermalLinearFlux14BitImage" then it save also the raw frame 160x120 to disk.

So I got the nice gallery photo from F1G2:
...

[tomas123]

Hi cynfab,

you are right.
On my Raspberry Pi this glitches rarely happens, but they appear.
See my log from the first 300 frames.

Code: [Select]

-rw-r--r-- 1 root root     28 2015-12-13 12:47:47.628956961 +0100 EP0x85#00221.bin
-rw-r--r-- 1 root root  38828 2015-12-13 12:47:47.838952429 +0100 EP0x85#00222.bin
-rw-r--r-- 1 root root  63830 2015-12-13 12:47:48.048947898 +0100 EP0x85#00223.binthere are also jumbo frames

Code: [Select]

-rw-r--r-- 1 root root 206830 2015-12-13 12:47:44.959014674 +0100 EP0x85#00207.bin
We know, that the USB Buffer size is limited to 512 Bytes (see lsusb -v and USB specification) .
The libusb driver glues the packets together to a "frame".

# lsusb -v -d 09cb:1996
...
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x85  EP 5 IN           // libusb_bulk_transfer()
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
          wMaxPacketSize     0x0200  1x 512 bytes

I think, its simple to handle this different frame size in software.
One possibly is to stitch the packets self to a full frame (thermal+jpg+status)

Please look also at the full time stamp. Do you have another frame rate??

and sometimes the EP 85 reads come a lot faster than other times.

another minor information:
Now we know, that the header is not 32, but 28 Bytes and the raw image begins with the two status lines (2 Pixel * 16 Bit = 4 Byte).
With a 32 Byte header the status lines are in the middle and on the right side of the image.

Code: [Select]

-rw-r--r-- 1 kenm kenm     28 Dec 12 09:11 EP0x85#00029.bin
-rw-r--r-- 1 kenm kenm  39852 Dec 12 09:11 EP0x85#00030.bin

Until yet I not thought deeper about this 4 columns.

Encryptededdy linked this, but I think our columns (120px) are not telemetry datas (lines with 160 px) :

More info about the 4 lines of addtl' information: http://media.wix.com/ugd/53cdb6_5191be73d1c943d78d2e1a095cb7f3b8.pdf

[tomas123]

Hi cynfab,

Code: [Select]

-rw-r--r-- 1 root root    167 2015-12-13 12:47:54.798802942 +0100 EP0x85#00254.bin
-rw-r--r-- 1 root root  39340 2015-12-13 12:47:55.008798450 +0100 EP0x85#00255.bin
-rw-r--r-- 1 root root  70983 2015-12-13 12:47:55.228793750 +0100 EP0x85#00256.bin
after looking inside the glitch frames I get this result:
1. All packets with size about 60k are correct jpeg images without errors (begins with magic byte sequence of jpeg "FF 08 FF C0").
2. All packets with size about 40k are broken raw images, which misses the first/second line
3. All packets with size of 168 Bytes are correct status strings
4. All packets with size of 28 Bytes are the header bytes of a regular full frame (thermal+jpg+status)



my interpretation:
The FlirOne has an internal (timing/sync) problem with the i2c bus to lepton sensor.
This results sometimes in loosing the first lines of thermal sensor and as a result of this we get the usb glitches.
-> litter this frames (my waste quote is 10/314 frames = 3%)

my jumbo frames (200k) are two regular frames (thermal+jpg+status) without the second status, which follows behind with 169 Byte

Code: [Select]

-rw-r--r-- 1 root root 103719 2015-12-13 12:47:46.988970775 +0100 EP0x85#00218.bin
-rw-r--r-- 1 root root 206879 2015-12-13 12:47:47.218965810 +0100 EP0x85#00219.bin
-rw-r--r-- 1 root root    167 2015-12-13 12:47:47.418961494 +0100 EP0x85#00220.bin
for another readers:
currently Flir doesn't public infos about the Leptons radiometric feature:
http://www.flir.com/cvs/cores/knowledgebase/index.cfm?CFTREEITEMKEY=914&view=67485
the FlirOne is currently the only chance to get radiometric datas from Lepton 3 sensor (with black body calibration, thanks to Flir!!)

[cynfab]

Hi tomas123,
All interesting stuff.
I think we need to look carefully at how the FlirOne SDK example and the decompiled SDK files work. As well as considering the use of libusb's Async API.
I should have some time this week to dig into things.

[cynfab]

I've figured out most of what the 28byte header is about:

A typical 28 byte header:

ef be 00 00 01 00 00 00 b3 02 02 00 ac 9b 00 00  <<< oops typo in orig post
60 66 01 00 a7 00 00 00 cf 4f 12 3c 00 30 ef 9e

From EPx85#00035.bin which is the combined thermal & jpeg.

ls -la
-rw-r--r-- 1 kenm kenm  91911 Dec 14 14:25 1.jpg
-rw-r--r-- 1 kenm kenm 131791 Dec 14 12:32 EP0x85#00035.bin

dd if=EP0x85#00035.bin of=1.jpg bs=1 skip=39880
91911+0 records in
91911+0 records out
91911 bytes (92 kB) copied, 0.140179 s, 656 kB/s

identify 1.jpg
1.jpg JPEG 640x480 640x480+0+0 8-bit DirectClass 91.9KB 0.000u 0:00.000

This jpeg has no EXIF data.

location of status start
28 + thermal image length + length of jpeg:
28 + 39852 + 91744 = 131624

dd if=EP0x85#00035.bin of=1.txt bs=1 skip=131624
167+0 records in
167+0 records out
167 bytes (167 B) copied, 0.000448197 s, 373 kB/s

cat 1.txt
{"shutterState":"FFC","shutterTemperature":307.850006103516,"usbNotifiedTimestamp":1176851589.8668,"usbEnqueuedTimestamp":1176851589.86892,"ffcState":"FFC_VALID_RAD"}

shutter states are FFC | ON
ffc states are Valid | Desired
(maybe more)

Byte Fields:

0 ef
1 be
2 00
3 00
4 01
5 00
6 00
7 00
8-b file (less 28 byte header) length: 02 02 b3 = 131763 + 28 = 131791
c-f length of thermal image:  9b ac = 39852 + 28 byte header = 39880
10-14 length of jpeg image: 01 66 60 = 91744 -> 91911 - 91744 = 167
14-17 length of status: a7 = 167
18-1b ??  still need to figure out what this is
1c-1d 00|10|20|30|40|50|60|70|80 ?? still need this
1e-1f checksum ?? this too.

   ...ken...

[tomas123]

Wow!!
I also looked inside the header but overlooked  thermal/jpg image size

c-f length of thermal image:  9b ac = 39852 + 28 byte header = 39880

for comparing as attachment 300 headers from my log
I filtered my glitches with "-size +90k", but sometimes (after a glitch) slips the status string to the header of the next "full frame"

The combination of image size >90k and magic bytes "ef be"  is the best marker.

Code: [Select]

$ for i in $(find . -type f -name "EP0x85*" -size +90k); do  ls -al $i; hexdump -C -n 32 $i;done
-rw-r--r--@ 1 tom  staff  101476 13 Dez 12:47 ./EP0x85#00006.bin
00000000  ef be 00 00 01 00 00 00  48 8c 01 00 ac 9b 00 00  |........H.......|
00000010  00 f0 00 00 9c 00 00 00  32 b8 42 8a 00 40 6e 6f  |........2.B..@no|
00000020
-rw-r--r--@ 1 tom  staff  101516 13 Dez 12:47 ./EP0x85#00007.bin
00000000  ef be 00 00 01 00 00 00  70 8c 01 00 ac 9b 00 00  |........p.......|
00000010  28 f0 00 00 9c 00 00 00  7d 1b da 22 00 50 a3 d0  |(.......}..".P..|
00000020
-rw-r--r--@ 1 tom  staff  101430 13 Dez 12:47 ./EP0x85#00008.bin
00000000  ef be 00 00 01 00 00 00  1a 8c 01 00 ac 9b 00 00  |................|
00000010  d0 ef 00 00 9e 00 00 00  85 07 39 f8 00 50 3e 6b  |..........9..P>k|
00000020
-rw-r--r--@ 1 tom  staff  101336 13 Dez 12:47 ./EP0x85#00009.bin
00000000  ef be 00 00 01 00 00 00  64 8c 01 00 ac 9b 00 00  |........d.......|
00000010  10 f0 00 00 a8 00 00 00  5a 4e ad 39 00 50 b4 c5  |........ZN.9.P..|
00000020
-rw-r--r--@ 1 tom  staff  101488 13 Dez 12:47 ./EP0x85#00011.bin
00000000  ef be 00 00 01 00 00 00  fc 8c 01 00 ac 9b 00 00  |................|
00000010  a8 f0 00 00 a8 00 00 00  6d cf 77 f5 00 70 d5 1e  |........m.w..p..|
00000020
-rw-r--r--@ 1 tom  staff  101719 13 Dez 12:47 ./EP0x85#00012.bin
00000000  7b 22 73 68 75 74 74 65  72 53 74 61 74 65 22 3a  |{"shutterState":|
00000010  22 46 46 43 22 2c 22 73  68 75 74 74 65 72 54 65  |"FFC","shutterTe|
00000020
-rw-r--r--@ 1 tom  staff  101647 13 Dez 12:47 ./EP0x85#00013.bin
00000000  ef be 00 00 01 00 00 00  f3 8c 01 00 ac 9b 00 00  |................|
00000010  a0 f0 00 00 a7 00 00 00  b4 a3 1b f0 00 30 20 d3  |.............0 .|
00000020
-rw-r--r--@ 1 tom  staff  141638 13 Dez 12:47 ./EP0x85#00014.bin
00000000  ef be 00 00 01 00 00 00  62 8d 01 00 ac 9b 00 00  |........b.......|
00000010  10 f1 00 00 a6 00 00 00  d1 80 6f 95 00 60 b7 0b  |..........o..`..|
00000020
-rw-r--r--@ 1 tom  staff  101847 13 Dez 12:47 ./EP0x85#00016.bin
00000000  ef be 00 00 01 00 00 00  bb 8d 01 00 ac 9b 00 00  |................|
00000010  68 f1 00 00 a7 00 00 00  df 46 b9 1b 00 50 dc 5b  |h........F...P.[|
00000020
-rw-r--r--@ 1 tom  staff  101861 13 Dez 12:47 ./EP0x85#00017.bin
00000000  ef be 00 00 01 00 00 00  c9 8d 01 00 ac 9b 00 00  |................|
00000010  78 f1 00 00 a5 00 00 00  05 29 cd 95 00 30 33 4c  |x........)...03L|
00000020
-rw-r--r--@ 1 tom  staff  101783 13 Dez 12:47 ./EP0x85#00018.bin
00000000  ef be 00 00 01 00 00 00  7b 8d 01 00 ac 9b 00 00  |........{.......|
00000010  28 f1 00 00 a7 00 00 00  3c 9f a7 40 00 40 cc ad  |(.......<..@.@..|
00000020
-rw-r--r--@ 1 tom  staff  101631 13 Dez 12:47 ./EP0x85#00019.bin
00000000  ef be 00 00 01 00 00 00  e3 8c 01 00 ac 9b 00 00  |................|
00000010  90 f0 00 00 a7 00 00 00  57 ce f1 93 00 50 8b 4c  |........W....P.L|
00000020
-rw-r--r--@ 1 tom  staff  101839 13 Dez 12:47 ./EP0x85#00020.bin
00000000  ef be 00 00 01 00 00 00  b3 8d 01 00 ac 9b 00 00  |................|
00000010  60 f1 00 00 a7 00 00 00  25 c2 4f bb 00 60 59 53  |`.......%.O..`YS|
00000020
-rw-r--r--@ 1 tom  staff  101871 13 Dez 12:47 ./EP0x85#00021.bin
00000000  ef be 00 00 01 00 00 00  d3 8d 01 00 ac 9b 00 00  |................|
00000010  80 f1 00 00 a7 00 00 00  c3 6a de 01 00 20 59 89  |.........j... Y.|
00000020
-rw-r--r--@ 1 tom  staff  100807 13 Dez 12:47 ./EP0x85#00022.bin
00000000  ef be 00 00 01 00 00 00  ab 89 01 00 ac 9b 00 00  |................|
00000010  58 ed 00 00 a7 00 00 00  e5 b8 51 52 00 40 66 41  |X.........QR.@fA|
00000020
-rw-r--r--@ 1 tom  staff  100775 13 Dez 12:47 ./EP0x85#00023.bin
00000000  ef be 00 00 01 00 00 00  8b 89 01 00 ac 9b 00 00  |................|
00000010  38 ed 00 00 a7 00 00 00  23 63 85 95 00 50 56 d3  |8.......#c...PV.|
00000020



[tomas123]

Hi cynfab,

your infos for endpoint 0x85 in formatted notation:


dump header

$ hexdump -C -n 32 ./EP0x85#00019.bin
-rw-r--r--@ 1 tom  staff  101631 13 Dez 12:47 ./EP0x85#00019.bin
00000000  ef be 00 00 01 00 00 00  e3 8c 01 00 ac 9b 00 00  |................|
00000010  90 f0 00 00 a7 00 00 00  57 ce f1 93 00 50 8b 4c  |........W....P.L|

->  file size  101.631 Byte
    header         -28 Byte
    ---------------------
    frame size 101.603 Byte

--------------------------------------
header

address   
00000000  ef be 00 00   magic bytes 0xEF 0xBE
00000004  01 00 00 00   0x00000001
00000008  e3 8c 01 00   0x00018CE3 = 101.603 byte frame size

0000000C  ac 9b 00 00   0x00009BAC =  39.852 byte thermal image size
00000010  90 f0 00 00   0x0000F090 =  61.584 byte jpg image size
00000014  a7 00 00 00   0x000000A7 =     167 byte status string size
                                     ------------
                             total:  101.603 byte frame size
00000018  57 ce f1 93   unknown
0000001C  00 50 8b 4c   unknown

--------------------------------------------
start adresses

size       adress   (in dezimal)
           0000000  magic bytes 0xEF 0xBE
+     28   0000028  thermal image with size of 39.852 byte
+ 39.852   0039880  jpg image with size of 61.584 byte
+ 61.584   0101464  status string with size of 167 byte
+    167
---------
 101.631 total file size

identify 1.jpg
1.jpg JPEG 640x480 640x480+0+0 8-bit DirectClass 91.9KB 0.000u 0:00.000

This jpeg has no EXIF data.

[tomas123]

Hi cynfab,

some infos:

my next step (on weekend) is to write a code for a V4L2 loopback devices ( /dev/video0 )
Then we can simple connect the FlirOne with Linux video world 

3 month ago I supplemented the great code from user tmbinc  with a loopback "driver" for the Flir E4, therefore I can recycle most of the the code.
see: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530

another thermal video concept with OpenCV comes from user Bud:
see my linklist: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729604/#msg729604

[cynfab]

Hi tomas123,

I was going to do a better job of documenting the EP 81 & EP 83 headers, but here are my raw notes:

Header for EP 81 data

16 bytes

cc 01 00 00 01 00 00 00 55 00 00 00 30 73 8c df


00-03 Header ID cc 01 00 00
04    01
05    00
06    00
06    00
08-0b 00 00 00 55 length of message
0c    ??
0d    ??
0e-0f checksum ??
10-end-1 status message
end   00

++++++++++++++++++++++++

header for the EP83 --> camerafiles.zip
24 bytes

10 55 00 00 01 00 00 00 b7 1a 00 00 0a 00 00 00
00 99 9d 5d 2c 91 a3 df

00-03 header ID 10 55 00 00
04    01
05    00
06    00
07    00
08-0b 00 00 1a b7 = 6839 length of CameraFiles.zip
0c
0d
0e
0f
10
11
12
13
14
15
16-17 checksum ??
18-end CameraFiles.zip


I'd like to see a v4l2 driver too, but I haven't the skills to create one from  scratch.
I'd also like to replicate my dual view android app, maybe as an offshoot of your v4l2 work.

   ...ken...

[tomas123]

Hi cynfab,

I copied the framebuffer and v4l2 code parts from here
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530
to our code.

This code is only a first buggy study!!

hardware: raspberry pi
$ uname -a
Linux raspberryLCD 3.18.13-v7+ #784 SMP PREEMPT Sat May 9 15:57:36 BST 2015 armv7l GNU/Linux


flir8a.c writes the thermal video stream only to the linux framebuffer.
You must switch from your x session to a virtual console (native shell) to see the framebuffer.
I wrote the code for my raspberry display 800x480 in 16 bit.
See comments in code for another resolution and bit depth.

compile with

Code: [Select]

$ /usr/bin/gcc -std=c99 '-I/usr/include/libusb-1.0'  -o flir flir8a.c -lusb-1.0 -lm


Code: [Select]

$ sudo ./flir
The framebuffer device opened.
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO

:xx Thu Dec 17 01:08:24 2015

: Thu Dec 17 01:08:24 2015
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x81: LIBUSB_ERROR_TIMEOUT

: Thu Dec 17 01:08:25 2015
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x83: LIBUSB_ERROR_TIMEOUT

ask for CameraFiles.zip on EP 0x83:

: Thu Dec 17 01:08:26 2015

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 41 00 00 00 f8 b3 f7 00 ]

Write successful!
EP 0x02 to be sent: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}
Write successful!
Sent 65 bytes with string: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 33 00 00 00 ef db c1 c1 ]

Write successful!
EP 0x02 to be sent 51 Bytes: {"type":"readFile","data":{"streamIdentifier":10}}
Write successful!
Sent 51 bytes with string: {"type":"readFile","data":{"streamIdentifier":10}}

: Thu Dec 17 01:08:26 2015




flir8b.c writes the thermal video stream to the linux framebuffer and /dev/video0

read my notes to install v4l2loopback on a raspberry pi
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530
and for a V4L2 loopback device under Ubuntu
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756831/#msg756831

don't forget to load the kernel driver:

Code: [Select]

$ sudo depmod -a  `uname -r`
$ sudo modprobe v4l2loopback

$ lsmod
Module                  Size  Used by
v4l2loopback           23348  0
videodev              122487  1 v4l2loopback
For v4l2 I removed the switch "std=c99" (avoid "/usr/include/linux/videodev2.h:2090:20: error: field ‘timestamp’ has incomplete type").
Compile with

Code: [Select]

$ /usr/bin/gcc '-I/usr/include/libusb-1.0'  -o flir flir8b.c -lusb-1.0 -lm

start with

Code: [Select]

$ sudo ./flir
The framebuffer device opened.
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
using output device: /dev/video0
     vid_format->type                =2
     vid_format->fmt.pix.width       =160
     vid_format->fmt.pix.height      =120
     vid_format->fmt.pix.pixelformat =1497715271
     vid_format->fmt.pix.sizeimage   =20480
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=160
     vid_format->fmt.pix.colorspace  =8
stop interface 2 FRAME
stop interface 1 FILEIO

start interface 1 FILEIO

:xx Thu Dec 17 00:52:26 2015

: Thu Dec 17 00:52:26 2015
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x81: LIBUSB_ERROR_TIMEOUT

: Thu Dec 17 00:52:27 2015
 >>>>>>>>>>>>>>>>>bulk transfer (in) 0x83: LIBUSB_ERROR_TIMEOUT

ask for CameraFiles.zip on EP 0x83:

: Thu Dec 17 00:52:28 2015

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 41 00 00 00 f8 b3 f7 00 ]

Write successful!
EP 0x02 to be sent: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}
Write successful!
Sent 65 bytes with string: {"type":"openFile","data":{"mode":"r","path":"CameraFiles.zip"}}

EP 0x02 to be sent Hexcode: 16 Bytes[ cc 01 00 00 01 00 00 00 33 00 00 00 ef db c1 c1 ]

Write successful!
EP 0x02 to be sent 51 Bytes: {"type":"readFile","data":{"streamIdentifier":10}}
Write successful!
Sent 51 bytes with string: {"type":"readFile","data":{"streamIdentifier":10}}

: Thu Dec 17 00:52:28 2015

: Thu Dec 17 00:52:28 2015
 bulk read EP 0x81, actual length 23
HEX:

STRING:
gMode":"operational"}}

Ask for video stream, start EP 0x85:

: Thu Dec 17 00:52:28 2015
 bulk read EP 0x81, actual length 108
HEX:

STRING:
?\?<??{"type":"openFileStatus","data":{"path":"CameraFiles.zip","streamIdentifier":9,"mode":"r"}}

use mplayer to get the stream

Code: [Select]

$ sudo mplayer tv:// -tv driver=v4l2:device=/dev/video0
MPlayer svn r34540 (Raspbian), built with gcc-4.6 (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing tv://.
TV file format detected.
Selected driver: v4l2
 name: Video 4 Linux 2 input
 author: Martin Olschewski <olschewski@zpr.uni-koeln.de>
 comment: first try, more to come ;-)
v4l2: your device driver does not support VIDIOC_G_STD ioctl, VIDIOC_G_PARM was used instead.
Selected device: Dummy video device (0x0000)
 Capabilities:  video capture  read/write  streaming
 supported norms:
 inputs: 0 = loopback;
 Current input: 0
 Current format: GREY
...
==========================================================================
Opening video decoder: [raw] RAW Uncompressed Video
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
Movie-Aspect is undefined - no prescaling applied.
[swscaler @ 0x562380] using unscaled gray -> yuv420p special converter
VO: [sdl] 160x120 => 160x120 Planar YV12


PS: I have the suspicion that the delay of flir8a.c (frame buffer) is a little bit greater as in the Flir App in mode ThermalLinearFlux14BitImage

[cynfab]

Hi tomas123,
wow that was fast, looks very interesting, I'll be playing with that code over the next few days.
Nice job!!
   ...ken...

[tomas123]

only cut and paste 

frame buffer works fine, but v4l2 needs some tuning...

the original frame buffer code comes from user tmbinc 
watch his video: https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg626608/#msg626608
and get the code (another resolution): https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344

please note the great font.h

[tomas123]

flir8b.c writes the thermal video stream to  /dev/video0 and works better than expected.
Runs since hours rock stable!

here my experiences with a live stream from raspberry pi to my notebook:

start Flir One video grabber on raspberry pi

Code: [Select]

$ sudo modprobe v4l2loopback
$ /usr/bin/gcc '-I/usr/include/libusb-1.0'  -o flir flir8b.c -lusb-1.0 -lm
$ sudo ./flir
start a video server on raspberry pi with avconv and stream /dev/video0 to my notebook with address 192.168.1.42
(parameters yet not adjusted)

Code: [Select]

$ avconv -f video4linux2 -i /dev/video0 -vcodec mpeg2video -r 25 -pix_fmt yuv420p -me_method epzs -b 2600k -bt 256k -f rtp rtp://192.168.1.42:1234
avconv version 9.14-6:9.14-1rpi1rpi1, Copyright (c) 2000-2014 the Libav developers
  built on Jul 22 2014 15:08:12 with gcc 4.6 (Debian 4.6.3-14+rpi1)
[video4linux2 @ 0x1ffc740] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2, from '/dev/video0':
  Duration: N/A, start: 1450392797.097902, bitrate: 4608 kb/s
    Stream #0.0: Video: rawvideo, gray, 160x120, 4608 kb/s, 1000k tbn, 30 tbc
Output #0, rtp, to 'rtp://192.168.1.42:1234':
  Metadata:
    encoder         : Lavf54.20.4
    Stream #0.0: Video: mpeg2video, yuv420p, 160x120, q=2-31, 2600 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo -> mpeg2video)
...
Press ctrl-c to stop encoding
frame=23557 fps= 25 q=2.0 size=   14522kB time=942.20 bitrate= 126.3kbits/s   
 

on my notebook:
start VLC and open the stream with
   vlc / open network

Code: [Select]

rtp://@:1234and we get a live stream 



my first test with motion http://www.lavrsen.dk/foswiki/bin/view/Motion/WebHome

our flir device:

Code: [Select]

$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index       : 0
Type        : Video Capture
Pixel Format: 'GREY'
Name        : [GREY]
Size: Discrete 160x120
Interval: Discrete 0.033s (30.000 fps)
install motion

Code: [Select]

$ sudo apt-get install motion
edit config file

Code: [Select]

$ sudo nano /etc/motion/motion.conf

# Maximum framerate for webcam streams (default: 1)
webcam_maxrate 10

# Restrict webcam connections to localhost only (default: on)
webcam_localhost off
but we cant select our frame format V4L2_PIX_FMT_GREY in motion.conf 

Code: [Select]

# v4l2_palette allows to choose preferable palette to be use by motion
# V4L2_PIX_FMT_SN9C10X : 0  'S910'
# V4L2_PIX_FMT_SBGGR8  : 1  'BA81'
# V4L2_PIX_FMT_MJPEG   : 2  'MJPEG'
# V4L2_PIX_FMT_JPEG    : 3  'JPEG'
# V4L2_PIX_FMT_RGB24   : 4  'RGB3'
# V4L2_PIX_FMT_UYVY    : 5  'UYVY'
# V4L2_PIX_FMT_YUYV    : 6  'YUYV'
# V4L2_PIX_FMT_YUV422P : 7  '422P'
# V4L2_PIX_FMT_YUV420  : 8  'YU12'
v4l2_palette 8
It's possible to convert /dev/video0 with ffmpeg to a compatible frame format and write to next device /dev/video1.
But I think it's better to make the converts in our code (like the current convert for frame buffer to RGB565 16 Bit)   

[cynfab]

tomas123,
I've run this with mplayer and it works as described, every so often I get a noise frame, but it seems to correct itself quickly.

Nice work. this will be the beginning of FlirOne for Linux (maybe MacOSX  and Windoze too)

  ...ken...

[tomas123]

...this will be the beginning of FlirOne for Linux (maybe MacOSX  and Windoze too)

I believe not.
Since last two years I wrote here hundreds of posts about

• hacking Flir RAW formats (exiftool)
• Plancks Law
• calculate temperature from flir radiometric sensor values with Flir calibration values
• stitch thermal panoramas
• raw videos and streaming of Flir Ex, Flir Exx  and Flir One 2. gen
• convert thermal images and videos from another manufacturers (Seek, Opgal) in Flir radiometric jpg / video and open it with Flir Tools for evaluation

... but only half a dozen users are interested in

I've run this with mplayer and it works as described, every so often I get a noise frame, but it seems to correct itself quickly.

Do you mean a FFC frame,  a regular 160x120 frame with a closed shutter ?
Otherwise flir8b.c is until missing a validation of frame size (make only a magic byte check), therefore we have sometimes noise glitches.
But I haven't seen v4l2 sync problems...

please post my a "ls -al" of 1 or 2 minutes EP0x85#00xxx.bin, so I can see how often your system got broken frames

[tomas123]

Hi cynfab,

I found this nice short source samples on github with lepton drivers on the basis of https://github.com/umlaeute/v4l2loopback :
https://github.com/andrewhodel/flirpi-mjpeg

"These are miscellaneous utilities that work with the lepton module on the raspberry pi."

a fork of https://github.com/groupgets/LeptonModule/tree/master/software/flirpi

forum: https://groups.google.com/forum/#!topic/flir-lepton/VcdpMY_vcho

the code is for Lepton G1 80x60 connected over SPI

leptsci.c is the code to access the flir module over SCI and fill an image buffer.

and we fill our image buffer over usb

please post my a "ls -al" of 1 or 2 minutes EP0x85#00xxx.bin, so I can see how often your system got broken frames

[cynfab]

Hi tomas123,
Looks interesting, I'm tied up all weekend and into next week, so I probably won't get to do anything with the FlirOne till Xmas.

   ...ken...

[cynfab]

Hi tomas123,
I got some some time to fiddle with the flir code and here is a "simplified" version that works pretty well.
With Guvcview it gets almost to 9fps on my i7 linux Mint 17.3 desktop.


   ...ken...

[tomas123]

With Guvcview it gets almost to 9fps on my i7 linux Mint 17.3 desktop.

sorry, my mistake...

Code: [Select]

r = libusb_bulk_transfer(devh, 0x83, buf, sizeof(buf), &actual_length, 100); ...because the "old" code polls permanently the empty EP 0x81 and 0x83 with a timeout of 100ms, therefore we got only a max frame rate of 1000/(2*100)=5 fps

if we set in the old code flir7i.c (write frames to file) the timeout of EP 0x85 to 10ms we get to 9 fps

this is one second and I count 9 thermal frames

$ /usr/bin/gcc -std=c99 '-I/usr/include/libusb-1.0'  -o flir flir7i.c -lusb-1.0 -lm
$ sudo ./flir
...
$ ls -al --time-style=full-iso *0x85*
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.054600840 +0100 #00551-EP0x85.bin
-rw-r--r-- 1 root root  38828 2015-12-20 23:20:34.084600631 +0100 #00552-EP0x85.bin
-rw-r--r-- 1 root root  73008 2015-12-20 23:20:34.114600423 +0100 #00553-EP0x85.bin
-rw-r--r-- 1 root root    167 2015-12-20 23:20:34.144600214 +0100 #00554-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.164600075 +0100 #00555-EP0x85.bin
-rw-r--r-- 1 root root  38828 2015-12-20 23:20:34.194599866 +0100 #00556-EP0x85.bin
-rw-r--r-- 1 root root  73079 2015-12-20 23:20:34.224599657 +0100 #00557-EP0x85.bin
-rw-r--r-- 1 root root  39880 2015-12-20 23:20:34.294599171 +0100 #00558-EP0x85.bin
-rw-r--r-- 1 root root  73415 2015-12-20 23:20:34.324598962 +0100 #00559-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.384598544 +0100 #00560-EP0x85.bin
-rw-r--r-- 1 root root  38828 2015-12-20 23:20:34.414598336 +0100 #00561-EP0x85.bin
-rw-r--r-- 1 root root  73175 2015-12-20 23:20:34.444598127 +0100 #00562-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.514597640 +0100 #00563-EP0x85.bin
-rw-r--r-- 1 root root  38828 2015-12-20 23:20:34.544597432 +0100 #00564-EP0x85.bin
-rw-r--r-- 1 root root  73647 2015-12-20 23:20:34.574597223 +0100 #00565-EP0x85.bin
-rw-r--r-- 1 root root 112768 2015-12-20 23:20:34.644596736 +0100 #00566-EP0x85.bin
-rw-r--r-- 1 root root    167 2015-12-20 23:20:34.664596597 +0100 #00567-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.734596110 +0100 #00568-EP0x85.bin
-rw-r--r-- 1 root root  39852 2015-12-20 23:20:34.764595901 +0100 #00569-EP0x85.bin
-rw-r--r-- 1 root root  73048 2015-12-20 23:20:34.794595693 +0100 #00570-EP0x85.bin
-rw-r--r-- 1 root root    167 2015-12-20 23:20:34.814595553 +0100 #00571-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.844595345 +0100 #00572-EP0x85.bin
-rw-r--r-- 1 root root  39852 2015-12-20 23:20:34.874595136 +0100 #00573-EP0x85.bin
-rw-r--r-- 1 root root  73575 2015-12-20 23:20:34.904594927 +0100 #00574-EP0x85.bin
-rw-r--r-- 1 root root     28 2015-12-20 23:20:34.964594510 +0100 #00575-EP0x85.bin
-rw-r--r-- 1 root root  38828 2015-12-20 23:20:34.994594301 +0100 #00576-EP0x85.bin
it's interesting , with 10ms timeout we get no jumbo frames (header+thermal+visible+status) but rather splittet frames

My next step is to write a pipe and stitch the splittet frames together to a full frame...

[Lord of nothing]

Its possible to search for Animals in the Woods with the Cam?
I know the resolution is very low. 

[tomas123]

@Lord of nothing
I think, this thread is better qualified for your question:
https://www.eevblog.com/forum/testgear/actual-resolution-of-flir-one-v2/

@cynfab

I saw, that you removed the frame buffer part.
As attachment my last "Flir to v4l2loopback" version without a framebuffer.

compile:

Code: [Select]

$ /usr/bin/gcc '-I/usr/include/libusb-1.0'  -o flir flir8f.c -lusb-1.0 -lm
$ sudo ./flir
Successfully find the Flir One G2 device
Successfully set usb configuration 3
Successfully claimed interface 0,1,2
using output device: /dev/video0
     vid_format->type                =2
     vid_format->fmt.pix.width       =160
     vid_format->fmt.pix.height      =120
     vid_format->fmt.pix.pixelformat =1497715271
     vid_format->fmt.pix.sizeimage   =19200
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=160
     vid_format->fmt.pix.colorspace  =8
using output device: /dev/video1
     vid_format->type                =2
     vid_format->fmt.pix.width       =640
     vid_format->fmt.pix.height      =480
     vid_format->fmt.pix.pixelformat =1196444237
     vid_format->fmt.pix.sizeimage   =1228800
     vid_format->fmt.pix.field       =1
     vid_format->fmt.pix.bytesperline=0
     vid_format->fmt.pix.colorspace  =8
stop interface 2 FRAME
stop interface 1 FILEIO
start interface 1 FILEIO
...

new feature:
- add chunks from EP0x85 to a full frame (thermal+jpg+status)
- calc addresses inside "full frame" with infos from your reversed engineered header
- write thermal images to /dev/video0 as grayscale video 160x120
- write visible images to /dev/video1 as mjpeg video 640x480

please note, you must load v4l2loopback with /dev/video0 and /dev/video1

Code: [Select]

// unload v4l2loopback
$ sudo modprobe -r v4l2loopback

$ ls /dev/video*
ls: cannot access /dev/video*: No such file or directory

$ sudo modprobe v4l2loopback video_nr=0,1

$ ls /dev/video*
/dev/video0  /dev/video1

now you can view thermal and visible video simultaneously (i.e. with your guvcview in two windows)

I haven't changed the initialization sequence for the end points, because it's the same like in the SDK. I think, it's the safest way for future firmwares etc.

Your advice on the correlation between "sum of time outs" and "frame rate" was great   
I played around with some time outs and found this as the best solution for my raspberry pi:

Code: [Select]

r = libusb_bulk_transfer(devh, 0x85, buf, sizeof(buf), &actual_length, 100);
...
r = libusb_bulk_transfer(devh, 0x81, buf, sizeof(buf), &actual_length, 10);
r = libusb_bulk_transfer(devh, 0x83, buf, sizeof(buf), &actual_length, 10); EP 0x81 and 0x83 are almost empty (time out of 10ms works fine) and EP 0x85 lose regularly some bytes between chunks, if the time out is shorter as 100ms.

For debugging the code prints the status messages from EPx85.
On the basis of usbNotifiedTimestamp we get the frame rate of 9fps.
As sample a part of the terminal log with a FCC (shutter movement).

Code: [Select]

{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.02701,"usbEnqueuedTimestamp":1175531262.02879,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.14032,"usbEnqueuedTimestamp":1175531262.14213,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.25351,"usbEnqueuedTimestamp":1175531262.25556,"ffcState":"FFC_VALID_IMG"}
{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.36882,"usbEnqueuedTimestamp":1175531262.37076,"ffcState":"FFC_VALID_IMG"}
{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.48126,"usbEnqueuedTimestamp":1175531262.48321,"ffcState":"FFC_VALID_IMG"}
{"shutterState":"ON","shutterTemperature":312.040008544922,"usbNotifiedTimestamp":1175531262.70933,"usbEnqueuedTimestamp":1175531262.7114,"ffcState":"FFC_DESIRED"}
{"shutterState":"ON","shutterTemperature":311.970001220703,"usbNotifiedTimestamp":1175531262.82371,"usbEnqueuedTimestamp":1175531262.8254,"ffcState":"FFC_PROGRESS"}
{"shutterState":"ON","shutterTemperature":311.970001220703,"usbNotifiedTimestamp":1175531262.93882,"usbEnqueuedTimestamp":1175531262.94059,"ffcState":"FFC_PROGRESS"}
{"shutterState":"ON","shutterTemperature":311.970001220703,"usbNotifiedTimestamp":1175531263.05156,"usbEnqueuedTimestamp":1175531263.0532,"ffcState":"FFC_PROGRESS"}
{"shutterState":"FFC","shutterTemperature":311.970001220703,"usbNotifiedTimestamp":1175531263.16526,"usbEnqueuedTimestamp":1175531263.16728,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"FFC","shutterTemperature":311.970001220703,"usbNotifiedTimestamp":1175531263.2799,"usbEnqueuedTimestamp":1175531263.28168,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"FFC","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531263.39377,"usbEnqueuedTimestamp":1175531263.39579,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"FFC","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531263.96461,"usbEnqueuedTimestamp":1175531263.96619,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"FFC","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.07757,"usbEnqueuedTimestamp":1175531264.07966,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"FFC","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.19097,"usbEnqueuedTimestamp":1175531264.19271,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.30465,"usbEnqueuedTimestamp":1175531264.30635,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.41937,"usbEnqueuedTimestamp":1175531264.42115,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.5326,"usbEnqueuedTimestamp":1175531264.53417,"ffcState":"FFC_VALID_RAD"}
{"shutterState":"ON","shutterTemperature":311.959991455078,"usbNotifiedTimestamp":1175531264.64702,"usbEnqueuedTimestamp":1175531264.64891,"ffcState":"FFC_VALID_RAD"}
Note the difference between shutterState  and ffcStat!

[cynfab]

Hi tomas123,
Yep, I removed the framebuffer code. That's because my current target is to stream the thermal and visible images from a headless embedded processor.
Here is a video I captured with Guvcview. Encoding is MPEG 1 so the pixels are a bit fuzzy.

Just remove the .txt and play it with totem on Linux.

I'll try our your new code asap.....Maybe tomorrow... lots of snow here, need to plow the driveway again... and again... and again........

   ...ken...