CSQ file format

[gtatters]

So far, I found a temporary solution, using a python script from https://github.com/mypalmike/csplitb
instead of perl.

Install python (v2.7 is fine), then install pip (https://github.com/BurntSushi/nfldb/wiki/Python-&-pip-Windows-installation), then

Code: [Select]

pip install csplitb
then break up the .csq using:

Code: [Select]

python.exe csplitb.py --prefix frame --suffix .fff --number 5 46464600525450 thermalvid.csq
"46464600525450" is the string to search for each fff start.

Anyhow, I managed to get it to work in windows!  If I figure out the perl approach, I'll post a solution.

[tomas123]

Hi gtaters,

I took a look inside the perl docs and here is the solution (windows 10):

the perl script.pl

Code: [Select]

#!/usr/bin/perl
$n = 100;
$pat="\x46\x46\x46\x00\x52\x54";

binmode F;
open F, '<:raw', 'IR_2017-05-19_0222.csq'
    or die "Could not open file !";
$file = do { local $/; <F> };
close F; 
 
# Flir Tools (comment out)
#$pat = "\x46\x46\x46\x00\x43\x41\x50";
for my $content (split(/(?=$pat)/, $file)) {
        open(OUT, ">seq" . ++$n . ".fff");
        binmode(OUT, ":raw");
        print OUT $content;
        close(OUT);
}


used with strawberry-perl-5.18.1.1-32bit-portable

Code: [Select]

>strawberry-perl-5.18.1.1-32bit-portable\perl\bin\perl.exe -f split.pl IR_2017-05-19_0222.csq

>exiftool -b -RawThermalImage seq*.fff -w _%f.jpgls
  168 image files read
  168 output files created

>ffmpeg -f image2 -vcodec jpegls -start_number 101 -i _seq%3d.jpgls output.mp4
ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers
  built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
[libx264 @ 04223000] kb/s:130.01
please unzip the attachment for viewing the video

[gtatters]

Hi Tomas,

I have made progress and had success!  Thanks to your help and in combination with a friend who works in perl, I've managed to create my own perl split.pl file I can run with arguments.  See code below. This can split based on some of the known FLIR headers, jpgels, or tiff.

I verified that this works on CSQ or SEQ files running perl on my Mac and running strawberry perl (portable) on a Windows 10 system.

Thank you for all your help.  Code below might be clumsy, but it works for me.

Code: [Select]

#!/usr/bin/perl

#use strict;
#use warnings;
use Getopt::Std;

our($opt_i, $opt_o, $opt_b, $opt_p, $opt_x);

my $n=0;
my $folder = "temp";
my $infilename = "filename.csq";
my $outfilebase = "frame";
my $outfilename = "";
my $pattype = "fff";
my $patfff = "\x46\x46\x46\x00"; # split a generic fff header (some seq files recorded on camera have simple headers like this)
my $patfcf = "\x46\x46\x46\x00\x43\x41\x50"; # split the fcf file type header (older video type)
my $patseq = "\x46\x46\x46\x00\x43\x41\x4D"; # split the seq file type header (based on camera controlled by computer)
my $patcsq = "\x46\x46\x46\x00\x52\x54\x50"; # split the csq file type header (compressed video format)
my $patjpegls = "\xff\xd8\xff\xf7"; # split out jpegls headers from .raw file generated from exiftool
my $pattiff = "II\\*\0";
my $outext = "fff";
my $file = "";

my (%opt)=();
getopts("h:i:o:b:p:x:",\%opt);
if ($opt{h}){
print qq(
    Usage: perl split -i filename -o outputfoldername -b basename -p splitpattern -x outputfileextension
    options:
    -i input filename
    -o output folder
    -b base output name
    -p split pattern (fff, fcf, seq, csq, jpegls, tiff)
    -x output extension (fff, jpegls, tiff)\n.);
    exit 
}

if (!defined $opt{i} & !defined $opt{o} & !defined $opt{b} & !defined $opt{p} & !defined $opt{x}){die "Error: Please specify input file, output folder, the output filename base, pattern to split, and output file extension.\n"}
if (!defined $opt{i}){die "Error: Please specify the input file. \n"}
if (!defined $opt{o}){die "Error: Please specify the output folder.\n"}
if (!defined $opt{b}){die "Error: Please specify the output filename base.\n"}
if (!defined $opt{p}){die "Error: Please specify the split pattern.\n"}
if (!defined $opt{x}){die "Error: Please specify the output file extension.\n"}

my $pattype = "$opt_p";

$folder = $opt{o};
$infilename = $opt{i};
$outfilebase = $opt{b};
$outext = ".$opt{x}";

if ($opt{p} eq "fff"){
        $pat = $patfff;
    } elsif ($opt{p} eq "fcf"){
        $pat= $patfcf;
    } elsif ($opt{p} eq "seq"){
        $pat = $patseq;
    } elsif ($opt{p} eq "jpegls"){
        $pat = $patjpegls;
    } elsif ($opt{p} eq "tiff"){
        $pat = $pattiff;
    }

     else {die "Error: Please specify the split pattern.\n"}

# Create output folder to store split files
unless(-e $folder or mkdir $folder) {
       die "Unable to create $directory\n";
}

print "Processing $infilename\n";

# Use binary mode for portability between operating systems
binmode F;
open F, '<:raw', $infilename;
$file = do { local $/; <F> };
close F; 


# Split infilename based on $pat
for my $content (split(/(?=$pat)/, $file)) {
        $outfilename = ">$folder/$outfilebase" . sprintf("%05d",++$n) . $outext;
        open(OUT, $outfilename);
        binmode(OUT, ":raw");
        print OUT $content;
        close(OUT);
}



A worked example and further details are elaborated here:
https://github.com/gtatters/Thermimage/blob/master/BashConvertFLIR.md

[tomas123]

thanks for sharing your results

[lukaszgryglicki]

Hi, thanks for the help.
I've bought FLIR E75 thermal camera and FLIR tools cannot be installed on Mac.
But I've written a Go program that takes *.csq file, splits into JPEGLS files then uses a shell script (which uses convert and FFmpeg) to generate a final image.
I've registered only to put a link to my program, hopefully somebody find it useful:
https://github.com/lukaszgryglicki/csqconv