TEQ to CSV conversion

[NHFranco]

Dear all,

Noob here.

I'm Nuno Franco, a biologist from Porto, Portugal, and I've started working on thermal imaging applied to animal welfare science.

Since I couldn't get proper funding, I had to make do with a TE - Q1, and I have to say that until now I've been getting interesting results. It saves in proprietary .TEQ files, in PNG and JPEG, the latter coupled with CSV files

Two colleagues from the Bioengineering department managed to write a MATLAB script that reads the CSV (now compiled to a small app) that takes each image or a batch of images, removes background temperature (i.e. whatever is not an animal in the picture) and gives mean and median surface temperature, with some nice histograms to go with it. We expect to write an open-access paper on it until the end of March.

It's a painstaking job to use the proprietary software to contour every animal in every picture, and then make an average, and this is why I asked my colleagues to develop the MATLAB script.

Problem is we have some data collected from before I knew I'd be using this app, but now we can't use it, because it's all in TEQ format.

So I'd like to know if there's any way of taking the data in TEQ files to build SCV files? (Joe-C )? . 

Thanks

[rhb]

If no one offers up a conversion program, post the information you have on the TEQ format and an example input file.  No promises, but likely it's a pretty simple task.   It would help if the TEQ example is something simple like a hot stripe against a uniform background.

[NHFranco]

Thank you.

TEQ files are relatively large (2 MB), so I've put one up on this link: https://app.box.com/s/pxjw8j7qisuk4vxp6e4k66t1alcqq8cy

I attach a screen grab of how it looks like in the TE Q1 Analyst software. There's not much on the TEQ file extension, the users' manual states that:

teq is customized format for saving Thermal Expert
image information in Thermal Expert Q1 program. It saves
current image information which contains temperature data,
colormap, visible image (if exist), number and position of
markers, and emissivity. When teq file is loaded, those
information are loaded and applied automatically. Since this
format is customized format, it cannot be used in ordinary
image viewer

[rhb]

A simple image is important.  The example image is too complex to attack at the byte level.  A warm or cold bar on a constant background is what's needed along with all of the data you can extract using their software (e.g. value ranges, compression type, colormap name, etc)

Have you asked  Thermal Expert for information about the format?  OEMs will sometimes provide information to academics that they won't supply to regular customers. 

[mahony]

Hi,
I had a quick look at the image format.
There is the raw data (16bit) after 498 header bytes. I tried to attach a .png image file but that does not seem to work...
Not sure about the rest but might be some calibrated image in there too.

Regards
Toni

Edit: .png was above 1MB -> jpg did the job 

Edit2: using int32 you can see the same image again after the first image block... interesting

[mahony]

Even easier than thought... just use the second block as float32 - this directly gives you temperatures I assume.
The attached image was read in a simple sequence as float32 values and scaled to display values from 22 (blue limit) to 38 (red limit). In the case of this image the offset is 221680 bytes, might be different in other images - just check it. See the final attached temperature image.

What I find really interesting is the fact that you have got the same 'line-shift' that I have. That is your leftmost line seems to be belonging in the rightmost place. I have got the same behaviour on my cam. 

Edit: just checked with your posted image above. The value seem to be spot on. Very easy file reverse engineering

[NHFranco]

Now I'm sure I'm way out of my league, here (hence why I asked coleagues from the bioengineering department to help me with this).

Is there a way I could convert the TEQ files to CSV? I can try to get a simpler image, as rhb suggested, if it helps.

BTW, I'm pretty sure my colleagues wouldn't mind sharing the program we are testing for CSV analysis. It's still at the Beta stage, but we're talking about making it freely available, once the paper it's published.

[rhb]

I was waiting to hear if someone had a translator before starting to write something.  Mahony provided what should be enough information.  My request for a simpler image was concern about things like the scan line being on the wrong side.  Mahony has the advantage of experience with thermal imaging.  I'm too cheap to buy one, though it would be nice to have.

I'll try to work in time to write a small C program today.  Normally I use awk for format conversions, but awk doesn't handle binary.  I've just never gotten enthused about python or perl for the simple reason I was very good with parsing very complex formats with awk.  This is an instance where python or perl would be a more portable choice.

[rhb]

I'm having problems with the format.  Could you post an image both in TEQ and CSV format? 

There appears to be a header, a pair of image blocks and then more data.  I'm concerned about the data after the images.

[rhb]

OK  Here's what I know:

496 byte header
384x288  16 bit int image
384x288 32 bit float image
600x600 32 bit image

The first two images appear to be thermal.  The 3rd image is the regular photo image stored 4 bytes per pixel.

[mahony]

Nice catch! I assumed there might be some more  Image data but was  not aware of the  visual photo. 
I am currently not at Home, otherwise I might able to write a quick C# app to extract the floating point temperature data and dump it to a csv. Maybe someone else can help out?
On the other hand, it should be only a couple on lines in matlab to read the temperature data from teq.

[rhb]

Your  header length got it close enough to take the time.  I don't have a TIR camera and at the current prices, not likely to.  I just happen to really despise proprietary formats.  I've made a lot of money translating them, but I should very much prefer to have spent that time doing something more interesting.

Once I have the CSV version so I can parse out the header I'll write a program to do the conversion to produce identical CSV files.

[NHFranco]

It can't save simultaneously in two different formats, but I can go downstairs to the lab and take a few simple consecutive images for each format.

[NHFranco]

Hi Mahony. 

We chose CSV because many thermal cameras can save in this format, or have software tools available for converting to it. But if can be done by adding a few lines on MATLAB, with your guidance I'm sure my colleagues could sort that out (as for me, I'm a Biology geek, not a tech geek  ).

[rhb]

It can't save simultaneously in two different formats, but I can go downstairs to the lab and take a few simple consecutive images for each format.

Pick a simple static image.  A picture of a contact thermometer would be ideal if you have one handy, but not necessary.  I want to examine the header for data not included in the CSV file.