Turning AFFIMAGE into real pictures

[Ultrapurple]

I am continuing my journey with the FLIR Integrator 5/20 Photon-based camera. So far I have established serial comms, found some commands, and even been able to retrieve what looks like image data from the serial port via the affimage? command.

Trouble is, I haven't yet been able to find out anything about the format of the data dump. So I'm appealing to the community for help.

The data clearly is an image. There's a header section followed by 320 x 240 16-bit hex words. I've even (somehow) managed to get something resembling an image out of the data by reading it as a RAW image (myfile2.raw) into Paint Shop Pro. Stripping out the CR/LF was key to that. Unfortunately I could only work out how to read it as 8-bit greyscale (the data's 16-bit) so I ended up with a posterised image of 640x240 (myfile2.bmp), which I then stretched to 640x480 (myfile2-as-640-wide.bmp). The samples were taken with a variety of things onscreen such as inverse video, a menu, different reticle - but I don't know whether these will have come through or if it's just raw image data.

I'm a bit stuck after this - so can anyone help?

I have attached six sample files taken from the camera's RS-232 output (dumped to disk using a terminal program and then extracted). These files, Sample_01 to _06.txt also include the "AFFIMAGE!154680" response from the camera, which I believe is the size of the dump in bytes (the file is twice that size because each byte is sent as two hex characters).

There's also a jpg screengrab from the video output that shows roughly what the (unexciting) picture should show. Unfortunately I knocked the camera (pan only) between taking the data dump and making the grab so the views aren't quite the same, but they're close enough. All the files are contained in the zip.

My feeling is that someone who speaks fluent exiftool will probably be able to come up with the One True Incantation^® necessary to turn the data into wine a manipulatable image - but I lack the experience and knowledge necessary. So can anyone help?

Of course, if there's something in the FLIR camera command set that will cause it to spit out nicely formatted TIFFs or somesuch I'm all ears for that!

Any and all help will be gratefully received and re-shared to the community.

[Fraser]

Currently searching my archive for AGEMA File Format information....

The attached is my first point of call as it details AFF.

Fraser

[Fraser]

Thermacam Explorer 99 was provided with the PM series cameras as a means to view the image files that the camera produced.
Sadly I do not think it will work with the RAW AFF data but I have never investigated creating an image file using AFF data.
I attach Explorer 99 from 2004 for interest. There is a later version from 2007 but it is too large to attach.

Explorer is a viewer and not an analyser software.

The FLIR Researcher software is capable of controlling Thermovision cameras and downloading images over RS232 so that likely works with AFF RAW data.

Fraser

[Bud]

The first and second images look byte-swapped from my experience. There were scripts published earlier elsewhere on this forum that swap byte order , cant remember though the thread and if the scripts were for exiftool or just some Perl scripts. You should be able to swap MSB/LSB in ImageJ , try playing with ImageJ. It is possible it will just open the image properly as long as you specify proper header offset.

[Miek]

It looks like you're most of the way there. I agree about it looking byte-swapped, and also once you get it correct it will just look like a single shade of grey - you'll have to adjust the levels in your editor to see any contrast.

I can't tell if you're doing any ascii -> binary conversion? If not, that may be why you're seeing the stripes.

I had a quick go at loading it up. Bit of linux commandline-fu to convert the hex:

Code: [Select]

$ cat ~/Downloads/ultrapurple-affimages/Sample_01.txt | sed 1d | xxd -r -p - > myfile.img
then loaded it in GIMP as "Raw Image Data", 16bit unsigned grayscale big-endian, 320x240, and adjusted the levels:

[bap2703]

There's some metadata before the image.
I think it's better to take the last 320*240*4 characters only (except the CRLF).

[Ultrapurple]

@Fraser, Bud, Miek, bap2703 - thank you all very much.

@ Fraser (and Miek) - section 9 of ICDig16.pdf is the key, thank you. I have yet to get ThermaCam Explorer to run (I'm using a Win10 PC but I have other machines back to XP).

@Bud - yes, I think you're spot on about the byte-swap.

@Miek - Ooh - a Linux speaker!!
I confess I have only had bad experiences with *ux; my Raspberry Pis run RiscOS and get programmed in BASIC! (Though I haven't tried using one to talk to a serial port and dump data to disk, which would be a logical thing to try.

@bap2703 - yes, you're absolutely right.

[Ultrapurple]

Hokay, some progress to report. I can now turn the data dumped from RS-232 into a picture, more or less, using some rather skanky BASIC code. (BBC BASIC is the only language I am comfortable with - I taught myself it when I was in my early teens and haven't ever really needed anything else, despite developing several successful microcontroller-based products over the years).

Back to a description of the code. First, I ignore the header section, then I scan the 'payload' (image data), converting each 4-character ASCII 'hex' into a figure, which I use to determine the lowest and highest data (and so set my black and white points).

Then I go through the payload again, subtracting the black level and scaling the remainder proportionately to the 'peak white' I established in the first run, and output each (now 8-bit) sample to a file, which I can then import as a raw (headerless) image into my old standby Paint Shop pro X.

The program wouldn't win any awards for style or efficiency but it does all the processing in about one second so I'm not disposed to optimise it (yet).

These images show the processed image as it arrives in Paint Shop Pro, and the same image after a 'one step photo fix'.

Hokay, some progress to report. I can now turn the data dumped from RS-232 into a picture, more or less, using some rather skanky BASIC code.

First, I ignore the header section, then I scan the 'payload' (image data), converting each 4-character ASCII 'hex' into a figure, which I use to determine the lowest and highest data (and so set my black and white points).

Then I go through the payload again, subtracting the black level and scaling the remainder proportionately to the 'peak white' I established in the first run, and output each (now 8-bit) sample to a file, which I can then import as a raw (headerless) image into my old standby Paint Shop pro X.

The program wouldn't win any awards for style or efficiency but it does all the processing in about one second so I'm not disposed to optimise it (yet).

The next step will probably be to add a suitable header to the file so I can use it properly as an image without having to go through the PSP import step. It's still going to be clunky though, at least for now. But at least I'm getting somewhere!

Then I suppose I'll start looking at nonlinear palettes and - who knows? Possibly making higher bit depth images.

[Ultrapurple]

Another sample - the data file, resulting image, and an analogue screen grab.

[Ultrapurple]

Just to add to the fun, I have been using the free version of BBC BASIC for Windows, which has a program+data limit of 32kB - yes, 32 kilobytes! (The paid versions have no such limit). Ordinarily I would simply have loaded the image data into a 320x240 array but unfortunately that significantly exceeds the memory limit. So I presently do the processing by file operations - reading in a byte at a time, assembling the bytes into words, performing the calculations, then outputting to another file.

I did mention it was a bit skanky, didn't I?

It certainly concentrates the mind when the entire memory map is significantly less than the amount required by one 8-bit 320x240 pixel image. I'm beginning to understand how the early mainframe programmers had to work, manipulating information between data tapes (or, perish the thought, punched cards - one 320x240 8-bit image would need the better part of a thousand cards).

One major bottleneck remains the 19,200 baud comms link with the camera. Given that each image comprises a header of a couple of thousand characters followed by image data of 320 x 240 sixteen-bit pixels, each requiring four ASCII characters to represent in hex, there's a bit more than 300,000 bytes to transfer for each image, or some 5 million bits. Divide 5 million by 19,200 bits per second and you find that each image takes around 5 minutes to read out of the camera.

[Bill W]

Sounds rather like the Argus3 speed when we added image capture !

As for file manipulation, had you considered reading into a spreadsheet ? Plenty of bit swapping functions etc available
Columns A to LH x rows 1 - 240.
Then use a new 'page' for the manipulations. 

Bill

[Ultrapurple]

Thanks Bill

I did consider other approaches but I was stymied by the data format: basically, 4000 or so lines like
46B946B946B346B146AF46AE46AF46AC46AC46AE46AC46AD46B146B246B046B746B746B746B6<cr><lf>
- I couldn't work out a way of importing such non-comma-delimited data. Perhaps there is a method, but I didn't research it thoroughly.

I suspect it's possible to switch the baud rate of the camera to something substantially higher but I've had problems with my el-cheapo RS232-USB cable getting confused. Issuing the command is no problem, but the camera then responds "OK" at the new baud rate, leaving no time for the host PC to adjust itself. I shall have to investigate resetting the terminal software, but my limited experiments so far suggest it's an issue at the hardware level.

[Fraser]

Ultrapurple,

This thread may be of interest as I think your camera contains the PCB interface that can connect to a TAXI communications PCB. Sadly the optional TAXI PCB itself is unobtainium and, even when available, very expensive. Some of my cameras came with this option fitted.

https://www.eevblog.com/forum/thermal-imaging/getting-digital-data-out-of-my-pm695/

It would likely be possible to extract higher speed digital data direct from the port that serves the TAXI board though.

Fraser

[Ultrapurple]

Thanks Fraser - something to consider in the longer run perhaps.

Looking at my admittedly poor X-Ray of the RSIL board, there's definitely a 'magnetic' on there (centre bottom):



Unfortunately the X-Ray is rotated 180° compared to the visible image but the 'magnetic' (marked 78250 // G0314) is pretty clear. I shall have to investigate what it does. My first thought was Ethernet but on reflection that's unlikely because there aren't enough independent sections for that. And anyway, realistically speaking, all the other wiring is wrong.



Mind you, with four 16c550 UARTs and four LT1381 RS-232 line driver/receivers - and a board name of RISL it does look rather more serial interface than anything else!

[Ultrapurple]

I have confirmed one useful point about the data that comes via the serial port: it represents a snapshot taken at the time the affimage? command is sent, rather than being 'live' pixels. This is useful because it means the subject only has to stay still for 1/25 of a second, rather than five minutes while the data is churned out.

[Miek]

You should be able to speed it up by increasing the baud rate. My camera supports up to 38400Bd by sending "BAUDRATE:38400"

[Fraser]

The cameras original deployment may have included RS422 and/or RS485 communications.

Fraser

[Fraser]

The 78250 is a 1:1 power transformer used for isolated RS232 communication links.
Commonly used with the MAX250/251

http://www.farnell.com/datasheets/2648937.pdf

https://uk.farnell.com/murata-power-solutions/78250c/transformer-0-002h-dil-rs232/dp/1610414

Fraser

[Fraser]

MAX250 use of the 78250 as part of an isolated communications link.

http://www.farnell.com/datasheets/1802416.pdf

Fraser

[Fraser]

I just had a thought based on an off-line comment from Ultrapurple and from seeing pictures from the camera..... I think I know what the RiSL board is as the Main RiCo has its own RS232 ports. I the RiSL could be the “Serial Link” interface board for accessories.

The camera was originally deployed with a visible light camera and Ultrapurple advised that the thermal camera warns that the visible light camera is not detected. There must be communications between the two cameras. That would account for one RS232 link of the RiSL board. The Visible light camera is a sophisticated design and resides in its own housing on the Pan/Tilt head.

The image captures from the camera reveal Pan Tilt mount data. That data will be coming from a controller within the original Pan Tilt head with which this camera was integrated. That will need a second RS232 port on the RiSL board.

The Lens may communicate with the camera RiCo. Could it be that there is a Serial port, or two (Zoom & Focus) used for lens controller communications ? That would account for one, possibly two RS232 ports on the RISL board. Granted, standard UART communications would work for the Lens assembly due to its close proximity so I may be wrong regarding the Lens control.

It could be that the FLIR Dual FOV camera module is designed to be versatile and has both primary any and auxiliary communications ports. The primary port(s) connect to the camera head user control panel and the Auxiliary RS232 ports support supplemental systems associated with the camera head. The Camera module effectively acts like a RS232 concentrator. (A sort of serial router)

For information, military and law enforcement camera heads often include a visible light camera, thermal camera, Laser range finder, Laser target marker and gimbal control/stabiliser systems. All such systems require communications with a main controller. RS485 and RS422 are multi-drop capable systems, so preferred, but RS232 remains a possibility.

Just food for thought 

Fraser

[Fraser]

I need to revisit the original thread that I started on this camera and study the pictures of the various PCB’s used in the camera. From memory, the lens block had a complex PCB associated with it, or at least bolted to the lens casing.

Fraser

[Fraser]

A quick check confirms that the cameras lens system is not routed via the RiSL board. It is connected to the RiCo via. 25 pin D Type connector pair and white wiring. The Lens assembly does employ a surprisingly powerful control board ! Two PIC microcontrollers and a CPLD. Exact use not known at this time except to say that it likely has control over Zoom and focus actuators plus feedback of lens element position to the RiCo.

In the second picture you will see that the RiSL board services the external D type connector with its I/O ports. This further supports my theory that the RiSL is an auxiliary device communications board. This board is not present in a PM695 system as the PM695 is a fully integrated solution with no auxiliary RS232 ports except for the remote control port used by a PC host or Remote Controller.

Fraser

[globoy]

I think you already figured this out but the data is probably camera telemetry followed by the raw 16-bit counts from the sensor.  If you want, you can do a quick linear scaling on the data to 8-bits which is then suitable to push through a 8->24 bit colormap for false color images.

 1. Find the minimum and maximum values in the data
 2. Scale the data to 8-bits using the minimum and maximum: p8 = 255 * (p16 - min) / delta
 3. Index a 256 entry colormap table with the 8-bit data to get RGB for each pixel in the image

I wrote a quick&dirty program that reads your text files and displays an image.  PM me if you're interested in it.

[Ultrapurple]

@globoy - thanks for this

Yes, I'm pretty-much doing what you're suggesting, with the exception of the final step to make a pretty palette. That's on my 'nice to have, later' list of things to do, so the method you suggest is one I'll keep in mind.

I managed to do some more work on camera control and live video capture. I (finally) found an Android app that would talk to the cheap USB-RS-232 adapters I bought, so I had two old phones connected to the FLIR - one via a dongle to capture stills & video, the other to send commands.

If I though it was painfully slow typing individual commands on a full-size keyboard and sending them to the camera, I hadn't reckoned on the hassles involved in using an on-screen phone 'keyboard'!

But the good news is that the setup was portable-enough to take out into the world (OK, my doorstep) and controllable-enough to take some pictures at both 20° and 5° fields of view, check out the autofocus, and see what the real image quality was like.

[Ultrapurple]

Sample stills from Thermacam Integrator 5/20, all captured from analogue composite video output in PAL mode. Direct (digital) captures would be sharper. The black border is emplaced by the camera to 'letterbox' the NTSC-resolution into a PAL space.

01 Basic wide field of view capture

02 Basic narrow field of view capture (note how the narrow view is exactly that encompassed by the central reticle in the wide view)

03 Narrow FoV with 'Rainbow' palette

04 Wide FoV with 'Rainbow' palette

05 Wide FoV of tree

06 Narrow FoV of tree