In FLIR One, at what points are the upscaling applied?

[Ben321]

I know that the final output is upscaled 4x (from 160x120 to 640x480) via the MSX technique that combines the visible light image and LWIR image, and that a 2x upscaled (to 320x240) image is stored as the embedded "raw" image in the JPEG file. But at what points are these upscalings performed? I assume that the the 4x MSX upscaling is performed in software (the actual smartphone app), as the FLIR Lepton does not contain the visible light camera, and therefore the Lepton chip doesn't have access to the visible light image to perform MSX scaling in-hardware. However the 2x upscaling of the LWIR only image COULD be accomplished in-hardware, but I don't know if it is.

Does anybody here know at what point the LWIR image is 2x upscaled? If it's done in-software, I know of an easy way to get around it. Buy a copy of the FLIR One SDK, write an app which simply takes the raw LWIR image, and saves it to a 16bit grayscale TIFF file. This would then make the TRULY raw data easily accessible to any Windows PC with software such as Photoshop. However, if the 2x is performed in the Lepton chip itself, then there's no way around it.

If there's a microcontroller in the FLIR One, and the 2x upscale is performed in the microcontroller, it would be more difficult, but a hardware mod would be theoretically possible. You'd need to desolder the microcontroller (or unplug it from its socket if it uses one), connect it to a microcontroller programmer device, dump the program to ASM code, rewrite the applicable parts of the assembly code program, re-assemble the code, upload it to the microcontroller, and then re-connect (resolder it or plug it back into its socket if it uses one) the microcontroller.

[tomas123]

 MSX works in Software
see https://www.eevblog.com/forum/testgear/question-about-flir-one-for-android/msg832208/#msg832208

With free (!) SDK you get also RAW Lepton sensor values
https://www.eevblog.com/forum/testgear/flir-one-thermal-imaging-camera-teardown-and-hacks/msg799649/#msg799649

[Ben321]

With free (!) SDK you get also RAW Lepton sensor values
https://www.eevblog.com/forum/testgear/flir-one-thermal-imaging-camera-teardown-and-hacks/msg799649/#msg799649

And by raw, you mean mean TRULY raw, as in absolutely no upscaling or other processing at all? Does the SDK permit you to write Windows PC software, so that one could use the FLIR One on a the USB port of a desktop or laptop computer (via a USB to MicroUSB adapter)? Or does it only give one the ability to write mobile apps for the FLIR One?

If in fact it does only give one the ability to write mobile apps, has anybody yet used it to write an app that snaps truly raw images (160x120 16bit pixels), and then save them as uncompressed 16bit grayscale TIFF files? If so, that is one app I'd LOVE to have. It might actually make it worth spending $250 on the FLIR One.

[cynfab]

No, the SDK is only for android, but with the work that tomas123 and I have done, you may be able to adapt the flir8x.c code to run on Windoze.
The data that comes from the Flir One G2 via USB, is 14bit data 160 x 120 pixels. It has had FFC applied, but not factory calibration ( that's what Camerfiles.zip is for).
Calibration is performed by the native methods supplied as a binary library by the SDK: libjnidevicewrapper.so
Hopefully the Flir One SDK is the beginning of what Flir is going to provide, but don't hold your breath...

   ...ken...

[Ben321]

No, the SDK is only for android, but with the work that tomas123 and I have done, you may be able to adapt the flir8x.c code to run on Windoze.
The data that comes from the Flir One G2 via USB, is 14bit data 160 x 120 pixels. It has had FFC applied, but not factory calibration ( that's what Camerfiles.zip is for).
Calibration is performed by the native methods supplied as a binary library by the SDK: libjnidevicewrapper.so
Hopefully the Flir One SDK is the beginning of what Flir is going to provide, but don't hold your breath...

   ...ken...

What's FFC? And where is this flir8x.c C source code file you referenced? Where can I get a copy?

As for Android apps written using the SDK, has anybody yet made one that takes the raw data and saves it as a 16bit grayscale TIFF file? If so, that's one app I'd love to have. It would actually make it worth buying a FLIR One for my Android phone. As it is now, that 2x scaling applied to the raw data that's embedded in the JPEG saved by the official app, makes it not suitable to me. But if you can make an app that actually saves the raw 160x120 16bpp pixel data to a TIFF file, that would make it worth it for me to buy a FLIR One. Note, that the raw data actually comes with 4 extra pixels in each line (based on some images I've seen), to make it 164x120. These maybe calibration data or something, I'm not sure. But the TIFF file saved by the app that I'm picturing now, would get rid of those extra pixels and save only a 160x120 TIFF image. Note that the file extension should be tif, not tiff, in order to be compatible with most software. If such an app doesn't yet exist, please consider creating such an app.

[tomas123]

Hi Ben321,

three month offline and now so many questions?
Why do you don't read (and post to) your old thread "Question about FLIR One for Android" ?
https://www.eevblog.com/forum/testgear/question-about-flir-one-for-android/msg769836/#msg769836
You are the op (original poster / thread starter)!!

Following some links as answers.

What's FFC?

answer

see for "What is FFC?":
http://www.flir.com/cvs/cores/knowledgebase/index.cfm?CFTREEITEMKEY=327&view=35774

And where is this flir8x.c C source code file you referenced? Where can I get a copy?

use attachment from this post and follow the step-by-step totorial

MSX works in Software
see https://www.eevblog.com/forum/testgear/question-about-flir-one-for-android/msg832208/#msg832208

No, the SDK is only for android,

there is also a free SDK for IOS

And by raw, you mean mean TRULY raw

it's the same post processed RAW like all Flir Cameras (like E4, E40 etc.), that "the response of "A/D Counts" are linear to Radiance with an offset"
see my notes below

It has had FFC applied, but not factory calibration ( that's what Camerfiles.zip is for).
Calibration is performed by the native methods supplied as a binary library by the SDK: libjnidevicewrapper.so
Hopefully the Flir One SDK is the beginning of what Flir is going to provide, but don't hold your breath...

the Flir One is also calibrated with a black body like the Flir E4 or E40
the calibration values (exiftool says planck values) are stored inside Camerfiles.zip and in every Flir jpeg image, which is saved by the Flir One app (with your Flir One).

Code: [Select]

> exiftool -planck* -atm* FLIROne-2015-11-10-19-44-55+0100.jpg
Planck R1                       : 16528.178
Planck B                        : 1427.5
Planck F                        : 1
Planck O                        : -1307
Planck R2                       : 0.012258549
Atmospheric Temperature         : 20.0 C
Atmospheric Trans Alpha 1       : 0.006569
Atmospheric Trans Alpha 2       : 0.012620
Atmospheric Trans Beta 1        : -0.002276
Atmospheric Trans Beta 2        : -0.006670
Atmospheric Trans X             : 1.900000
all steps to calculate a calibrated temperature from a 14 Bit Flir RAW sensor values are documented
see attachment RAW2Temp.xls in my post
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072

more details:
http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,4898.msg23944.html#msg23944
and the attachment pages_from_Toolkit_IC2_Dig16_Developers_Guide_1_01.pdf
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/?action=dlattach;attach=83950

see also my attachment E4_planck.xls in this post to illustrate "RAW values are linear to Radiance [W/m²] (with an offset) and not linear to Temperature [K]".

that the response of "A/D Counts" are linear to Radiance  with an offset.

one day later and no reaction 

for better understanding I made a excel sheet

the Flir E4 has a spectral range from 7.5–13 um (the peak 10.6 um) http://support.flir.com/DsDownload/Assets/63901-0101_en_50.pdf
 
I wrote a table for the planck law at 10600 nm wave length over the temperature range
There you can see, that the spectral radiance in [W / m^2] (per nm wave length) is not exact linear

As for Android apps written using the SDK, has anybody yet made one that takes the raw data and saves it as a 16bit grayscale TIFF file?

answer:

Hi sonic,

I saw in your code, that you modified the function onFrameProcessed().

Here is my code modification for onFrameProcessed() for saving full size raw images to *.raw

Another quick n' dirty solution works fine:

Save the ThermalLinearFlux14BitImage RAW image:

FLIRONEExampleApplication/app/src/main/java/com/flir/flironeexampleapplication/PreviewActivity.java

Code: [Select]


// for raw images
import java.io.FileOutputStream;

   public void onFrameProcessed(final RenderedImage renderedImage){
        thermalBitmap = renderedImage.getBitmap();
        updateThermalImageView(thermalBitmap);


        /*
        Capture this image if requested.
        */
        if (this.imageCaptureRequested) {
            imageCaptureRequested = false;
            final Context context = this;
            new Thread(new Runnable() {
                public void run() {
                    String path = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
                    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd-HH-mm-ssZ", Locale.getDefault());
                    String formatedDate = sdf.format(new Date());
                    String fileName = "FLIROne-" + formatedDate + ".jpg";
                    try{
                        lastSavedPath = path+ "/" + fileName;
                        renderedImage.getFrame().save(new File(lastSavedPath), RenderedImage.Palette.Iron, RenderedImage.ImageType.BlendedMSXRGBA8888Image);

                        MediaScannerConnection.scanFile(context,
                                new String[]{path + "/" + fileName}, null,
                                new MediaScannerConnection.OnScanCompletedListener() {
                                    @Override
                                    public void onScanCompleted(String path, Uri uri) {
                                        Log.i("ExternalStorage", "Scanned " + path + ":");
                                        Log.i("ExternalStorage", "-> uri=" + uri);
                                    }

                                });
                        if (renderedImage.imageType() == RenderedImage.ImageType.ThermalLinearFlux14BitImage){
                            // save the Lepton RAW file
                            String rawName = "FLIROne-" + formatedDate + ".raw";
                            lastSavedPath = path+ "/" + rawName;
                            File file = new File(lastSavedPath);
                            FileOutputStream fop = null;
                            fop = new FileOutputStream(file);
                            if (!file.exists()) {
                                file.createNewFile();
                            }
                            //int rawLength = renderedImage.pixelData().length;
                            byte[] LeptonRAWBytes = renderedImage.pixelData();
                            fop.write(LeptonRAWBytes);
                            fop.flush();
                            fop.close();
                        }

                    }catch (Exception e){
                        e.printStackTrace();
                    }
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {

                            thermalImageView.animate().setDuration(50).scaleY(0).withEndAction((new Runnable() {
                                public void run() {
                                    thermalImageView.animate().setDuration(50).scaleY(1);
                                }
                            }));
                        }
                    });
                }
            }).start();
        }
...


If the image type is set to "ThermalLinearFlux14BitImage" then it save also the raw frame 160x120 to disk.

So I got the nice gallery photo from F1G2:
...

You can simple edit the code to change from  16 Bit *.raw files to a 16 Bit *.tif format.

here you see the difference between a  RAW ThermalLinearFlux14BitImage (SDK app 160x120) and a software upscaled thermal image (Flir One app 320x240)

Flir One G2 Lepton 160x120 Mode: ThermalLinearFlux14BitImage



Flir One G2 Lepton 160x120 with hacked config file (meaning without the crop factor)

.. I think, I posted in this forum many informations about Flir RAW, but only few readers also read this. 

[tomas123]

It would actually make it worth buying a FLIR One for my Android phone.

The price of the Flir One G2 Camera is only a promotion for the Lepton sensor.
It's the same end-user price as for a single Lepton sensor 160x120 with shutter.
Additional you get a black body calibration and the activation of the blocked feature "RAW linear to radiance".

160 x120 is the resolution of a FLIR E6 which cost  $2.500 in US / 3.000 EUR in Europe

I love the F1 G2 

[cynfab]

tomas123,
My mistake, silly me... you are of course correct that there IS an SDK for IOS as well as Android.

[tomas123]

Hi cynfab,

I wrote here something more about RAW and radiance, because I think, that you also interests in...

[Ben321]

What I'm asking for, regarding that thermal raw TIF saving, is for somebody to post an APK file of an app with that capability here on these forums. I do not have Android Studio or the JDK or the FLIR SDK. I have absolutely no way to compile any source samples people provide on these forums. So referencing some source code does me no good. However if somebody could compile it and post the APK, then I could use the app as-is, and from that I could determine if it would actually be worth buying a FLIR One.

[cynfab]

tomas123,
Thanks, I am really interested in Raw and transforming  to real temp profiles, but I don't have the bandwidth, or need at the moment.
All I need right now is what we have in the flir8x code. Accurate temperatures are of interest, but my application needs AGC'd images like we have.
Others are welcome to start with that code and produce whatever...
I've found the SDK fairly easy to use and have built a few apps with it. (my code is too ugly ATM to release, but doesn't break any new ground).

   ...ken...
btw your posts are a great resource for anyone seriously interested in developing for the FlirOne G2.