FOne Ultimate

[Ben321]

This is an app for FLIR One that I've created. It is intended to be pronounced "Eff One Ultimate". FOne is short for FLIR One, and is intended to avoid infringing on FLIR's trademark product name FLIR One, as I don't think FLIR has trademarked the name FOne. It is not yet available at the Google Play store as I haven't yet sent it to FLIR for approval (if you look at the FLIR One SDK developer website, and read about the app publishing process, it says you must submit it to FLIR prior to posting it on Google Play). It is now 95% complete, and is full of tons of features that FLIR's official app doesn't have, while only missing 3 features that the official FLIR One app does have. Note that it is only compatible with the Gen2 FLIR One, and not the newer Gen3 FLIR One and FLIR One Pro. This is because image dimensions (for both the infrared and visible light images) are hardcoded into the app. If you don't already have a Gen2 FLIR One, you should buy one now, before FLIR stops selling them, if you want your FLIR One to work with my app.

Missing features in my app are:
spot meter
alignment of visible and IR images for MSX
saving FLIR proprietary radiometric JPG images

Added features in my app:
access to all image types
ability to save raw images as 16bit grayscale tif files
ability to save raw video in a format I designed myself to support 16bit grayscale (no standard video format like AVI supports this)
ability to control the app via a desktop/laptop PC over WLAN, using UDP
ability to stream Raw IR or Thermal IR video to a desktop/laptop PC over WLAN, using TCP


The rest of this post is a user-manual on its features (note that position terms like "above", "left", or "bottom" are assuming that your device is held in portrait orientation):

When the FLIR One is turned on and plugged in, the text Connected will appear under the app's title on the title bar. When the FLIR One is unplugged or turned  off or disconnected the text will change to say Disconnected.

Below the title Just below the image window, on the left side of the screen there's a battery percentage window.

At the bottom of the screen are 6 buttons. These are Settings, Settings 2, Start Remote, Save Image, Save Video, and Start Stream.
The menu that is displayed for the Settings button contains the buttons for setting Image Type and Palette (which is used for IR Image and MSX Image image types), as well as a check box to enable/disable automatic calibration, and a button to manually perform a single calibration.

The available Image Types are Raw IR, Thermal IR, IR Image, MSX Image, Visible Image, and Raw Visible.
The Raw IR image is 120x160, and is the least processed image format (though technically not completely raw, as hot-pixel removal and other such operations are performed internally either by the FLIR One's firmware or internally in the parts of the SDK that are pre-compiled binaries and therefore can't be altered by the app developer using the SDK in their app).
The Thermal IR image is 240x320. This image is generated internally by FLIR's SDK (not my app) by cropping the image to raw image to 90x120 and then upscaling to 240x320. Also the raw values are used in a formula (along with emissivity and Planck constants) to compute actual temperatures, so that the pixels represent temperature (measured in centikelvin) rather than actual raw microbolometer sensor readings.
The IR Image image is 480x640. This is an upscaled version of the Thermal IR image and has been histogram equalized (or at least brightness stretched), as well as having had a color palette applied to it (again via either the firmware in the FLIR One or in a pre-compiled binary in the SDK).
The MSX Image image is 480x640. This is a combination of IR Image and Visible image, where an upscaled copy of the Visible Image is high-pass-filtered and used to add detail to the overall image. Again, this is some kind of internal process, and is not an algorithm that I implemented.
The Visible Image image is 240x320. This is one of the 2 image types that uses the visible light image from the FLIR One's visible light camera. This is a cropped and rotated copy of the Raw Visible image. Again cropping occurs internally in either the firmware or in a pre-compiled binary in the SDK. While uncompressed (unlike the Raw Visible image), that is only because this image is generated after decompressing the Raw Visible image.
The Raw Visible image is a JPEG compressed copy of the 640x480 image from the FLIR One's visible light camera. As far as I know, there is no way to access the uncompressed copy of the visible light image. This is the rawest form of the visible light image that is possible to get with the FLIR One SDK.


The Settings 2 button opens up a menu which has (at this time), the following features. IP Address and Port Number for remote UDP control of the app. The IP address is a filter, to make it so the app only accepts UDP packets from a computer with the specified IP address (using 0.0.0.0, which is the default, means it accepts UDP packets from all computers on the LAN). The Port Number is the local port that it binds to to listen for UDP packets. It also the IP Address and Port Number for TCP streaming. This is the remote address and port number that FOne Ultimate will try to connect to to stream. The software on the remote computer must be acting as a server (listening for incoming connections), as the FOne Ultimate app acts as a TCP client (attempting to connect to the remote computer).


The Save Image button saves a still picture, who's format depends on the Image Type. For Raw IR and Thermal IR image types, the format is 16bit grayscale uncompressed TIFF (.tif file extension). For IR Image, MSX Image, and Visible Image image types, the format is 32bits (8-bits-per-channel 4-channels B,G,R,unused) color uncompressed Bitmap image (.bmp file extension). For Raw Visible image type, the format is 24bits color compressed JPEG (.jpg file extension). For all image types, except for Raw Visible (because it's compressed, so I don't have access to the individual pixel data), the saved image is rotated by my app, prior to saving, based on the orientation of the phone.

The Save Video button starts recording a video in BVF format (a video file format I created for this very purpose, with file extension .bvf). While recording, the button changes to say Stop Video, which stops the recording. Depending on the speed of your device, the rotation operation may not be able to be performed in real-time, and as such no orientation-based rotation is performed for video being recorded (only for saved still pictures). To rotate it to the correct, orientation, you will need to copy the .bvf file from your phone to your PC, and use PC software I created to rotate it to the correct orientation. Also because this is a video file format I created myself, no current standard video player (WMP, VLC, etc) will play it, so you will instead need to use the BVF video player I wrote specifically for this purpose (note that this BVF player also allows you to export an MP4 video file to play with standard video players, and also note that this feature requires that a copy of ffmpeg.exe is present in the same folder as the BVF video player's exe file, in order to use that function). You can download a copy of the latest (as of me posting this) version of FFMPEG at http://ffmpeg.zeranoe.com/builds/win32/static/ffmpeg-20170702-c885356-win32-static.zip


The Start Remote button makes it start accepting commands sent coming in as UDP packets from a "remote" computer. Each command is a 5 letter ASCII (1-byte-per-letter) string. The available commands are as follows (note that they are case sensitive, and if the command has 4 letters/numbers, there is a space after it to make it exactly 5 characters long).
"IMGT1" = set Image Type to Raw IR
"IMGT2" = set Image Type to Thermal IR
"IMGT3" = set Image Type to IR Image
"IMGT4" = set Image Type to MSX Image
"IMGT5" = set Image Type to Visible Image
"IMGT6" = set Image Type to Raw Visible
"PAL1 " =  set Palette to Grayscale
"PAL2 " =  set Palette to Ironbow
"PAL3 " =  set Palette to Rainbow
"PAL4 " =  set Palette to Rainbow HC
"PAL5 " =  set Palette to Arctic
"PAL6 " =  set Palette to Lava
"PAL7 " =  set Palette to Wheel
"PAL8 " =  set Palette to Coldest
"PAL9 " =  set Palette to Hottest
"SAVEI" = save image
"SAVEV" = start recording video
"STOPV" = stop recording video

When in UDP remote controlled mode, the Start Remote button changes to Stop Remote, and pressing it takes it takes the app back out off UDP remote controlled mode.


When you press the Start Stream button, it attempts to establish a TCP connection to the remote computer. If it successful, it starts streaming the video in the following format
First 4 bytes per frame are a little endian 4-byte-integer that contains the Image Type's number (0 to 5), followed by the pixel bytes for that frame. It's a very simple format. The remote receiver of the stream  only needs to know the image dimensions and bit depth and channel count of each of the formats (which is already specified here in this manual), and set up a byte array accordingly to read the TCP stream of pixel bytes into. When streaming, the Start Stream button changes into the Stop Stream button. Pressing Stop Stream stops the app from sending frames over the TCP connection, and then disconnects the TCP connection.





Here I have linked (too big for a forum attachment) to a ZIP file I uploaded to Mediafire with the following contents. https://www.mediafire.com/?ptt0gthg8mn5pga
the debug APK file for the FOne Ultimate app (note it's not 100% complete yet, and there may still be some bugs)
the BVF video player's EXE file (this is essential for viewing the videos saved by FOne Ultimate)
the BVF video rotator's EXE file (this is essential for rotating the video frames in BVF video files, whenever they were recorded with the phone in any orientation other than portrait)

Note that the slider in BVF Video Rotator doesn't select the frame to rotate (when rotation occurs, all the frames get rotated), it simply changes the currently selected frame being viewed so you can examine them for visual clues for what direction to rotate the video in (if you don't remember what orientation you had the phone in at the time you recorded the particular video). When you press one of the 4 buttons to rotate the video, it rotates all the frames in the video, by the amount indicated by the button that you clicked.

[joe-c]

Hi Ben32,
first of all, thanks for this App.
i was able to use it and my second phone and the Flir One to get a "WIFI Thermal Camera".
really nice. 

I will provide a c# test version (actual only support the default raw IR Mode).
maybe it's possible to get the app running on a tiny android stick (previously for TV) with auto start.
So just start the Fone with the Stick and wait some seconds and the live image appears on PC.

Some Issues:
on my "Mobistel Cynus F6" i have to set the system Font to small, to get the button usable to leave the Settings 1 menu.
on my "Samsung Note 2" it works.
sometimes a crash if switching the image mode (both phones).
sadly i failed to get the "Start Remote", i just see a error message below the title with "java.net.BindException: bind failed: EA..."
I don't understand why the stream use the TCP connection but the "Remote" use the UDP... can you tell me please?

some wishes:
1. add some kind of settings file or let the APP store the IP Settings. I have to change the IP each time I start the app.
2. it would be great if the Server can "ask" for a frame, instead of streaming the data the whole time.
that could let the application process a full frame and ask for the next. now the PC has to drop frames if they come to fast.
additional it will have the nice ability to ask for different frames. for example ask for raw ir and raw vis jpg to show both same time.

Joe-c

[Ben321]

Hi Ben32,
first of all, thanks for this App.
i was able to use it and my second phone and the Flir One to get a "WIFI Thermal Camera".
really nice. 

I will provide a c# test version (actual only support the default raw IR Mode).
maybe it's possible to get the app running on a tiny android stick (previously for TV) with auto start.
So just start the Fone with the Stick and wait some seconds and the live image appears on PC.

Some Issues:
on my "Mobistel Cynus F6" i have to set the system Font to small, to get the button usable to leave the Settings 1 menu.
on my "Samsung Note 2" it works.
sometimes a crash if switching the image mode (both phones).
sadly i failed to get the "Start Remote", i just see a error message below the title with "java.net.BindException: bind failed: EA..."
I don't understand why the stream use the TCP connection but the "Remote" use the UDP... can you tell me please?

some wishes:
1. add some kind of settings file or let the APP store the IP Settings. I have to change the IP each time I start the app.
2. it would be great if the Server can "ask" for a frame, instead of streaming the data the whole time.
that could let the application process a full frame and ask for the next. now the PC has to drop frames if they come to fast.
additional it will have the nice ability to ask for different frames. for example ask for raw ir and raw vis jpg to show both same time.

Joe-c

I use TCP for streaming because it is better for transferring a large amount of data. TCP guaranties every last byte is transferred. With UDP instead of sending as a stream, it sends packets of data (known as "datagrams", which is what the D in UDP means). When a UDP packet is larger than a few hundred bytes, it can sometimes get dropped in transfer (especially if passing through a lot of routers, like when going between countries on the internet). A command packet of only 5 bytes in size (like the way my UDP remote control works for this app) is almost guarentied to work 100% of the time, and UDP is simpler to use because you don't have to maintain a connected state as with TCP (so UDP is simpler to program than TCP).

Don't try to send any image type over the TCP connection other than Raw IR or Thermal IR. Any of the image types that are 640x480 will crash the app. I'm still not 100% sure why, even though I've spent hours looking at the code. Since Raw IR and Thermal IR are the formats people are probably most interested in (and can't be accessed in the official FLIR One app), I'll probably give up trying to figure out what is causing this bug, and let the bug remain. This is why I have it programmed to disable the Start Stream button when these image types are selected. Trying to change the image type to one of these unstreamable image types in the Settings menu (or via the UDP command from another computer), while streaming is already occurring, will crash the FOne Ultimate app.

[CustomEngineerer]

It is intended to be pronounced "Eff One Ultimate". FOne is short for FLIR One, and is intended to avoid infringing on FLIR's trademark product name FLIR One, as I don't think FLIR has trademarked the name FOne.

 

[askfor2000]

Here I have linked (too big for a forum attachment) to a ZIP file I uploaded to Mediafire with the following contents. https://www.mediafire.com/?ptt0gthg8mn5pga
the debug APK file for the FOne Ultimate app (note it's not 100% complete yet, and there may still be some bugs)
the BVF video player's EXE file (this is essential for viewing the videos saved by FOne Ultimate)
the BVF video rotator's EXE file (this is essential for rotating the video frames in BVF video files, whenever they were recorded with the phone in any orientation other than portrait)

Why I can not download the APK ?

[Ben321]

Here I have linked (too big for a forum attachment) to a ZIP file I uploaded to Mediafire with the following contents. https://www.mediafire.com/?ptt0gthg8mn5pga
the debug APK file for the FOne Ultimate app (note it's not 100% complete yet, and there may still be some bugs)
the BVF video player's EXE file (this is essential for viewing the videos saved by FOne Ultimate)
the BVF video rotator's EXE file (this is essential for rotating the video frames in BVF video files, whenever they were recorded with the phone in any orientation other than portrait)

Why I can not download the APK ?

The APK file is in the ZIP file that I linked to. Download the ZIP file, and extract all the files (or just the APK file). You will see that the APK file is one of the files.

[eternalone]

I looked it over. It's ok. It won't change the pallette, wont save images, can't lock the exposure.

On galaxy s5.

[Ben321]

I looked it over. It's ok. It won't change the pallette, wont save images, can't lock the exposure.

On galaxy s5.

Palettes have no effect except in IR Image and MSX Image modes. In Raw IR mode, it is literally raw unprocessed images. In Thermal IR mode each pixel's value is the temperature at that pixel (as measured in centikelvin units). Palettes are never applied to visible light images because they already are full color RGB images.  You will need to switch your mode to IR Image or MSX Image, if you want to use the palettes.

I'm not sure why it's not taking pictures for you. Can you please tell me the steps you took to create this bug? I'll attempt to see if I can duplicate on my phone. Also, please tell me what version of Android you are using. I have only one Android device (so I can't really test my apps on many different devices), and it is a Motorola Droid Maxx, with Android 4.4.4.

No, it can't lock exposure. That feature isn't (at the time I started working on the app) available in the publicly distributed FLIR One SDK (though it was present in the official FLIR One app, meaning it was already present in FLIR's own internal SDK used by the software designers in their company). It may have since been added to the publicly available SDK, but I don't know. And I'm not about to wreck my app by upgrading my FLIR One SDK, just to try to get one more feature. Sometimes newer versions of things introduce glitches, and I don't want a possible glitch in a newer version of the FLIR One SDK to end up breaking my app. So it is likely that exposure lock will NEVER be available in my app. If you need it for some reason, use the official FLIR One app instead.

[eternalone]

Android 6.0.1

It's no big deal though.

I use thermal camera+

It has full resolution that the flir one is capable of, pallette changes, exposure lock, and non msx mode.

I wouldn't use the flir one app even if they paid me.

[Ben321]

Android 6.0.1

It's no big deal though.

I use thermal camera+

It has full resolution that the flir one is capable of, pallette changes, exposure lock, and non msx mode.

I wouldn't use the flir one app even if they paid me.

There's an app called "Thermal Camera +"? Can you send me a link? I'd like to try that out.

[joe-c]

There's an app called "Thermal Camera +"? Can you send me a link? I'd like to try that out.

Here:
https://play.google.com/store/apps/details?id=georg.com.thermal_camera_plus&hl=en

To the "exposure"... its more a level and Span setting. The app scale each frame for display.
if you disable that auto scale to fix level and span you can use the last values for scaling the following frames.
or does somebody mean other to "exposure"?

Additional... could you please let the app auto store the last IP settings... that would be great. 

[Ben321]

There's an app called "Thermal Camera +"? Can you send me a link? I'd like to try that out.

Here:
https://play.google.com/store/apps/details?id=georg.com.thermal_camera_plus&hl=en

To the "exposure"... its more a level and Span setting. The app scale each frame for display.
if you disable that auto scale to fix level and span you can use the last values for scaling the following frames.
or does somebody mean other to "exposure"?

Additional... could you please let the app auto store the last IP settings... that would be great. 

Not a free app. Anybody know of a free version?

[frenky]

Here: https://play.google.com/store/apps/details?id=georg.com.flironetest_01&hl=en

But 3,29€ for supporting the developers efforts is really not that much...

[Vipitis]

I have a big question.

Does it support the CAT S60 phone as camera?

It works with only a few apps...

But it's the gen2 sensor.

[frenky]

It does:

PLEASE NOTE:
###
1. This app isn't compatible with the new FLIR ONE v3 and FLIR ONE Pro.
2. This application requires the FLIR One USB thermal camera or S60 device (hardware) to be connected to your smartphone.
###

https://play.google.com/store/apps/details?id=georg.com.flironetest_01&hl=en

[Vipitis]

I was asking for the OP app.

I already tired the Thermal Camera + app by George, he brought me to this forum.

I just downloaded the Ultimate app because I am highly interested in the additional functions for various applications.

I installed the .APK and then rebooted my phone. I get a grey image- not even anything visible - no message.

I really hope that CAT S60 can be supported in the final product.

[Ben321]

I was asking for the OP app.

I already tired the Thermal Camera + app by George, he brought me to this forum.

I just downloaded the Ultimate app because I am highly interested in the additional functions for various applications.

I installed the .APK and then rebooted my phone. I get a grey image- not even anything visible - no message.

I really hope that CAT S60 can be supported in the final product.

Cat S60 uses a Lepton sensor, just like the FLIR One does. However, its  thermal camera module does not report itself to the phone as a USB device with the same VID and PID as the FLIR One. The apps written for the FLIR One look for a USB device with a specific VID and PID. If the attached device has a different VID or PID, the app will not be able to work with that device, even if otherwise that device's circuits are very similar to the FLIR One. You will need a separate Cat S60 SDK to write apps for the thermal imager in the Cat S60 phone. I don't even know if FLIR has released an SDK yet for the Cat S60.

[Vipitis]

There is definitely a SDK avaible for the CAT S60.

It should be easy practise to add CAT S60 support to your Flir One project as Bullit said in a press release on CATs website.

I am sure it's possible because George has done it with the Thermal Camera app.

[Ben321]

There is definitely a SDK avaible for the CAT S60.

It should be easy practise to add CAT S60 support to your Flir One project as Bullit said in a press release on CATs website.

I am sure it's possible because George has done it with the Thermal Camera app.

There are specific functions and classes designed just for FLIR One, in the FLIR One SDK. If I created another app for the S60, then I'd need to use its SDK instead of the FLIR One SDK, so while I could create a separate app for the S60, I wouldn't be able to create a single app which works with both cameras.

[Vipitis]

That's sad to hear.

I am sure it's somehow possible, as George managed to do it with the Thermal Camera app.

I was really interesting in your app but it sounds like it won't be an option for me for a while.

[MURONI]

Hello (sorry for my english, I'm ... French :-( ),
I followed your explanation for the image of the thermal camera Flir One on my PC.
I can not get the connection. . . Yet the address entered in Settings 2 is the same as on my PC with Set, and pressing Start Stream.
(image attached)
Can you help me ?
Thank you very much
http://drone-photographies.fr/Capture.JPG

[joe-c]

Hello,
you use a system font size of 125% right?

Sadly the the detection of the ip is not perfect. the problem is, the server application is just a program which hold a function at a specific port. if you want connect from outside, your Network card has a IP from somewhere, and this IP is to connect to it. If you have more than one Network card its possible, that the application shows the false IP.

So better look inside your router to get the matching IP.
A good setting is a firewall with "own rule" so it ask for each new connection and you handle it yourself.
in this case you start the server and nothing happens, but if you try to connect the first time the firewall blocks and ask you if you what to allow this connection.

Additional it looks like you use the "old" version 1.6.0.0... for using the flir one with fone ultimate better use the last 1.7.0.0, there the flir one server is directly visible and has some tiny improvements.
best wishes

[MURONI]

Hello,
thank you for your reply.
I allowed the router to accept all incoming connections FONE ULTIMATE. No results.
Attached to the IP address obtained with cmd ipconfig ( http://drone-photographies.fr/Capture.JPG
Best regards

[MURONI]

Hello,
I was able to connect disabling Avast. 
Now all is correct 

[Vipitis]

Hello,
sorry for reviving this.

I am in the search for a better capturing(for post processing) app for the CAT S60. Previously you said that it wasn't easily possible. Can you tell me which version of the FLIR SDK was used for the development, as you noted that it won't work with the newer devices.

Thanks!