Vipitis has raised a very good question that I did not cover in my review of the P2 Pro. This was an oversight as I personally do not use thermal cameras to record video.
A little bit of information regarding video recordings from thermal cameras …….
With an all-in-one self contained thermal camera that offers video recording capability, the manufacturer will select video processing elements of the design to match the intended performance specifications. The firmware will be ‘tuned’ to provide the best possible frame rate during video recordings but the resolution of such video recordings can vary greatly. Clearly recording a high resolution video will place greater demands on the systems processor as it not only has to deal with the thermal imaging side of the system, but it will also have to upscale and capture at decent frame rates. More pixels = more work and potentially slower frame rates if the processor cannot cope. A manufacturer will consider resolution, frame rate and cost in order to provide the user with an acceptable specification that the camera will meet.
If we consider a thermal camera dongle then the scenario changes in terms of the processing hardware selection. Where a Mobile Phone is used to do the video processing of the thermal image data that is coming into its USB port, we need to consider potential bottlenecks, upscaling requirements, video capture process, processor limitations and the resultant impact on the achievable frame rate. As I was once told….. “ life is relatively simple when you design a thermal camera from scratch and have control over the BoM. However marrying a thermal imaging camera front-end to a Mobile Phone back-end is a total nightmare, as you have no control over the specification and configuration of the Mobile Phone hardware”. This is why some designers actually prefer the Apple iPhones for such tasks as the specification is well documented and current sales line limited in number.
So back to marrying a thermal camera dongle front-end to a mobile phone… There are so many things that can go wrong or be sub-optimal with such a ‘marriage’. The first challenge is hardware level compatibility where the appropriate USB or Apple interface is selected. The data communications bearer must be capable of passing data at a rate adequate for the intended video capture frame rate. USB is a pig when it comes to Video data streaming as it is packet based rather than the very efficient stream that IEE1394 (FireWire) used. That said, USB 2.0 and above usually copes OK for most common video data rates. Once the thermal camera data is inside the phone, it is processed by the APK and Software APP to create a thermal image. This is a load on the processor but we must also consider that the same processor is likely doing other “housekeeping” tasks that may also demand attention in conflict with the thermal imaging process that is ongoing. The processor needs to be powerful enough and configured to prioritise the thermal imaging process if dropped frames are to be avoided. Now add to the processors workload the requirement to not only upscale the thermal camera images at full frame rate, but also encode them into MP4 format for saving as a video capture file. All this work is time critical if dropped frames are to be avoided or the frame rate not impacted negatively. This is quite the “ask” for a mobile phone that was not actually intended for such duties. Sure modern mobile phones can capture high resolution visible light camera data but that side of the phone was part of the original designers realm where parts were carefully selected and optimised for the task. We, on the other hand, are bolting on a USB thermal camera that has different needs not considered during the phones design.
So where does this scenario leave us ? Well it is clear that if the mobile phone host to which we connect a thermal dongle camera does not have the required multi-tasking “horsepower” in its processor, the recording of smooth high frame rate video will not be possible. What will happen is that the phone will do its best to meet the needs of the thermal camera application but the frame rate will be impacted by its inability to complete the capture process in the allotted time frame. Dropped frames will be a fact of life. If the thermal camera dongle is attached to a very powerful modern mobile phone that has more processing power than any phone user could wish for, the situation changes. We now have the required processing capability and “headroom” to meet the needs of the thermal camera dongle data processing, upscaling and video capture to MP4 format. Great ! But there remains a risk to the processes smooth running for video capture. We still need the phone processor to prioritise the needs of the thermal camera video capture task over its other housekeeping tasks and there must be no data bottlenecks outside the processor. The USB port specification needs to be adequate for the frame rate that will be transferred from the camera dongle to the phone processor. Thankfully USB is now pretty reliable in this respect but it still has its limitations at USB 2.0. USB 3.1 is far more useful for video capture tasks but both the source and host need to be USB3.1 compliant. Just because a USB 3.1 compliant connector is present, it does not necessarily mean that the electronics connected to that port are compliant and it could be USB 2.0 ! It is not uncommon to find this issue with budget equipment claiming USB 3.0 or 3.1 compatibility….. they are USB 2.0 and USB3.0/3.1 is backward compatible with USB2.0. The problem is that the data rate capabilities are very different.
If we are very fortunate and the “Planets are in alignment” we can connect a thermal camera dongle to a decent performance mobile phone and the full frame rate capabilities of the thermal camera dongles will be presented on the phones display and will be recordable to a MP4 file at full frame rate in a resolution appropriate for viewing on a MP4 playback device of the users choice. For this to happen both the hardware and software must work in harmony with no inefficient coding or poor optimisation of the processors capabilities. Sadly it has often been found that the APP that is running on the mobile phone has to be very tolerant of different host mobile phone brands, operating system versions and hardware capabilities. This results in inefficiencies that can lead to lower performance than might have been possible if the app was written for a specific mobile phone model and tuned for peak performance whilst on thermal imaging duties.
Some trivia….. FLIR created the FLIR One Generation 1 mobile phone jacket thermal camera for the Apple iPhone only. They did this as they could then write efficient code for the particular series of iPhone in order to get the best from its processor to deal with the needs of the thermal camera module. FLIR did not want to create an Android version of the FLIR One Gen 1 as there were too many variables that could cause issues with performance. Not least of which was the many different case sizes and formats of Android phones. For the FLIR One Gen 2 camera project, FLIR decided to make both iOS and Android compatible versions. This was a major decision for them as the APP coders advised of all that could go wrong with trying to meet the needs of the thermal camera when FLIR had no visibility or control over the brand, model, performance, operating system version and configuration of the host mobile phone. A decision was made to create an APP that complied with Android programming best practices in order to gain best multi platform compatibility. FLIR could do little more than this in the circumstances BUT they made the decision to design their Android FLIR One G2 solution based upon a well known and powerful mobile phone of equivalent performance to the iPhone that had already been proven adequate for the task.
The mobile phone FLIR chose for their Android solution was the venerable Samsung S5. This was an expensive flagship phone at the time but FLIR wanted their FLIR One G2 to perform well in the market and the S5 had the processor to ensure decent frame rates within the <9fps limit imposed by Government regulations. The App for the FLIR One G2 could not be optimised for the S5 to the point that only an S5 could be used so compromises in the fine tuning of code were necessary. FLIR struggled to achieve the same performance of their FLIR One G2 iOS solution, even when using the S5 ! Further development did close the gap between the iOS and Android versions performance but the iOS was more highly tuned and always had the edge over the Android App. This is the cost of programming for a broad spectrum of platforms that may have very different configurations despite all running Android.
So to summarise all of the above…… if you are a videographer wishing to use thermal imaging in a video format, beware of the pitfalls associated with mobile phone dongles designed to be compatible with a broad spectrum of Android based Host mobile phones. Frame rate can be VERY variable even when powerful host phones are used.
Worthy of note is that even some Industrial grade thermal imaging cameras that offer video recording cannot record video at the same rate that they can display it in the live scene. It all comes down to processing power and a decision by the manufacturer on how much processing power they will build into the camera. Thermal videography is not normally a high priority in general thermography work. High speed thermography is a totally different ball game. Just look at my FLIR SC4000 to see that !
It should also be understood that Radiometric video is not normally found on budget thermal imaging solutions so post capture analysis of individual frames is not possible. Science cameras are often capable of high frame rates whilst also offering high frame rate capture of Radiometric frames that may be treated as still frames for image analysis. That said, the “video recorder” for such cameras is often a dedicated external device with a very high data rate interface to the thermal camera. Basically, if you have to ask the price, you cannot afford it.
Fraser
Home
Help
Search
Login
Register
