First Therm-App Pro arrives (640 x 480 x 25Hz uncooled LWIR)

[Ultrapurple]

My Therm-App Pro has arrived. It boasts 640 x 480 resolution, up from the 384 x 288 of the regular Therm-App, and 25Hz frame rate. To the best of my knowledge, it is the first one 'in the wild' and the serial number of mine is 0001  .

Visually, it is identical to the regular Therm-App. Mine has a 30mm f/1.1 lens, which is physically rather bigger than the standard 19mm (or my favourite for the regular model, the 13mm f/1.0). I expect the FOV of the 30mm lens to be similar to an 18mm lens on the 384 x 288.

(Correction added 6 October - the back of the Therm-App Pro is slightly thicker than the regular camera; mine looks like it was CNC machined from solid. You can just see some tooling marks on the web between the handle and main body).

So why no thermal pictures yet? Because I can't presently get it to work on my phone, that's why. When I plug it in it isn't recognised at all, let alone by any of the apps. As it's Friday today and the manufacturer will not be open, I'll have to wait a day or two before I can get any technical support.

I will report back as and when I manage to get it working. Meanwhile, here are a couple of rather blurry snaps taken quickly immediately after it arrived.

[Fraser]

Nice !

That looks a decent size lens as well 

Sorry you are having connectivity issues. Hopefully it is something simple.

Fraser

[Ultrapurple]

Thanks - so far I've tried everything I can think of in terms of different ways of running the app, rebooting, rebooting with the camera connected (not something one usually does), closing all the background apps, and everything else besides. I have some experience with the regular Therm-App and the various driver apps, and I can't think of anything I might be doing wrong.

My next step, when I get home, will be to try connecting the camera to a different device and see whether that can run my new toy. I have a tablet and at least one older phone that work fine with the regular Therm-App but haven't had other software such as ThermViewer added (just in case that's the problem on this phone, however when I tried starting ThermViewer and plugging in the camera it didn't want to know).

Fingers crossed, eh?

[Ultrapurple]

Initial at-home experiments suggest it's a power supply problem on the phone - the Therm-App Pro powers up on my Nexus 10 tablet and fires up the old copy of the ThermApp app that resides there, but unfortunately the tablet isn't compatible with the current version of ThermApp (to be fair, the Nexus 10 is showing its age).

Another experiment with a different setup also powers up the camera and starts the app, which is gratifying. My next step will be to try and dig out my last few phones, install the latest version of ThermApp, and see if any of them can drive it. I do remember vaguely hearing someone at Opgal say there were some power consumption issues with the early engineering prototypes, so it may just be that my cheap Chinese phones simply can't provide enough current to start the camera.

Experiments continue, but I'm very short of time this weekend. It's frustrating - had the camera been delivered on time I could probably have had it running by now!

[Ultrapurple]

Aha! First Light! (well, almost). I found an old Nexus 5, installed the up-to-date ThermApp software, then plugged in the camera and held my breath. After the usual white into screen that detected a serial number from the camera, and then agreeing to the T's and C's it downloaded the calibration files from Opgal's server - a very good sign. (This download, and th Ts & Cs, only happens once).

A screen appeared - but with garbage rather than a thermal image. The image did change very slightly as it was pointed at different things but definitely not in the way you'd expect for a thermal camera, even way out of focus. The update rate was slow and jerky, sometimes hanging for seconds at a time. The temperature tag, however, did vaguely relate to what I was pointing the camera at: a cold drink read lower than general ambient.

It's interesting to note that the header on the screen says Therm-App Pro, rather than just Therm-App.

It looks as though there is either a communications issue or a shortage of CPU power. The Nexus 5 is also rather long in the tooth, like the Nexus 10 mentioned earlier, so there could be many reasons it isn't working. But I'm several steps further on than I was, which is good. Now let's see what other phones I can find!

[Ultrapurple]

Latest tests have confirmed there is indeed a power issue on several of my phones so I've updated my old Nexus 5 to Android 6.0.1 and been given a beta update of the driver. I am starting to get recognizably 'thermal' images, but they look like the horizontal and vertical hold  are set wrong and the picture dances round the screen. But at least they are talking to each other, after a fashion, so things are headed in the right direction.

I will keep you posted.

[Ultrapurple]

Right - progress.

I updated my Nexus 5 to Android 6. It recognised the Therm-App Pro and tried to get pictures from it, but failed on horsepower. Opgal was able to reproduce the issue on one of their test phones and told me it was simply down to the Nexus 5 being too long in the tooth and not having enough processing power - a perfectly reasonable explanation. The Therm-App Pro has roughly three times the number of pixels of the regular camera so it's not unreasonable to think it will take at least three times the processing. And, after all, the Nexus 5 is something of a dinosaur these days.

They have now released the following minimum spec to run the Therm-App Pro:

System chip: Qualcomm Snapdragon 820 MSM8996
Processor   : Quad-core, 2200 MHz, Kryo, 64-bit
Graphics processor: Adreno 530
System memory: 4 GB RAM

They suggested something like a Samsung Galaxy S7, so I'm looking for one of those at a reasonable price on flEabuy. I'll continue to report progress.

[Fraser]

Ultrapurple,

Sorry to hear this news.
Sadly the phone dongle approach does appear to demand a lot of the host phone. Unlike a complete thermal camera solution that is 'tuned' and IC' selected to meet the processing requirements, a generic mobile phone is not always up to the task. As you state, a 640 x 480 thermal camera may have a lot of processing overhead that exceeds what many common and affordable phones can provide. I have been after a used Samsung S5 for a while..... they seem to hold their price well !

I was a little concerned to hear about the power requirements of the dongle. I wonder how much current ir draws at power on ? If the TEC or heater were in use it would draw increased current for a few seconds and then drop back to a nominal operating current. If the Dongles current requirements are significant it will drain a mobile phone battery quickly. It may also strain the expensive phones OTG supply circuitry which is not a great idea. Most phones will start throttling back on processor speed if the internal temperature starts to rise too much.

We may be witnessing the practical limitations of phone Dongles here. The 640 x 480 camera dongle is pushing the envelope on both phone processing power and power availability. Internal power, as on the FLIR One, may be an option in future designs but such will increase dongle size. External power is always an option via an OTG 'Y' Cable but makes the solution messy.

The processing limits of common affordable phones may be the true limiting factor. People tend to purchase a thermal camera dongle as the most affordable solution. If they then have to buy the very latest, most powerful mobile phone, the total cost can escalate significantly. Imagine needing the likes of the Apple iPhone X to run your dongle 

This thread is very interesting reading and I thank Ultrapurple for documenting his experience 'warts and all'

Fraser

[Ultrapurple]

Thanks Fraser

I admit I was disappointed when I found the phones I had readily to hand weren't able to run the Therm-App Pro for one reason or another, but then the Nexus 5 was launched in 2013 so is a veritable dinosaur; the other phones I checked are cheap Chinese devices that generally deliver a good user experience - I suspect the power requirement of the Therm-App Pro is right at the top end of what the Micro USB spec can deliver.

The performance of the Nexus 5 suggests that it's not that far down from what's actually needed for sensible processing of the images but of course there's no way to up the processor power. So out with the dinosaur and in with a new device.

I have been considering putting an external power supply onto the Therm-App Pro via some kind of Y adapter, if only to see if my cheap Chinese phones are capable of running the software. I have tried using an external, powered USB hub but that didn't work. I vaguely remember managing to run a regular Therm-App via a hub but I wouldn't be surprised if such an arrangement confused the software.

The Therm-App series of cameras do not use a TEC for heating or cooling, so the power consumption is probably largely down to the sheer amount of processing being done in the camera hardware. Even the regular camera has a pretty sizeable gate array inside; I haven't opened up the Pro to see what it contains but I can't imagine the gate count is any lower! The sensor an ROIC will also consume (at least) proportionally more than their lower pixel count brethren.

I don't necessarily agree that we're at the limit of what can be done with a dongle-type device. Processing power per watt continues to increase; the four years since the then-flagship Nexus 5 came out has seen a massive improvement in computing speed in phones and even as Moore's Law starts to hit the one-atom-per-transistor level there will probably still be people working out how to attach metallisation to individual electrons in their orbits...

Experiments on the Therm-App Pro continue. I'm pleased that Opgal are helping me and I'm well aware that I'm at the bleeding edge of product testing on this device. As I've mentioned before, the serial number label on mine is 0001; the internal serial number is  xxx003, suggesting that there are perhaps just two engineering prototypes before it. It's early days and frankly I'm quite excited to be part of it.

[Fraser]

Ultrapurple,

I totally agree that mobile phones will keep getting more powerful, BUT, what I was trying to say was that the Dongles will dictate a very high end phone in order to work correctly. Such phones tend to be expensive and may reduce the cost advantage that they currently hold over complete thermal camera systems. Thermal cameras are advancing as well and becoming cheaper.

You are most definitely at the cutting edge of the 640 x 480 pixel thermal dongle. I am pleased that Opgal are being supportive and I feel sure they appreciate a technical person, such as yourself providing them with technical comment and feedback  . Being a 'pathfinder' can be fun, but it is made easier if the product is supplied as a Beta version and at a discount 

Fraser

[TheSteve]

Sounds like the camera will have a very limited market. It's requirements are far from ideal. Hopefully they sort out the issues and you get it working.

[nidlaX]

It sounds like they skimped on the design of the dongle to rely so much on the phone for processing. That being said, is it unreasonable to expect a Nexus 5 to process 640 x 480 framerate video?  I think Opgal can do a little better here.

[Ultrapurple]

Sounds like the camera will have a very limited market.

Yes, I'm sure you're right: it is an expensive device for most people and has no pretensions (yet) to accurate thermographic readings, but it's right up my street because most of the time all I'm interested in is making pretty pictures.

I am well aware that other people have professional uses for better-calibrated (and more expensive) thermal imaging devices.

It sounds like they skimped on the design of the dongle to rely so much on the phone for processing.

I respectfully disagree. The phones I've been trying are old, cheap, low-to-mid-range types.

{..} is it unreasonable to expect a Nexus 5 to process 640 x 480 framerate video? 

Yes it probably is unreasonable to expect a Nexus 5 to cope and I have more or less said so in my earlier posts. It is important to remember that I am experimenting here with the very first production sample and choosing to make the results public.

I think Opgal can do a little better here.

On the contrary, I think Opgal are doing rather well.

Other than a few end-of-life ex-fire service examples, good quality 320 x 240 resolution 25 / 30Hz frame rate thermal cameras are hard to get if you live outside North America, and 640 x 480 cameras at that frame rate are virtually impossible (unless you're buying research-grade cameras that cost as much as a luxury car). As I've mentioned elsewhere, ThermalExpert declined to sell me a 640 x 480 30Hz 'dongle' camera because they couldn't organise an export licence - and it's interesting to note that even their own 640 x 480 promotional videos shot in Europe were done with the 9Hz version of their own camera.

[Ultrapurple]

First light.

30mm f/1.1 lens, probably iffy focus. Running on Samsung Galaxy S7, no obvious problems with dropped frames or suchlike. More will follow when I've had a chance to play properly.

[Fraser]

 

Fraser

[marshallh]

Looks like very low noise so far, will look even better once you get focus sorted 

[Ultrapurple]

The images are definitely much less noisy than the other Therm-App cameras. But there's an issue with the driver app at the moment - it's only displaying about a third of the pixels. For reasons unknown it's only using a 384 x 288 window (= the same res as the older sensor) and then upscaling it. Saved still images do the same, but video is saving at full resolution. I spent most of today trying to get to the bottom of various oddities that just didn't feel quite right and in the end proved for definite that there's something odd going on (see images). I'm sure it's a straightforward fix for Opgal - an update to the driver app I expect - so I hope soon to be able to make really good thermal images.

The samples below were made using a 6.8mm lens, which exhibits strong vignetting. It's not really designed for 640 x 480 17µm sensors, though it does reasonably well. It was that lens that first made me think something was amiss - there was strong vignetting on one corner, which ain't right. You can just see the vignetting on the bottom right corner of the colour image; the black & white Night Vision mode image was from the recorded video and shows equal vignetting at each corner, which is as it should be.

Note: the thermal image below isn't truly representative of the quality of the Therm-App Pro - I expect to be able to get much better results in due course.

[Ultrapurple]

Opgal has been able to reproduce the issue and is working on a fix.

I expect it will simply be a small amendment to the driver software. I will report progress.

I have received a new version of the APK and it definitely seems to work.

ThermApp Plus and ThermViewer presently do not.

There are some residual marks from calibration visible on the images at low thermal contrast. I'm hoping that ThermApp Plus or ThermViewer will begin to support the Pro (I'm in conversation with the authors).

[Ultrapurple]

I have had the chance to do some initial tests on the Therm-App Pro. There's a short video on Flickr showing the results with a 35mm f/1.1 lens. Vehicles are easily seen at 3/4 of a mile.

I also did a superresolution test, accidentally stacking two superresolution filters and ending up with video at 2560 x 1920 (!) - you can see the results here.

For various reasons I run the superresolution software on a virtual XP machine in my (fast) home PC. Rendering the ~48 second video took about two hours. I expect it would be a good deal faster if I wasn't using a virtual machine. The source video was not of particularly good quality - I miss the image control and cleanup facilities built into ThermApp Plus and ThermViewer. Hopefully one or both of those will soon support the Pro. However, these videos do demonstrate the low noise of the Therm-App Pro. In due course I intend to do some side-by-side tests with the lower resolution Therm-App.

[Vipitis]

I am interested in the topic of super resolution.

Could you share a bit more? Like the software you used?

I don't think the results of a low resolution input will be any good. But I would like to try it

[Ultrapurple]

Hi Vipitis

Just to be sure we are talking about the same thing, superresolution is not the same as simply re-scaling a video. Superresolution extracts details smaller than a pixel by motion estimation techniques, so it brings out details that weren't apparent in the original video file. There is a good description here and a more technical description on Wikipedia here.

I have tried several video superresolution systems but the one that has worked best for me is Video Enhancer 2.2. It doesn't accept all video file types so you may need local or online converters, but it is reasonably easy to use and works well with the 25Hz Therm-App Hz. I don't think I've tried it with with the 8.7Hz version of the camera, but I guess it should work after a fashion. I expect the wobbly nature of the 8.7Hz video (caused by the 'rolling shutter' effect) would confuse the algorithm.

You can download Video Enhancer 2.2 for a 30-day free trial. Tip: if you run it in a virtual machine and the trial ends, you can ditch the VM, start another, and get another 30 days. This is very useful if you only occasionally need the software, otherwise I'd recommend buying it. If you look around you can find discount codes for at least 20% (and there's a Facebook link that gives a 30% discount). (I want to make it clear that I have no association with the company and don't receive any special discounts or anything.)

Another piece of software I tried was STOIK Video Enhancer, which I didn't find quite as good. It also watermarks videos when in trial mode.

ThermViewer, for Therm-App, has built-in superresolution that works very well indeed, upgrading the resolution to 768 x 576 in real time. I haven't yet posted any ThermViewer-enhanced video anywhere but I will do shortly (at the time of posting this I don't have access to the file I have in mind but I will update this post later).

[Vipitis]

thanks for the information,

I will look into them.

but for me there is only a single option to capture video. and that is the myFLIR app. It always uses MSX, locking the tempscale causes some issues with the shutter going crazy, and upscales internally to 640x480 which causes some bad interpolation on edges and ghosting artifacts.

I need to find a way to extract raw 60x80 video to attempt anything.

[Ultrapurple]

Side-by-side tests of the Therm-App 8.7Hz, Therm-App updated to 25Hz, and Therm-App Pro

I had a few minutes spare so I made some stills with the three Therm-App cameras. My Therm-App updated to 25Hz still gives relatively poor results on The Therm-App app because of offset issues, but these are corrected more or less automatically by ThermViewer (and can be made to appear much better by choosing the various enhancement options). However, to make the playing field as level as I could manage I turned off all the clever stuff in ThermViewer for this test. ThermViewer also heavily watermarks still images so the sample here is a still from a video (which ThermViewer upsamples to 720 x 576via superresolution). The 8.7Hz Therm-App, and Therm-App Pro, were both driven by the stock app.

With me so far?

In essence, within the limits of what I had available and the software that would drive the various cameras I tried for a level playing field. The scene is nothing special - a typical corner of an office, with a reasonable run-of-the-mill thermal contrast range.

The 8.7Hz and 25Hz 384 x 288 cameras had 19mm f/1.1 lenses. The Therm-App Pro had a 35mm f/1.1 lens (giving roughy the same FOV as the 19mm on the smaller sensor). All were allowed to warm up for a similar length of time (a few minutes) and shot hand-held from about the same position.

I believe the Therm-App Pro shows significantly reduced noise compared to the other two cameras, much better resolution of course, and better definition. In particular, look at the phone - you can clearly see the buttons on the Pro image but they're hard to make out on the other images.

What do you think?

[Fraser]

Thanks for the pictures 

When it comes to scene fine detail, the relatively low spatial resolution of thermal cameras does become obvious. Imagine a 320 x 240 pixel web cam versus a 640 x 480 webcam. Even the VGA webcam will produce relatively low detail images at a distance. In the thermal camera domain we are used to the thermal images being relatively low in fine detail when viewing objects at a distance. Such is normally acceptable for thermography but can be a severe limitation in making pretty images of say a landscape scene. Some clever forum members have used narrow FOV lenses and image stitching to produce very impressive high pixel count images.

Your 640 x 480 pixel camera has four times the number of pixels, when compared to the 320 x 240 pixel version, covering the same field of view and that is a significant increase in image detail. A VGA thermal camera can provide very impressive thermal images compared to what we have come to accept from regular cameras running 160 x 120 or 320 x 240 pixel cores.

I have a beautiful 640 x 480 pixel DRS CCTV pan tilt camera but some of the advantages of the higher resolution sensor are lost due to a wide FOV for area coverage rather than fine detail. What it does provide is more detail than the same FOV viewed by a 320 x 240 pixel camera, which was the manufacturers intention for CCTV applications.

VGA thermal cameras are very nice to have and can provide some very pretty images. Manufacturers have previously used Image Fusion with visible light images and MSX in an attempt to fool the eye into thinking more thermal image resolution is present, than actually is. Such is normally not necessary with a VGA camera.

The next step up in resolution to 1024 x 768 pixels will likely be the most impressive step in resolution increase as you can obtain very nice fine detail with that number of pixels  With time, we may see such cores being offered in cameras that can be afforded by mere mortals like us 

Fraser

[Ultrapurple]

Thanks Fraser

I think thermal imaging is presently roughly 30 years behind visible light cameras. About 25 years ago my employer needed a digital camera for taking location reference pictures in the field. Only two models were really available - one offering 320 x 240 and the other 640 x 480 pixels. That's visible, not thermal. We chose the higher resolution because, while all agreed that the photos from the 320 x 240 were just about good enough, the extra resolution was better. I think we paid something like £1000 for the camera, which may well have been a QV-10. Either way, the price of the camera was a very small part of the whole project cost.

320 x 240 resolution is roughly the same as a NTSC VHS video tape. 384 x 288 (as per Therm-App et al) equates broadly to PAL VHS, while 640 x 480 is comparable to a DVD or mini-DV video recording. There definitely are 1024 x 768 uncooled sensors available out there on the civilian market, though they're rather expensive (FLIR has at least one range of that resolution, which starts at the price of a luxury car - and a full set of lenses can double that). I strongly suspect there are higher resolution uncooled LWIR sensors around, for military or at least experimental purposes. No-one's talking much about those, though.

Not all applications require the same aspect ratio: a scope might benefit from a square (or even round!) image, whereas a driving aid might find an extreme widescreen view more relevant (drivers generally don't need to see the clouds). Some relatively commonly available imagers are near-square (5:4, eg 640 x 512) as opposed to the 4:3 of more common types (320 x 240, 384 x 388, 640 x 480 etc).

I'm well aware that there are technical differences between the way thermal and visible light sensors presently work, and that the manufacture of thermal sensors is much harder (not to mention complex, needing a readout IC (ROIC) bonded directly to, and at least the same size as, the thermal sensor matrix). But my suspicion is that there may be a development around the corner with some hitherto-unknown thermally sensitive arrangement that simplifies - cheapens - the sensors. Lenses remain an issue, with nothing presently much better than germanium or zinc selenide, although the chalcogenide glasses may be catching up). Everything after the sensor, in terms of image processing and display, is essentially the same as that used for visible light images. That part is only more expensive because relatively few are made and there are none of the huge economies of scale that you get in, say, phone cameras.

As I've said elsewhere, I look forward to the day we can all buy multi-megapixel thermal imaging cameras. High resolution thermal images can look good - see here, here, here, here, here, here, here, here, here, here, here and here for just a few examples of the state of the art in stitched hi-res thermal images. Some of these took quite a while to put together. Perhaps it's a pipedream, but  imagine if hi-res thermal imaging was as simple as a point-and-shoot mobile phone camera. And as inexpensive. Time will tell; those few of us today making pretty pictures with thermal cameras are true pioneers.