Author Topic: High resolution thermal cores for sUAS  (Read 9928 times)

0 Members and 1 Guest are viewing this topic.

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
High resolution thermal cores for sUAS
« on: February 19, 2019, 08:07:16 pm »
We currently use IR imagery for locating equipment defects with sUAS and the FLIR XT. It's my understanding that the XT and XT2 are essentially the TAU 2 core made by FLIR.

Great, so our current best integration option is a 640x512 radiometric camera on a gimbal with <30mK sensitivity. We've come a long way, but I want more resolution.

Producing quality thermal orthomosaics without significant user input correction to the processing is the ultimate goal here and for that we need more resolution

I see mostly resolutions below our current spec discussed here, but there has to be more right? What other cores are on the market that should we be looking for in the <10lb range and resolution higher than 640x512?

P.S. what's all this opto-mechanical microscan magical resolution doubling feature I'm reading about? It sounds promising, but there's not a lot of discussion on it. I assume a true high res core is better than a core claiming the same resolution while utilizing the "doubling" technology
« Last Edit: February 19, 2019, 09:48:09 pm by Hyper_Spectral »
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #1 on: February 19, 2019, 08:29:10 pm »
at 640x512@30hz and radiometric you are already way at the top.
There are higher resolution uncooled detectors, only a few. The very highest resolution uncooled sensor is a TWV1912 made by Fairchild/BAE - and to my knowledge there is one available camera build around it by Sierra Olympic, the Vayu HD. While I was able to hold myself back from asking them about the price, I am expecting it to be in the 30k range.

But implementation will be much more difficult, eventhough they market it for airborne use, it does not produce the radiometric flir images that some photogrammetry apps now support. I would ask for a demo, if you are in such a position where they answer your and workout the software workflow with help from the developers of whatever software you are currently using, if they offer specelized solution.

Any kind of airborne detector that is cooled, can have even higher resolutions, these are normally build for police helicopters, or military aircraft and focus on target tracking, range indication and high zoom levels and less radiometric data for mosaics.

Is superresolution the thing you talk about? It normally works on a camera that is in about the same place and takes 4 images in short succession and using the hand wiggle or wind to move the sensor by a fraction of a pixel resulting in a single high resolution image when combined. In photogrammetry you have a point cloud, where every point is based on data gathered, to get better resolution, just gather more points by flying over the same section from a different angle and direction. In theory there are many ways to interpolate more data points based on the seemingly random intervals you have with your reconstruction when making up a wavelet direction. But I have yet to read about it in detail. I am not sure how advanced software is in the field, but using a visible light camera to create the pointcloud gives you more data points due to the high resolution, if possible a solid model could be reconstructed and the thermal pixels projected onto them. Giving you a "3D MSX" dataset.
 

Offline Chanc3

  • Frequent Contributor
  • **
  • Posts: 439
  • Country: gb
Re: High resolution thermal cores for sUAS
« Reply #2 on: February 19, 2019, 10:20:34 pm »
sUAS and thermal imaging cameras basically describes my career! There are higher resolution cameras out there, but you won't find many theatre radiometric. Is that a requirement?

They're not that big, but not much has been done to develop them into a usable system on drones.
 

Offline TooQik

  • Regular Contributor
  • *
  • Posts: 56
  • Country: au
Re: High resolution thermal cores for sUAS
« Reply #3 on: February 20, 2019, 07:50:28 am »
These might be worth a look:

https://ipi-infrared.com.au/product/keii-hl-1024-uav-special-module/

http://en.keii.com.cn/index.php/Product/detail/paretn_id/0/cat_id/14/goods_id/27#path

I have no idea on quality, cost etc though. Others here might be able to provide better examples.
« Last Edit: February 20, 2019, 08:02:49 am by TooQik »
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #4 on: February 20, 2019, 02:58:12 pm »
at 640x512@30hz and radiometric you are already way at the top.
There are higher resolution uncooled detectors, only a few. The very highest resolution uncooled sensor is a TWV1912 made by Fairchild/BAE - and to my knowledge there is one available camera build around it by Sierra Olympic, the Vayu HD. While I was able to hold myself back from asking them about the price, I am expecting it to be in the 30k range.

But implementation will be much more difficult, eventhough they market it for airborne use, it does not produce the radiometric flir images that some photogrammetry apps now support. I would ask for a demo, if you are in such a position where they answer your and workout the software workflow with help from the developers of whatever software you are currently using, if they offer specelized solution.

Any kind of airborne detector that is cooled, can have even higher resolutions, these are normally build for police helicopters, or military aircraft and focus on target tracking, range indication and high zoom levels and less radiometric data for mosaics.

Is superresolution the thing you talk about? It normally works on a camera that is in about the same place and takes 4 images in short succession and using the hand wiggle or wind to move the sensor by a fraction of a pixel resulting in a single high resolution image when combined. In photogrammetry you have a point cloud, where every point is based on data gathered, to get better resolution, just gather more points by flying over the same section from a different angle and direction. In theory there are many ways to interpolate more data points based on the seemingly random intervals you have with your reconstruction when making up a wavelet direction. But I have yet to read about it in detail. I am not sure how advanced software is in the field, but using a visible light camera to create the pointcloud gives you more data points due to the high resolution, if possible a solid model could be reconstructed and the thermal pixels projected onto them. Giving you a "3D MSX" dataset.
Thank you for providing that information, I've put in requests for information from both Fairchild and Sierra imaging.

Superresolution doesn't sound quite like what they're referring to, but it's definitely possible it's just a marketing wank version of superresolution. There was a research article on it, but I seem to have misplaced the link. You can find the brochure here:
https://www.infratec-infrared.com/downloads/en/thermography/flyer/vc-hd/infratec-variocam-hd-b-mail-en.pdf

Your last paragraph is very close to the current "standard" post-processing we see, but let me explain for the sake of everyone interested in this topic.

Right now there are two key pieces of software used in the sUAS mapping industry, Pix4D and Agisoft Photoscan. They use different algorithms, to my understanding, but they're probably pretty close in technique at this point. Pix4D is probably the most common, though. This is thanks mostly due to it's massive improvement in the last 2-3 years. The point clouds and orthomosaics produced from RGB imagery we're getting now versus just a year or two ago are drastically different and much better. Thermal datasets are also much better, but they still suck. Bad. This is mostly due to the differences in resolution in my opinion. A 4000x3000 image is much easier to match to another 4000x3000 image. Matching 640x512 imagery has to be difficult. Most of the orthomosaics you see produced are made with 90/90 overlap at high altitudes sacrificing final GSD. We want a final GSD of 5cm/px (100-150' AGL with the XT 13mm lens) while ~10cm/px GSD is much more reasonable when you consider the post-processing capabilities and time requirements for simply collecting the data. Consider the flight time is 3-4x the time at less overlap. These problems ultimately lead to a "product," but not without significant input into Pix4D (manually creating tie points between the RGB point cloud and thermal imagery). Poor final GSD, high time investment in flights and processing, and a poor final product are what we see in the thermal post-processed data world right now. Sure, you can do it. But it's not worth it, imo.

Flying over the same area multiple times probably would help with matching and the point cloud, but it's the same thing as adding more overlap (in a less efficient manner) and it doesn't equate to the realm of LiDAR where I do see a need for densifying point clouds. LiDAR is another topic a little further down the road for us.
« Last Edit: February 20, 2019, 08:19:24 pm by Hyper_Spectral »
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #5 on: February 20, 2019, 03:04:21 pm »
sUAS and thermal imaging cameras basically describes my career! There are higher resolution cameras out there, but you won't find many theatre radiometric. Is that a requirement?

They're not that big, but not much has been done to develop them into a usable system on drones.
This is purely for mapping and finding defects via AI/ML. The pretty picture is used by humans to verify an issue, and it's nice to have, but finding and visualizing defects is the ultimate goal here. So no, it doesn't need to be radiometric.

I am interested in the cores/sensors you're referring to though, do you have a datasheet?


These might be worth a look:

https://ipi-infrared.com.au/product/keii-hl-1024-uav-special-module/

http://en.keii.com.cn/index.php/Product/detail/paretn_id/0/cat_id/14/goods_id/27#path

I have no idea on quality, cost etc though. Others here might be able to provide better examples.

Very cool and a good start. Does anyone have information on the cores used in those sensors? I'm trying to find a datasheet but it's not listed. The fact that the website doesn't list the spectral bands or actual sensitivity is a bit alarming.

**Big Edit**: surprise surprise, the KEII HL-1024 is listed quite frequently on Russian websites, consistent with some other companies producing UAV sensors I've seen in that area of the world (WIRIS).

Actually, it looks like the HL-1024 and CL-1024 are the same camera..? Same core?
CL-1024
http://en.keii.com.cn/index.php/Product/detail/paretn_id/0/cat_id/14/goods_id/27#path

Specs HL-1024 https://www.pergam.ru/catalog/bpla/podveski/hl-1024.htm#specification

Found a video on that specific camera, does the video clip from the thermal camera look heavily edited to anyone else or is it just me?


This only gets more interesting, it looks like Aeryon is familiar with Keii and wanted to provide this as a solution themselves:
https://www.unmannedsystemstechnology.com/wp-content/uploads/2013/10/Keii-MC-1-Payload-Inspection-Brochure.pdf
« Last Edit: February 20, 2019, 08:17:41 pm by Hyper_Spectral »
 

Offline eKretz

  • Frequent Contributor
  • **
  • Posts: 870
Re: High resolution thermal cores for sUAS
« Reply #6 on: February 20, 2019, 03:48:10 pm »
That video is pretty clearly not actual footage from the camera. Looks like computer modeled video to me.
 
The following users thanked this post: svgurus

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #7 on: February 21, 2019, 12:54:52 pm »
Some information from BAE systems, the producer of the aforementioned TWV1912 core, who seems to actually be interested in helping the relatively small part of the market who wants these optics.

I'm not sure who the second company is, no email yet, but I'll report back when I do

"Seeing that you are a remote pilot, I assume that you are looking for companies that are already making a camera for a drone using our full HD imager, correct?  If so, please know that Sierra Olympic has an advanced prototype of our full HD thermal sensor.  We are doing a full launch of the product in April, so you should expect to start seeing companies making drone cameras with our sensor by year end. 

I specifically already know one company that is going to be doing precisely that.  I will forward your email to them so they can reach out to you."
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #8 on: February 21, 2019, 01:06:55 pm »
Another company who got back to me is called Workswell, I've use their WIRIS 640x512 radiometric cameras in the past and they worked well.

It was a little bulky and putting it on a gimbal wouldn't be as easy as the FLIR XT and it's basic DJI mount, but that could change with their new camera.

Problem is, it's a standard 640x512 sensor with superresolution, and I haven't seen superresolution imagery actually improving photogrammetry results. Another issue is the lack of geotagging capability without some sort of S/CAN BUS GPS output from the sUAS

Here's their information:
https://www.drone-thermal-camera.com/products/workswell-wiris-pro-uav-thermal-imaging-camera-for-industry/

Further reading, an excellent article on the methodology of upscaling of thermal imagery using CNN (neural networks):
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6111996/
« Last Edit: February 21, 2019, 01:24:25 pm by Hyper_Spectral »
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #9 on: February 22, 2019, 12:39:05 pm »
Here's the final correspondence I expect to receive from BAE regarding the sensor and who will be manufacturing the actual case and gimbal:

"The company that will be making the drone camera is not Sierra Olympic.  I forwarded your name to the other company I am working with and I expect they will reach out to you in due time - like by year end.

The information you seek will be published by the camera makers once they configure their cameras with our sensor.  The easiest way to think of what is going to be possible with our full HD sensor, is that it is the same performance as our VGA sensor available now, but with 6.5X the number of pixels. 

BAE makes the detector.  Our partners, like Sierra Olympic, make the camera, gimbal, lens decision, etc."

Very cool, I wish I had a price and a better timeline but it looks like for a high res thermal core from a reputable company we'll have to wait.
« Last Edit: February 25, 2019, 01:33:03 pm by Hyper_Spectral »
 

Offline eKretz

  • Frequent Contributor
  • **
  • Posts: 870
Re: High resolution thermal cores for sUAS
« Reply #10 on: February 23, 2019, 09:02:15 am »
That will be pretty amazing. In 10 years perhaps there will be 4k resolution thermals! And the Workswell camera actually works well?  ;D
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #11 on: February 23, 2019, 01:29:43 pm »
Glad you are in contact with BAE, I hope Sierra Olympic responds to you as well. I want to know the price of things.

The paper you linked is very very informative for me. It really sounds like they put a lot of testing and thought into different single frame superresolution methods and their examples look great. I need to read it again while awake and take some notes, but there wasn't any software they offered to download and try our own, right?
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #12 on: February 25, 2019, 01:42:09 pm »
Glad you are in contact with BAE, I hope Sierra Olympic responds to you as well. I want to know the price of things.

The paper you linked is very very informative for me. It really sounds like they put a lot of testing and thought into different single frame superresolution methods and their examples look great. I need to read it again while awake and take some notes, but there wasn't any software they offered to download and try our own, right?
Price, integration via SBUS/CANBUS/PWM, and final size/weight are going to be the biggest factors for the cores uses in sUAS imo

Regarding the paper, it appears most of the research papers like that are based on somewhat proprietary code and it's not going to be GUI or even command line based. There are many CNN repositories out there though that are becoming quite commonplace. Software may exist, but no doubt it needs to be tailored/trained to each camera.

An important distinction that needs to be made is between super resolution and upscaling. These are two very different strategies. Within super resolution there's also multiple strategies, we care about the capture of multiple images to make one high res image, instead of using a single image and trying to denoise and upscale it. My thought process is that "real" pixels are better than interpolated pixels.

That will be pretty amazing. In 10 years perhaps there will be 4k resolution thermals! And the Workswell camera actually works well?  ;D
4k thermal imaging would be amazing, truly reaching the threshold of need.

It did works well, it was a daily joke to ask if it was working well when we had 3 of them lol
« Last Edit: February 26, 2019, 02:54:22 pm by Hyper_Spectral »
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #13 on: March 13, 2019, 12:00:43 pm »
ULIS, anyone?

http://www.viewprotech.com/index.php?ac=article&at=list&tid=127

Found this chinese company producing cameras and gimbals that look like clones of DJI's products and dug a little deeper to find the company who provides the cores is ULIS. Even better is that they offer a thermal core in 1024x768
https://www.ulis-ir.com/products/pico1024.html

I'll update this with more information, hopefully it's implemented in a camera somewhere.
 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #14 on: March 13, 2019, 04:01:39 pm »
I only have experience of the 384x288 and 640x480 versions of the Ulis sensors but I think they're reasonably good.  Opgal use them in a number of their products, as do many other companies. The sensors come from France so they're not subject to the same parochial regulations as sensors made in the USA.

I am rather hoping Opgal launches a 1024x768 version of the Therm-App...

Ulis are a bit cagey about their detailed information. I entered into a conversation with them about the 1024x768 sensor but failed to persuade them to let me see the technical datasheet. Others may have had more luck. The few Ulis datasheets I've seen are all stamped with various (commercial) protective markings.
Rubber bands bridge the gap between WD40 and duct tape.
 
The following users thanked this post: Hyper_Spectral

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #15 on: March 14, 2019, 11:57:17 am »
The whole thing is a fricken mess. I get that this is a competitive industry but sharing specs and pushing more people to integrate the core will sell more cores.

 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #16 on: March 14, 2019, 01:55:29 pm »
The whole thing is a fricken mess. I get that this is a competitive industry but sharing specs and pushing more people to integrate the core will sell more cores.

It's not really the industry's fault. I'm sure any manufacturer would love so sell as many cores/cameras as the market would stand.

The problem is that any decent thermal camera is 'dual-use' (ie handy for the bad guys shooting back at you) and the governments of more-developed nations are keen to prevent the "wrong people" getting their hands on the technology. Unfortunately, this means that many innocent users are denied access to products that the manufacturers' commercial departments would love to be able to sell.

There is some change in the air. As has been said quite colourfully elsewhere by a native of that country, some Chinese suppliers don't observe international regulations in the same way as, say, Flir, meaning that there is likely to be an increasing tide of indigenously developed sensors and cameras of increasing sophistication arriving on the market. I believe that production of a 1024x768 sensor has been announced; it's not too big a stretch of the imagination to guess that higher resolutions are in development. I suspect 320x240-class Chinese-sourced thermal cameras will be almost commodity items fairly soon, and that much higher resolutions will also be available for far less than one expects to pay at the moment.
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #17 on: March 14, 2019, 08:49:49 pm »
The whole thing is a fricken mess. I get that this is a competitive industry but sharing specs and pushing more people to integrate the core will sell more cores.

It's not really the industry's fault. I'm sure any manufacturer would love so sell as many cores/cameras as the market would stand.

The problem is that any decent thermal camera is 'dual-use' (ie handy for the bad guys shooting back at you) and the governments of more-developed nations are keen to prevent the "wrong people" getting their hands on the technology. Unfortunately, this means that many innocent users are denied access to products that the manufacturers' commercial departments would love to be able to sell.

There is some change in the air. As has been said quite colourfully elsewhere by a native of that country, some Chinese suppliers don't observe international regulations in the same way as, say, Flir, meaning that there is likely to be an increasing tide of indigenously developed sensors and cameras of increasing sophistication arriving on the market. I believe that production of a 1024x768 sensor has been announced; it's not too big a stretch of the imagination to guess that higher resolutions are in development. I suspect 320x240-class Chinese-sourced thermal cameras will be almost commodity items fairly soon, and that much higher resolutions will also be available for far less than one expects to pay at the moment.

You do make fair points, but I've been on the ITAR side of things way more classified of this in the past and it wasn't anywhere near that level of hassle.

I don't recall any restrictions on the sale of high res thermal cameras to the private sector in the US?..  I figured US based companies using them domestically with substantiation for their use would be exempt. That's how it is for the 640x512 cameras right now.

If an OEM has their cameras produced in another NATO country, say france, can they not export it to the US for sale? Exports to unsecure countries being restricted makes sense, but not when it's OEM manufacturing for a specific company selling in the US market.

I continue to truly think the sUAS industry is just a mess over here and aside from the few that do want higher res thermal cameras they really aren't needed in most industry that uses handhelds. sUAS based aerial mapping is like a whole new realm without major competition right now. DJI, a toy drone manufacturer, owns the market and literally influences the FAA's decisions.
« Last Edit: March 14, 2019, 08:53:02 pm by Hyper_Spectral »
 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #18 on: March 15, 2019, 01:39:02 pm »
@Hyper_Spectral - I think you've hit the nail on the head: the situation is very different within the USA from much of the rest of the world. A US Citizen can buy, off the shelf for private use, items that are essentially banned from export (yes, I know, you can license just about anything if you're big enough). I have seen many things on eBay.com that I would love to buy from here in the UK but the sellers simply can't send the things.

As I mentioned, I think things will change. Once (say) Chinese Full HD 60Hz sensors are a commodity item there will simply be no point in (arguably) "harming US industry" by preventing the export of comparable devices.  eBay already has many Chinese 9Hz 320x240 telescope-style thermal imagers at around the GBP500 price point (~US$660); I suspect it won't be long before we start seeing higher frame rates and then 640x480-class devices. Performance won't be as good as a sniper scope with a big fat expensive germanium lens, but it's easy to see the way the wind is blowing.

Ten years ago the idea of a good resolution miniature thermal camera on a drone was pretty much still in the realms of science fiction. Who knows what the next ten years will bring?
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #19 on: March 15, 2019, 08:27:49 pm »
Ten years ago the idea of a good resolution miniature thermal camera on a drone was pretty much still in the realms of science fiction. Who knows what the next ten years will bring?

This is the root of the problem. It seems like its expected to take another 5+ years, but the cores already exist. All it takes is for flir to integrate it, or for them to produce it in the US. Even with a 1-3 year development time for the packaging of the core for use on a sUAS, its probably been 3 years or more since the high res microbolometer cores become available to OEMs (surely a large US company could make them an OEM for $ early). 

Based on the information provided by Fairchild, there are people looking into it, but why is it only just now being developed?

To sell a thermal core in the US, must the thermal core be produced in the US?

 

Offline TooQik

  • Regular Contributor
  • *
  • Posts: 56
  • Country: au
Re: High resolution thermal cores for sUAS
« Reply #20 on: March 19, 2019, 12:44:48 pm »
Here's one that uses a ULIS 1024 x 768 sensor array that I hadn't seen before:

https://www.sofradir-ec.com/products/xga-resolution/atom-1024.html

Data sheet is just after the description.
 

Offline LaserSteve

  • Super Contributor
  • ***
  • Posts: 1270
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #21 on: March 19, 2019, 11:11:49 pm »
One trick is to mechanically displace the sensor 1 or 2 pixels horizontally  .  I've had non-thermal cameras that do that very well...
You trade off time for resolution.  Its crude, but it works.

Steve
"What the devil kind of Engineer are thou, that canst not slay a hedgehog with your naked arse?"
 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #22 on: March 20, 2019, 11:20:07 am »
One trick is to mechanically displace the sensor 1 or 2 pixels horizontally  . ...  Its crude, but it works.

Steve

I was under the impression that this resolution-enhancement technique moved the sensor half a pixel (in each dimension) and combined four exposures to make a single higher-resolution image.

Essentially, you're relying on fill-factor effects: an individual pixel is not usually equally sensitive across its whole span and height; there is normally a sensitivity minimum at each edge (where it abuts each adjacent pixel). By shifting the array so that the area of maximum sensitivity moves to the position where previously there was little or none, the effective resolution can be more or less doubled in both axes. The limiting factors are the fill factor and sensitivity profile of each pixel: it may be that it's possible to enhance actual resolution more on one axis than the other, depending on the design of the sensor.

It's a technique I've only ever seen used on thermal imaging systems but there's no reason why it shouldn't be applied to other wavelengths.
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #23 on: March 20, 2019, 11:30:48 am »
The Leonardo SLX Merlin is just one example of a thermal camera that uses "microscan" to enhance resolution by a factor of four (from 1024x768 to 2048x1536) - the datasheet includes a couple of nice images including a rough diagram of the microscan platform. I'm sure there's much more, much better documentation out there somewhere - for example, I expect there are a few patents involved.

Before anyone gets too excited, the Merlin is a bit big and power-hungry to fit on a domestic UAV. I suspect a private buyer might also have problems persuading the manufacturer to sell them one.
« Last Edit: March 20, 2019, 11:33:45 am by Ultrapurple »
Rubber bands bridge the gap between WD40 and duct tape.
 
The following users thanked this post: TooQik

Offline TooQik

  • Regular Contributor
  • *
  • Posts: 56
  • Country: au
Re: High resolution thermal cores for sUAS
« Reply #24 on: March 20, 2019, 11:32:27 am »
One trick is to mechanically displace the sensor 1 or 2 pixels horizontally  . ...  Its crude, but it works.

Steve

I was under the impression that this resolution-enhancement technique moved the sensor half a pixel (in each dimension) and combined four exposures to make a single higher-resolution image.

Essentially, you're relying on fill-factor effects: an individual pixel is not usually equally sensitive across its whole span and height; there is normally a sensitivity minimum at each edge (where it abuts each adjacent pixel). By shifting the array so that the area of maximum sensitivity moves to the position where previously there was little or none, the effective resolution can be more or less doubled in both axes. The limiting factors are the fill factor and sensitivity profile of each pixel: it may be that it's possible to enhance actual resolution more on one axis than the other, depending on the design of the sensor.

It's a technique I've only ever seen used on thermal imaging systems but there's no reason why it shouldn't be applied to other wavelengths.

Isn't this a similar technique to that used in 3D scanners to build up a point cloud?
 

Offline Ultrapurple

  • Super Contributor
  • ***
  • Posts: 1027
  • Country: gb
  • Just zis guy, you know?
    • Therm-App Users on Flickr
Re: High resolution thermal cores for sUAS
« Reply #25 on: March 20, 2019, 11:36:15 am »

Isn't this a similar technique to that used in 3D scanners to build up a point cloud?

I'm afraid I don't have any experience on that subject but there are quite a number of techniques for 3D scanning so it's possible one method includes a related approach. Anyone better-informed know more?
Rubber bands bridge the gap between WD40 and duct tape.
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #26 on: March 20, 2019, 08:34:34 pm »
Shifting the sensor by half a pixel (or 1.5, etc...) can be done. It is a feature of some stills cameras. Hasslebladfor example shifts their sensor in two axis and takes 6 captures (because of bayer matrix). Sony offers a similar feature in their high end cameras called Pixel shift. These only work when you put your camera onto a solid tripod. This is pretty much superresolution done for you by the sensor stabilizing system. I don't think sensor stabilization is common in thermal cameras, because the pixels are much larger and the resolution is lower, so small shakes don't get noticed that much.

For 3D models photogrammetry is used. I am using the freeware VisualSFM, but there are two main software options for aerial photogrammetry, theses take advantage of the geotagging to reduce the matching needed. In theory you just have to take more images(fly slower, higher framerate) to get more data points, or get a narrower FoV(change lens or fly lower), but this does not work out in practise. The 3D point cloud that is generated has far less points than pixels you record, and a good amount of overlay is needed to get a good reconstruction. The next step is to connect those 3D points into a 3D model by connecting them with triangles into a polygon, now you go back into your captured images and project them back at the model to texture it. Having a better point cloud only helps you to get more detail, not more resolution really. This texture mapping probably ignores the small overlaps and does not use it for superresolution, it might be something that has been looked into - but I haven't read about it.

It's a idea I head in the last year that I wrote down and collected resources on. So if anyone knows more - let me know.

 

Offline Max Planck

  • Regular Contributor
  • *
  • Posts: 97
  • Country: pl
Re: High resolution thermal cores for sUAS
« Reply #27 on: March 24, 2019, 07:19:19 pm »
 Hyper_Spectral,
you could also have a look at Infratec Variocam modules or SCD's modules.

Max
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #28 on: March 26, 2019, 02:37:27 pm »
So I came across an Israeli company who produced 1024x768 cores way back in 2013. Someone must be familiar?

http://www.scd.co.il/Bird-XGA-17
 

Offline Fraser

  • Super Contributor
  • ***
  • Posts: 13148
  • Country: gb
Re: High resolution thermal cores for sUAS
« Reply #29 on: March 26, 2019, 02:42:29 pm »
Hyper Spectral,

I know of BIRD and it’s products. They produce cores for military applications. Israeli Defence contractors are very challenging to work with and you have to be friends with MOSSAD when exporting the more sensitive technology. Get it wrong with a sensitive Israeli product and you can honestly expect a visit from MOSSAD representatives for a chat.

The Israelis are a very capable technology producer. RICOR Stirling Coolers we’re chosen to fly on NASA missions and probes due to their quality.

Fraser
« Last Edit: March 26, 2019, 02:45:24 pm by Fraser »
If I have helped you please consider a donation : https://gofund.me/c86b0a2c
 
The following users thanked this post: Hyper_Spectral

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #30 on: March 26, 2019, 04:33:50 pm »
Typical Israel stuff, I'll leave it there. I figured they wouldn't mind exporting to the US but eh you never know
 

Offline Max Planck

  • Regular Contributor
  • *
  • Posts: 97
  • Country: pl
Re: High resolution thermal cores for sUAS
« Reply #31 on: March 26, 2019, 06:06:23 pm »
I was thinking more about:
https://scdusa-ir.com/

which is their direct US representative or
https://www.infratec-infrared.com/downloads/en/thermography/flyer/security/infratec-variocam-hd-head-security-h-en-mail.pdf

from Infratec, a 1024 × 768 pixels module giving 2,048 × 1,536 resolution with an opto-mechanical MicroScan.

By the way, on their web pages for different modules, SCD often publishes articles describing rather in depth the design and features of their products. Interesting reading. 

Max
« Last Edit: March 26, 2019, 06:08:36 pm by Max Planck »
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #32 on: March 26, 2019, 06:43:54 pm »
Thank you Max, I had inquired with them about 3 weeks ago and they plainly said no, we do not produce for that type of application and we do not intend to.

So it's a matter of a 3rd party picking up and integrating that core.
 
The following users thanked this post: Max Planck

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #33 on: March 27, 2019, 12:33:24 pm »
I just emailed the USA portion of the company in hopes of them having more information for the US market. The email listed is undeliverable so I used their contact us form. I'll report back if I hear anything more. The only core that fits requirements is the Bird XGA 1024x768. The other cores are far too large unless we go up in sUAS size and carry a single payload.
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #34 on: March 28, 2019, 05:01:40 pm »
as the BAE/Fairclip 1920x1200 sensor was mentioned here - BAE is integrating it into tanks: https://www.baesystems.com/en-us/product/360-multifunction-vehicle-protection-sensor

but this also means there have finished to work on the lenses and have at least two options. That was part of the Sierra Olympic holdback for pushing the core.
 
The following users thanked this post: Hyper_Spectral

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #35 on: March 29, 2019, 03:48:39 pm »
Very interesting, and a plausible explanation. I wonder if there's any increased complexity in lens design for higher resolution microbolometers  :-//
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #36 on: March 29, 2019, 05:15:18 pm »
Definitely there is. As the sensor area gets larger, lenses get larger. That is a mail selling point behind d new technologies that use 12 or 10micro pixels.
Most lenses use Germanium and that has a rather high refractive index, which is again countered by the long and broad wavelengths. You get the chromatic abboration effect, but because the sensor is broad monochrome - you just get blurring instead of color splitting near he edges.
But in the past, early microbolometer have been larger in area, and needed huge lenses. Just google for uncooled thermal lenses and you can find old military equip on eBay that uses huge elements.
Germanium is heavy, and making a larger sensor will require lager and more heavy optics, that might be most important for airborne application, but gets a concern for long range handheld equip as well.
 
The following users thanked this post: Hyper_Spectral

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #37 on: March 29, 2019, 05:59:03 pm »
The pixel pitch of such microbolometers is also decreasing, but maybe not at the rate needed to use a smaller lens already developed.

Hopefully some more information is released soon. BAE certainly knows of someone integrating it for UAS, but that's literally all we know. I'll try to pull some strings through my colleagues but I doubt I'll get far.
 

Offline Vipitis

  • Frequent Contributor
  • **
  • Posts: 867
  • Country: de
  • aspiring thermal photography enthusiast
Re: High resolution thermal cores for sUAS
« Reply #38 on: May 11, 2019, 11:27:16 pm »
Thermal Expert now lists prices for their 1st and 2nd gen cores on the European website. https://www.thermalexpert.eu/deutsch/produkt%C3%BCbersicht/w%C3%A4rmebild-einbaumodule-thermal-cores/ There is also a XGA variant but it is missing the price. There is no English translation yet.
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #39 on: May 13, 2019, 03:00:31 pm »
Thermal Expert now lists prices for their 1st and 2nd gen cores on the European website. https://www.thermalexpert.eu/deutsch/produkt%C3%BCbersicht/w%C3%A4rmebild-einbaumodule-thermal-cores/ There is also a XGA variant but it is missing the price. There is no English translation yet.

Good catch! Based on their history, I'm a bit worried about noise. Hopefully its a sign that more information will be released soon
 

Offline bugi

  • Regular Contributor
  • *
  • Posts: 249
  • Country: fi
  • Hobbyist using the ultra slow and unsure method
Re: High resolution thermal cores for sUAS
« Reply #40 on: May 13, 2019, 05:50:01 pm »
Thermal Expert now lists prices for their 1st and 2nd gen cores on the European website. https://www.thermalexpert.eu/deutsch/produkt%C3%BCbersicht/w%C3%A4rmebild-einbaumodule-thermal-cores/ There is also a XGA variant but it is missing the price. There is no English translation yet.
Also, "Delivery only to tradesmen, industry, trade or research institutions."
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #41 on: May 13, 2019, 06:13:00 pm »
Thermal Expert now lists prices for their 1st and 2nd gen cores on the European website. https://www.thermalexpert.eu/deutsch/produkt%C3%BCbersicht/w%C3%A4rmebild-einbaumodule-thermal-cores/ There is also a XGA variant but it is missing the price. There is no English translation yet.
Also, "Delivery only to tradesmen, industry, trade or research institutions."

It's not hard to say you're using it for industry so I don't expect that to be a problem. Maybe it's more for preventing use by another nations military
 

Offline bugi

  • Regular Contributor
  • *
  • Posts: 249
  • Country: fi
  • Hobbyist using the ultra slow and unsure method
Re: High resolution thermal cores for sUAS
« Reply #42 on: May 13, 2019, 06:48:16 pm »
Thermal Expert now lists prices for their 1st and 2nd gen cores on the European website. https://www.thermalexpert.eu/deutsch/produkt%C3%BCbersicht/w%C3%A4rmebild-einbaumodule-thermal-cores/ There is also a XGA variant but it is missing the price. There is no English translation yet.
Also, "Delivery only to tradesmen, industry, trade or research institutions."
It's not hard to say you're using it for industry so I don't expect that to be a problem. Maybe it's more for preventing use by another nations military
If they believe me as easily, they'll believe pretty much anything else, including the stories from those that actually work for that another nation's military, pretending to do real business (especially when they actually do real business, too).
Also, if they want to be as easy about it, they could just as well just drop that whole limitation. Unless it is something about having to satisfy some legal/bureaucratic requirement which isn't really significant in practice.
And yet more, lets assume I just state "for industry", then they want more info... up to a point when a question comes that I won't be able to prove (say, they ask which company I'll be doing things for and want to confirm it from that company). Oops, I'm caught lying. Good chance I wouldn't be able to buy anything from them after that.

(The simpler solution sometimes used is to create a mini-company by oneself. I've been thinking to do that a few times, but it also has some bad sides to it.)

All theoretical for my case, though, I wouldn't be buying any of those any time soon, the TE-Q1 should be good enough for my needs for some years. I just thought to point that small limitation.
 
The following users thanked this post: Hyper_Spectral

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #43 on: May 23, 2019, 02:32:28 pm »
When asked about their competition beginning to package 1920x1200 thermal cores for sUAS, FLIR UAS Business Development Manager Brett Kanda told me "We are working on it!"

I'll keep pushing  :horse:
 
The following users thanked this post: ArsenioDev

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #44 on: October 19, 2021, 02:50:42 am »
Well, hello there. After going down one of those... rabbit holes.. I came across my own post. I'm still in the industry of drones and still a nerd at heart. Recently, we (a certain aviation regulatory agency in the USA) evaluated the use of drones in ARFF (aerial rescue and firefighting).. and it spurred me to research where the industry stands 2.5 years later. Turns out, not a lot has changed. Hindsight is a (you know what).
 
Surely others have looked into this in the past few years? Interested in keeping this conversation open!
 

Offline Fraser

  • Super Contributor
  • ***
  • Posts: 13148
  • Country: gb
Re: High resolution thermal cores for sUAS
« Reply #45 on: October 19, 2021, 10:11:08 am »
Infiray (IRAY) have made significant progress in the thermal imaging core manufacturing realm. It may be worth looking at their product range.

https://www.infiray.com/FTthermalmodule.html

https://www.infiray.com/LTthermalmodule.html

https://www.infiray.com/Micro3thermalmodule.html

And the cooled Phoenix ….

https://www.infiray.com/phoenixthermalmodule.html

Fraser
« Last Edit: October 19, 2021, 10:14:13 am by Fraser »
If I have helped you please consider a donation : https://gofund.me/c86b0a2c
 

Offline ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: High resolution thermal cores for sUAS
« Reply #46 on: October 19, 2021, 02:03:57 pm »
There's a bunch of damn interesting stuff coming out these days, heck at AUSA last week I was playing with a FIVE MEGAPIXEL COOLED MWIR CAMERA that BLEW my mind. SCD is doing some incredible stuff, but sadly they wouldn't even quote me on the price when I was there. https://youtu.be/PuT_c9kngL4
On the plus side, got a lot of contacts and in-roads at Lynred(Ulis) and they've got a brand new REALLY neat little LWIR core, an uncooled shutterless ASIC bonded job that does all the really headache inducing algos onboard and spits out easy to use color pixel data or 14 bit raw.
 

Offline Hyper_SpectralTopic starter

  • Regular Contributor
  • *
  • Posts: 86
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #47 on: October 20, 2021, 02:22:03 pm »
Infiray (IRAY) have made significant progress in the thermal imaging core manufacturing realm. It may be worth looking at their product range.

https://www.infiray.com/FTthermalmodule.html

https://www.infiray.com/LTthermalmodule.html

https://www.infiray.com/Micro3thermalmodule.html

And the cooled Phoenix ….

https://www.infiray.com/phoenixthermalmodule.html
The FT1024 looks interesting, quite the small package indeed. Interestingly enough, thermal sensitivity really isn't that big of a deal when you're looking for changes in the range of 5c+ so the Vox microbolometers are currently the ticket.

There's a bunch of damn interesting stuff coming out these days, heck at AUSA last week I was playing with a FIVE MEGAPIXEL COOLED MWIR CAMERA that BLEW my mind. SCD is doing some incredible stuff, but sadly they wouldn't even quote me on the price when I was there. https://youtu.be/PuT_c9kngL4
On the plus side, got a lot of contacts and in-roads at Lynred(Ulis) and they've got a brand new REALLY neat little LWIR core, an uncooled shutterless ASIC bonded job that does all the really headache inducing algos onboard and spits out easy to use color pixel data or 14 bit raw.
First of all, that's absolutely ridiculous. Second of all, good luck getting it out of the Israeli's hands. I can only hope this technology trickles its way down to those of us at the bottom  ;)

-

It's good to see higher res cores taking hold, but the drone industry is never going to move off the Tau 2 cored XT/XT2 from FLIR unless those high res options are readily available and I just don't see that yet. What are the real world challenges of getting our hands on one of these cores in the USA? Probably pretty tough even if we threw down $50k in cash. FLIR is really dropping the ball here.. but I suppose that DoD cash cow pays well enough for them to not care about the smaller market of non-military drone stuff.
 

Offline ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: High resolution thermal cores for sUAS
« Reply #48 on: October 20, 2021, 06:27:01 pm »
It's good to see higher res cores taking hold, but the drone industry is never going to move off the Tau 2 cored XT/XT2 from FLIR unless those high res options are readily available and I just don't see that yet. What are the real world challenges of getting our hands on one of these cores in the USA? Probably pretty tough even if we threw down $50k in cash. FLIR is really dropping the ball here.. but I suppose that DoD cash cow pays well enough for them to not care about the smaller market of non-military drone stuff.


So the hard part is getting a MOQ high enough that they'll actually talk to you, most of the time they are not interested in selling one-two offs. As for who to talk to in the USA, just talk to the SCD USA offices, they're the point of contact and local distributor. Same with Lynred, they have a USA office and distributor.
 

Offline thermalengineer

  • Contributor
  • !
  • Posts: 46
  • Country: us
Re: High resolution thermal cores for sUAS
« Reply #49 on: September 09, 2022, 12:54:03 am »
It's good to see high resolution thermal cores using in body temperature measurement. I do look forward to exploring more than 1920*1080 thermal products.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf