Attempt for a multi-spectral night vision system (Vis/Nir and LWir)...

[Lambda]

I'd like to present you a little tinkering following my little trials in thermal imaging based on a camera I cobbled together from a second-hand camera core (see https://www.eevblog.com/forum/thermal-imaging/my-first-little-diy-thermal-camera-project/ ):

So:

- A thermal camera (LWIR: 8-14 um) with a 15mm/1.0 Germanium lens + image capture by the small monocular associated with the camera, through a 2 francs six cents webcam (no Grabber at hand...)

and

- My usual IMX290 MM (CMOS) based system but equipped with a small 8mm/1.3 CCTV lens, instead of my big LOMO 160/2 lens. In order to be able to obtain in 2 parallel SC sessions simultaneously, by playing with the zooms, 2 images with about the same scale.
2 SharpCap (software dedicated to astro applications: live stacking, real time video...) sessions, and a window of one of the sessions made translucent by means of a small software found on the net "peek through", for example. and overlaying by hand a translucent window (the one in LWIR for example) on the main one (Vis/Nir).
Video tests (sorry for the second one, I'm in the field of the cam in my socks.... I just had the idea suddenly during the night... 😃 couldn't wait to try it!), and splash of cold and hot water on the floor, to visualize well the thermal side of the image....

As the superposition of the two videos is not very clear, we can see the difference between the Vis/Ir alone, in a part of the image, and the Vis/Ir + LWIR superposition in the rest of the image..... nice to realize the effects, highlighting the tehrmal gradients, the vegetation and some parts of the ground heated during the day.....

In short, a COTI or Fusion system (image intensifier (tubes) + thermal in military application) "poor man's" civilian hobbyist version... 😃 😃
COTI/Fusion systems have been around for 10-20 years, let's say in the military, and you can find them on the small FLIR/FLUKE/SEEK thermal cameras used in the civilian sector for thermal diagnostics in microelectronics, building, mechanics.....
https://www.google.com/search...

The interest is to make the two technologies complementary on a single video stream in real time:

- In Vis/Nir in Night Vision, good resolution, but we are still dependent on a minimum of residual light (stars, moon, PL, etc...) and deep shadows are more difficult to scan, and for any immobile object visually blending into the landscape, even in intensified mode, it is sometimes difficult to detect,....

- In LWIR, if it's above absolute zero (-273.15°C) it's glowing... 😃, and between 8um-14um, we catch everything around us, in normal life, at room temperature, between 0°C and like 100°C without worrying (This is extremely and falsely simplified, (cf Planck's law, Stefan...) but just to say....), On the other hand, it works rather in low resolution compared to Vis/Nir applications in general, the visual renderings are confusing (we are very far from our "original/physiological" visual domain, the notions of emissivity take precedence over the usual reflectivity....), and then through certain "transparent" materials for our eyes as for ordinary glass or plexiglass, the thermal does not pass, these materials are opaque.... Water/mist (rain) can be a problem too....

In short, the idea is to merge the best of both spectral worlds to make "multi-spectral" night vision ....
here the manipulation, with the camera in piggy back on my usual device (with the LOMO here, but for my tests, it was replaced by the small CCTV 8mm/1.3 lens)
And the two test videos in a row....... captured with VLC... The fluidity is better in real life. to accelerate and walk around in it, to concentrate on for example the vegetal rendering heated the day before, and which jumps to the eyes at night, to compare to the Vis alone, or some parts of heated grounds or buildings, or air exhaust chimney outlets co....




Thank you for your reading patience!!!

to be continued! Lot of space for improvement: managing parallaxe for close scenary, adjusting contrast and gain for both camera (SC is good for that), adding a touch of color for the translucent window for clarifying the image in general....

Regards.
Lambda

[Vipitis]

I like the idea and execution.

It's something I have thought about as well. But my projects have been pretty much non moving for a long while. What I was thinking about was using a Ge window as a beam splitter. Since it does reflect the visible and nIR light really well and let's LWIR through. You could but the visible camera 90° off axis with the window at 45° and try to get the same optical axis. I do have a watec 902h ultimate and a few C/CS mount lenses that I will try to setup in this configuration to my thermal camera once some cables get to me. But I won't be able to capture them both at the same time as I am limited by the number of recorders I have.

You ended up overlaying the image, but my end goal was to get a color image by mixing channels. Astronomers capture narrow band images and mix them down to RGB. They usually do this in chromatic order. Meaning the shortest wavelength is blue and the longest red. If they have a third or more bands, they use green or really complicated pixel math.

My hope was to setup 2 or 3 cameras with the same optical path(to avoid parallax) using some more beam splitters and filters. Put LWIR to Red, nIR(800nm cut) to Green and visible to Blue. Hope to get a really crazy RGB image back and compare some things in the world.

I am sure there are ways to get it done using hot mirrors or special prisms, but I don't have SWIR or MWIR, so there is a huge gap.

Maybe UV is an option as well, I have seen people do UV only, vis, nIR only back to RGB and it looks like a really trippy normal visible image.

[Ultrapurple]

A very interesting setup 

I hope to be able to do some multi-spectral experiments before long - time is my main constraint.

I have a nice LWIR/MWIR beamsplitter that I hope to use at 45° so I can overlay LWIR and MWIR images without parallax error. I have only recently obtained reliable MWIR capability so I have yet to set this up.



Next on the list will be SWIR (1.8...0.9µm), NIR, visible and UV. I have (somewhere) a hot mirror that should let me split SWIR (and NIR) from visible and UV, again so I can co-locate.

I already have a camera and lens setup that will capture NIR / vis / UV simply by changing filters, though even my experiments with that have also been time-limited.



I am not aware of any practical way of adapting these principles to extend the sensitivity as far as my X-Ray machines - although I have achieved interesting results simply combining visible with X-Ray via a different method.



One day I shall retire and have time to play with all my toys...

[Ultrapurple]

I did a bit more on multi-spectral X-Rays here.

[Lambda]

Hello

Well, I'm still working on my subject and I've made some changes:

- LWIR lens with a longer focal length (about 40mm/1.0);
- Tinkering with a relay lens in place of the basic webcam lens, using the image delivered by the thermal camera viewer, with a built-in zoom, to be able to have more latitude in image scale adjustment, combining this with the digital zoom offered by SharpCap. This will allow the Vis/Nir image to be correctly superimposed on the LWIR image;
- Replaced my brave LOMO160/2 for the 290MM camera with a 100/2 from an old Russian image intensifier (a T3C-2), Helios lens, in order to reduce a bit the different image magnification gap between the thermal camera and the 290. A nice thing is that this lens is better chromatically corrected for long wavelength Vis and Nir, as it is designed to be used with the intensifier with a Nir source. Compared to the LOMO, the image delivered is more crisp.
- Replaced my Watec120n+ which unfortunately died with a small monochrome Runcam Night Eagle 2 pro camera: I kept the WATEC housing to insert the tiny Runcam body and take advantage of the Computar 6mm/0.8 lens already mounted on the housing.

[See figure below]
(sorry, annotations are in french...  But easy to understand)

Another small test video targeting a row of air-conditioning vents located on the roof of a neighbouring building housing electrical installations for the city network: about 100m from my position.

Capture in VLC, as usual, much smoother in reality (100 ms of unit pose for the 290, and 30Hz of refresh for the thermal camera)



Concentrate between 15sec and 2min 17sec approximately. A little bit of work to start up with VLC....

I lack resolution on the thermal camera (160x120), I need to find a good thermal lens with a longer focal length (70mm or more for example) to have to zoom less in the second SC session managing the webcam. The rendering would be a bit finer, resolved. The centering latitude of the thermal image in relation to the Vis/Nir image would be greater.
I'm still working on the contrasts too.... but it's taking shape...

You can see 2 of the vents (cubic structures with the exhaust venetian) that work by evacuating hot air, while keeping enough resolution thanks to the Vis/Nir layer. You can also see towards the end some hotter wall areas and also behind the trees against the front of the building, some hot areas.

So the video covers the 0.4-1 um band and the 8-14 um band roughly simultaneously....

Step by step i try to increase the monitoring distance range of this guizmo....

To be continued!

à+!

Lambda

[Lambda]

Hello again.

New attempt on a more interesting or at least fun, subject having landed on a football field, 100m ahead from my balcony: two choppers (Police and i guess fire brigade or civilian safety (?)). Some training? i do not know... a school footbal team was playing just in the neighborhood.

https://youtu.be/EVj6x51XipY

It's a bit long, sorry, but I wanted to make a somewhat exhaustive sequence: two separate display windows, LWIIR on one side and Vis/Nir on the other. And then "merge" the two windows, by hand, to obtain the desired "multispectral" vision..... At the end of the video, when one of the choppers takes off, you can see the distortion/stroboscopic effect due to the CMOS scanning system of the 290, in "shutting roller" mode, with the tiny exposure times used for the IMX290MM CMOS sensor from the Vis/Nir camera (of the order of ms, I didn't check, just quickly adjusting this parameter on the fly before the "show" starts....).

You can see the resolution of the signal provided by the 290, without losing too much information, at the thermal level (one of the choppers had landed a good 15 minutes before the second one. I let you guess which one... )

I have to seriously work on the general ergonomy of this system wich is really crude to manipulate/prepare before performing an observing session "on the fly".

Regards.

Lambda

[DaJMasta]

Some neat work, there may be a bit of software that could give you some extra flexibility when compositing - OBS Studio.  It's designed for live video work, but can capture directly from windows on a desktop, and you can transform, adjust opacity, and apply filters to colorize for each video source as needed, so you may be able to capture your video, colorize it for whatever you think is useful, and then overlay it to output/record all in one workspace and not have to capture the rest of the windows desktop.  What's more, along with colorization, the filters you can apply also let you do contrast/brightness/gamma adjustments to try and get as much detail as possible from the source video, even if not controllable by the camera/lens sufficiently.

Haven't done any overlays with it, but I've been using this software for my multispectral capture and have been happy with the adjustment options, ease of use, and stability of it.