Simple image stacking for super resolution and noise reduction

[frenky]

I have been playing with this free software for Astrophotography: http://www.astrokraai.nl/software/latest.php
It does realy great auto stacking and upscale of multiple images.

I took 10 handheld thermal images of the same wall:


And then stacked and upscaled it in AutoStakkert!2:


Results spreak for itself.

[eKretz]

I see pretty good noise reduction, but not much difference in apparent resolution. Thanks for pointing it out though, will investigate further.

[Rerouter]

If the camera position never changes, you can't really draw much more resolution out via just image stacking, but it did seem to help with the noise.

Microsoft at one point had an awesome photo stitching platform that worked well for stuff like this "photosynth", it even corrected out the parrelax this would normally cause,

Though seems out of focus?

[frenky]

Camera position was different in each of 10 shots.
This is needed to capture sub pixel data.
Original sensor resolution is 384x288 but I upscaled orig image to better show comparison.
I used Lancoz upscale which is pretty good in upscaling so actual resolution increase is not that apparent in stacked image.

Will redo experiment with a subject having more details.

[Rerouter]

I should warn you, you only need the change your position by fractions of a pixel each time, Ideally in both X and Y, much more and you will add blurriness due to parallax, for astrophotography its rarely an issue as the distances and rotation rates rarely cause issue unless you take extremely large image sets.

[Ultrapurple]

I've worked with superresolution, stacking and panoramas in thermal imaging. The results can be remarkably good.

For simple noise reduction via image stacking, RegiStax is good, as also is AutoStakkert! (the exclamation point is part of the name). Both can also do some resolution enhancement. Below is an image made from ~70 images stacked in AutoStakkert! this morning (the subject is a tiny electric blanket; I was monitoring it in case it overheated). The red outlines show areas that were superimposed form a randomly selected unprocessed frame. There's a definite reduction in noise. The camera was on a tripod so superresolution wasn't useful.

Superresolution can be very effective. ThermViewer (for ThermApp and XTHERM cameras) has it built in, as do many other apps. I've experimented with several ways of employing superresolution on video recordings and the best I've seen so far is Video Enhancer 2, which has recently become a free program (you need to make up an email address to get a registration code; the address doesn't have to work). An example of superresolution on a thermal video is here.

Panoramas are another way of increasing effective resolution. Again, there are many packages out there but I find the free Microsoft Image Composite Editor works well. Here's an 18 megapixel sample (click to see original and more info):




Whatever you do, have fun experimenting - and please report back via EEVblog so we can all benefit from your research.

[frenky]

Tnx for encouraging words. 
I find your images and videos really fascinating and I really appreciate you sharing your knowledge.

[Vipitis]

Registax allows for drizzling which makes us of the subject rotating over time (like a 5 hour astrophotography capture session) and superresolution from there. This could help with stacking efforts by just rolling the camera a little to the left and right over moving it x,y on a parralal plane to the subject.

Not sure if AutoStackerrt! Has something similar as well.

I wanted to play with stacking, wavelets and sharpening as well for panoramic shots. But I have to downsize my frames to 1:1 pixel scale for proper processing but I haven't done the imagemagick for it yet.

[Rerouter]

Taking your first really large Panorama is one of the quickest ways to learn where the imperfections in your lenses lie.

Got board one afternoon, so set up a little X/Y stepper rig to take something like 1400 photos on a 400mm zoom lense that was then stitched into one big pano, but sadly i had to go and mask off every bloody photo as there was distortion at the corners with such a high zoom.

[Ultrapurple]

If you've got time to kill, superb results can be obtained by combining the techniques if your subject is suitable (ie, still):

• video the scene, panning in a zigzag fashion, apply superresolution, panorama process
• using a tripod, take a series of 10-sec videos, methodically moving the camera in a pan-and-scan between runs, then use RegiStax or similar to create individual de-noised images which can then be panorama processed
• as above but jiggle the tripod a bit to simulate hand shake so you can run superresolution before stacking and pano processing
  

All good, clean, time-consuming fun!

Sometimes you just have to trust to luck, though. Like when you only have access to a place for a limited time and you just have to hope you've captured everything you need, in good enough quality to work with later. An example is the image below (click to embiggen) of Abu Simbel in Egypt.


Abu Simbel (Egypt) thermal panorama

I didn't expect to be let into Egypt with a thermal camera, let alone be unmolested when I started waving it around. In fact, the only place there was the remotest problem was atop the Aswan Dam, where the armed guards wanted to assure themselves I wasn't taking video, which is banned. (Stills are fine, but I didn't want to have to explain myself to them...)

[Ultrapurple]

... there was distortion at the corners with such a high zoom.

Software like Microsoft Image Composite Editor corrects that for you automatically provided you have a reasonable (10-20%) overlap between images.

[ebastler]

@all, when you talk about "super resolution", do you mean just interpolating on a grid smaller than the sensor pixel size, or actually resolving adjacent details which are less than two pixel sizes apart?

If it's the latter, do you know how the image processing software achieves that? (Do they document their approaches/algorithms?) Is there a deconvolution step involved?

[Ultrapurple]

Superresolution in the context we use it refers to sub-pixel rendering, ie scraping out more actual resolution than the pixel count would have you believe. There's a decent description on Wikipedia. You'll also find a user-friendly description of how it works in practice on the Infognition site. To quote the key information from the latter,

"Super-resolution (SR) works effectively when several low resolution images contain slightly different perspectives of the same object. Then total information about the object exceeds information from any single frame."

[Rerouter]

super-resolution in practice relies on assuming that each pixel is captured is just a single point value at the center of the position, generally a vector to make the math easier, you match the rotation and translation between images, and add those new points from each additional image.

At the end you run whatever filtering process you want and use all the points you have amassed to paint the new image, generally at higher resolutions. this allows for times where you doing non evenly divisible scaling, you could treat it like a metal grain structure, you overlay a grid, fill in the points that you have data for, filter pixels that have multiple points, and for data points that have nothing, expand from adjacent neighbors.

[Tom45]

I've been using a software program called PhotoAcute for many years with good results.

  https://www.photoacute.com/studio/order.html

Unfortunately, they have suspended development and it is no longer sold although it seems that you can still download it and then send an email to get a code to take it out of demo mode, (see the forum link on their order page). There is a chance that they will come out with a new version.

In the meantime, the various alternatives mentioned in this thread are good to know.

Tom

[rhb]

Solve this problem:

Ax=y

where x & y are 2D matrices.

For y, use an image with a number of randomly chosen samples deleted.  Repeat the sampling process N times  That's the multipixel version of the single pixel camera.  You're trading time and processing for spatial resolution.

One A matrix is the discrete Fourier basis  evaluated evaluated at the randomly kept pixels.  Or solve for the coefficients of a JPEG, but use an L1 solver in either case.  GLPK is my choice.  There are arbitrarily large and complex examples of an A matrix as you might like that it solves easily.

It would be interesting to compare what a traditional Wiener processing stream is able to do.

[lolli-us]

For those of us who use Adobe Photoshop there's a free and effective action file to do superresolution by photographer Tony Northrup (link to the action in the below video)



I've used it extensively for normal photos and it's quite effective. It's also convienient as one can build it into a more extensive processing workflow.

My two cents...