DIY Focus Stacking for Macro Photography

[mawyatt]

Focus stacking in macro photography involves techniques to deal with the very shallow Depth of Focus. For example when capturing images around 5X magnification the DoF can be as shallow as 0.028mm (28 microns)! Techniques and software have been developed to capture multiple images at different focus positions, either by changing the lens focus or moving the camera/lens or subject, then "stacking" them together to render an image with much larger in focus areas.

At high magnifications moving the camera/lens is usually accomplished by a precision linear rails, either manual or motor driven. Commercial motor driven focus rails from Wemacro, Stackshot and others are very good and automated. When the desire is high resolution (high magnification) and a large Field of View, then focus stacking alone can't cover the intended area, so a type of stacking that involves a macro "Panorama" of multiple stacking images is required, often called Stack and Stitch, or S&S. There isn't any inexpensive available commercial equipment for S&S, so this opportunity begs for the DIY approach. Also if one wants to venture beyond what the standard commercial focus rails offer in performance, a DIY approach become attractive.

Photomacrography is a great site for those interesting in macro photography, including DIY type efforts from custom developed lens to complete rig setups.

https://www.photomacrography.net/forum/ 

Here's a few examples that I've been involved with.

Some videos.

https://www.photomacrography.net/forum/viewtopic.php?t=39392&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39396&highlight=


Custom developed S&S system based around Raspberry Pi Development.

https://www.photomacrography.net/forum/viewtopic.php?t=38511&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=38548&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=38604&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=38512&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39318&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39428&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39755&highlight=

Stepper Motor Controller Development.

https://www.photomacrography.net/forum/viewtopic.php?t=39473&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39689&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39428&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=39756&highlight=

Piezo Electric Stage Development.

https://www.photomacrography.net/forum/viewtopic.php?t=40679&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=40681&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=40682&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=40510&highlight=

LED Strobe Light Development.

https://www.photomacrography.net/forum/viewtopic.php?t=40999&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=41353&highlight=

https://www.photomacrography.net/forum/viewtopic.php?t=41464&highlight=

Chip image source for some older renderings, latest results are still proprietary. Download images for full resolution, the last image is a sliver (cracked off during attempted wafer split) of a very old abused wafer that was a test image for the Fully Automated S&S System mentioned, its ~29,000 by 22,000 pixels.

http://img.gg/taIZ99M

Few older examples.




Hopefully some folks will find this interesting.

Best,

Mike

[RoGeorge]

An older focus stacking related idea of mine, that I will probably never pursue, so just dropping it here:
- use an electrically controlled focus lens (based on electrowetting, so the focus adjusting to be faster than for a mechanically controlled focus), either DYI (https://youtu.be/tOEyxavTiGw) or ready made (https://www.corning.com/worldwide/en/innovation/corning-emerging-innovations/corning-varioptic-lenses.html).  If the adjustable lens is not fast, maybe using a mechanically vibrated ocular.
- high speed camera taking snapshots while the focus plane is changed with the controlled lens
- FPGA implementing a live focus stacking algorithm

Shortly said, an automated focus stacking camera/microscope.  The result should be a live image (movie) with increased depth of field.

In cases where direct eye observation is required (optically), a fast LCD screen can be put in the optical path of the microscope, to dim out those pixels identified by the stacking algorithm as out of focus.  The LCD will have only an on/off level for each pixel (each pixel driven by the focus stacking algorithm), so the eye could integrate over time only the sharp pixels for each focal plane.   

[DrG]

This is news to me and thanks. It sounds like HDR but for focus. Instead of stacking different exposures, which are easy to shoot using bracketing, the same idea is done with focus? That is making my head hurt

Bookmarked this thread and that site -

[SilverSolder]

You can do focus stacks with your normal SLR (or at least variable focus!) camera by taking several pictures (min 2 of course), loading them into Photoshop, and run the Photomerge function.  Easy as!

Of course, things get harder and harder the shallower the depth of field.  The mechanics of holding the camera becomes the main problem at some point.

[DrG]

You can do focus stacks with your normal SLR (or at least variable focus!) camera by taking several pictures (min 2 of course), loading them into Photoshop, and run the Photomerge function.  Easy as!

Of course, things get harder and harder the shallower the depth of field.  The mechanics of holding the camera becomes the main problem at some point.

By bracketing f-stop? or am I just not getting it?

[RoGeorge]

Focus stacking in macro photography
...
Piezo Electric Stage Development.
...
Hopefully some folks will find this interesting.

Took the time to browse a few of the links, and wow!  Very nice projects.   
And the die photos, after focus stacking, are amazing.   

About the piezo stage project, does that model need pulses to move, similar with a stepper, or is it some sort of voltage to position actuator?  From your project logs it seems to me that your piezo stage is rather a piezo actuator than a piezo stepper.  If so, did you happened to experiment with a stepper piezostage, too?

I'm asking because one of my way too many bucket list projects is to experiment with a DIY nanostage (just a lump of piezo material for each axis, with an inertial weight attached at one side, and the free moving stage attached at the other side of the piezo lump, moving just by friction and inertia and a sawtooth waveform driving the piezo, no Archimedes screws, only free moving axis, similar with the idea from the inertial motor), and I will be very curious to learn how well a DIY piezo stage can perform. 

[SilverSolder]

You can do focus stacks with your normal SLR (or at least variable focus!) camera by taking several pictures (min 2 of course), loading them into Photoshop, and run the Photomerge function.  Easy as!

Of course, things get harder and harder the shallower the depth of field.  The mechanics of holding the camera becomes the main problem at some point.

By bracketing f-stop? or am I just not getting it?

By bracketing the focus - imagine a manual focus lens, you turn the focus ring a tiny bit between each shot, so different parts of the 3d subject come in focus in each one.

What Photoshop (and other applications) do with those shots is to pick the parts of each image with the sharpest focus, and weld them all together into one image, where everything is sharply focused! 

[todd_fuller]

Seems like the now-defunct Lytro camera would be perfect for this application. It was able to change field of view in software after the fact. Pretty interesting technology. Their real camera was $$$ though.

[Conrad Hoffman]

I resort to stacking quite often. Some of the newer digitals have it built in. My Z6 does, but unfortunately the lenses I like to use are manual focus so I have to do it manually. I've tried focusing with the lens, focusing on a rail and focusing just the body, with the lens in a fixed position. Best results have been focusing the lens. The images can be put together with the free CombineZP, but it crashes with a lot of large images. Picolay is another free one, or there's the standard Helicon Focus and maybe a couple others.

[Thomas]

I used Helicon Remote and Helicon Focus with my SLR for focus stacking. Interesting
https://www.heliconsoft.com/heliconsoft-products/helicon-remote/
https://www.heliconsoft.com/heliconsoft-products/helicon-focus/
Below is a picture of my watch, stacked from 28 images.

[SilverSolder]

I resort to stacking quite often. Some of the newer digitals have it built in. My Z6 does, but unfortunately the lenses I like to use are manual focus so I have to do it manually. I've tried focusing with the lens, focusing on a rail and focusing just the body, with the lens in a fixed position. Best results have been focusing the lens. The images can be put together with the free CombineZP, but it crashes with a lot of large images. Picolay is another free one, or there's the standard Helicon Focus and maybe a couple others.

Picolay has the best alignment algo of all the ones I've tried, far superior to Photoshop which tends to give up on low contrast images.

[DrG]

I resort to stacking quite often. Some of the newer digitals have it built in. My Z6 does, but unfortunately the lenses I like to use are manual focus so I have to do it manually. I've tried focusing with the lens, focusing on a rail and focusing just the body, with the lens in a fixed position. Best results have been focusing the lens. The images can be put together with the free CombineZP, but it crashes with a lot of large images. Picolay is another free one, or there's the standard Helicon Focus and maybe a couple others.

Just doing some reading about this. Do you make a distinction between focus stacking and focus bracketing? https://www.mu-43.com/threads/what-is-the-difference-between-focus-stacking-and-focus-bracketing.81617/

My current DSLR does not have focus bracketing. Some years ago, I played around with the Canon SDK and I wondered if it could be used to control the focus stepper. Looking around this morning, apparently the answer is yes - e.g., DSLR Bracketer http://milosparipovic.com/#/dev and now I am looking into DSLR Controller app for android https://dslrcontroller.com/ to control my EOS 200D.

*drat* there is a couple hundred hours of very cool stuff here curse you people 

[Conrad Hoffman]

I resort to stacking quite often. Some of the newer digitals have it built in. My Z6 does, but unfortunately the lenses I like to use are manual focus so I have to do it manually. I've tried focusing with the lens, focusing on a rail and focusing just the body, with the lens in a fixed position. Best results have been focusing the lens. The images can be put together with the free CombineZP, but it crashes with a lot of large images. Picolay is another free one, or there's the standard Helicon Focus and maybe a couple others.

Just doing some reading about this. Do you make a distinction between focus stacking and focus bracketing? https://www.mu-43.com/threads/what-is-the-difference-between-focus-stacking-and-focus-bracketing.81617/

My current DSLR does not have focus bracketing. Some years ago, I played around with the Canon SDK and I wondered if it could be used to control the focus stepper. Looking around this morning, apparently the answer is yes - e.g., DSLR Bracketer http://milosparipovic.com/#/dev and now I am looking into DSLR Controller app for android https://dslrcontroller.com/ to control my EOS 200D.

*drat* there is a couple hundred hours of very cool stuff here curse you people 

I've never heard things described as in that link. Taking multiple images at different focus points and combining them to achieve "impossible" depth of field is AFAIK, focus stacking. Never heard focus bracketing before, but I don't get out much. ;-)

Once you start playing with this, you'll probably find it addictive.

[SilverSolder]

To attempt to define some terms...

Exposure bracketing is basically shooting extra pictures exposed above and/or below the starting setting, to make sure you get at least one correctly exposed image (and/or use it for HDR photos).

Focus bracketing sounds like the same idea applied to focusing...  to be convenient, it really needs to be built in to the camera.

Focus stacking is a deliberate, intentional use of focus for a series of single shots, that you intend to use to create a greater whole.

[mawyatt]

Focus stacking in macro photography
...
Piezo Electric Stage Development.
...
Hopefully some folks will find this interesting.

Took the time to browse a few of the links, and wow!  Very nice projects.   
And the die photos, after focus stacking, are amazing.   

About the piezo stage project, does that model need pulses to move, similar with a stepper, or is it some sort of voltage to position actuator?  From your project logs it seems to me that your piezo stage is rather a piezo actuator than a piezo stepper.  If so, did you happened to experiment with a stepper piezostage, too?

I'm asking because one of my way too many bucket list projects is to experiment with a DIY nanostage (just a lump of piezo material for each axis, with an inertial weight attached at one side, and the free moving stage attached at the other side of the piezo lump, moving just by friction and inertia and a sawtooth waveform driving the piezo, no Archimedes screws, only free moving axis, similar with the idea from the inertial motor), and I will be very curious to learn how well a DIY piezo stage can perform.

The piezo stages are operated Continuous Closed Loop and use a strain gauge in a bridge configuration for position sensing. The piezo elements are continuous voltage driven by a low noise high voltage amplifer with the stain gauge as feedback and command position from a 16 bit DAC controlled by a RPi. See details at PM. Note linearity and the 750nm range! Think this was with 4nm DAC increment resolution.

https://www.photomacrography.net/forum/viewtopic.php?f=25&t=40510&hilit=Piezo+Stage

Hope this helps,

Best,

[mawyatt]

A few answers to some questions.

1) Generally in-camera focus stacking doesn't work well when the magnification goes beyond ~ 2X, it's better to move the camera/lens and/or subject. Perspective issues become a problem. Check out PM for more details.

2) Often special lens are used in stacking other than the standard camera "macro lens". Reversed lenses, stacked lenses, inspection lenses, microscope lenses, reproduction lenses, and bellows lenses for example. See PM for more details.

3) Special telecentric lenses, or near telecentric, are often used for serious Stack and Stitch (S&S) work. S&S is like a macro panaorama where multiple focus stacks are stitched together to render a much larger image. We've done 29000 by 23000 pixels chip images, and soon much larger!!

4) Lighting becomes more difficult as the magnification increases. Many use flash/strobes to help with vibration issues. Highly diffused light is generally required, many times with multiple layers of diffusion.

Anyway, lots of things to deal with, the mentioned PM site is devoted to macro imaging and a great resource.

https://www.photomacrography.net/index.htm

Here's a very low res (original is ~40MB JPEG) chip image example. It's a Direct Digital to Antenna (DD2A) chip (patent 7903016) from long ago, process was IBM Silicon Germanium 9HP 90nm BiCMOS.

Best,

[schmitt trigger]

I am also subscribing to interesting thread.

The images are beyond awesome.

[DrG]

A few answers to some questions.
/----/
Best,

Yeah, I have been doing some due diligence and reading on the wide range of the subject matter. It is intriguing.

I also went to the site and looked around and read the posting policies and so on. Thanks for making me aware of it.

I don't have the equipment to do it properly right now but it is on my radar. I may try an attempt with what I have. Believe it or not, one consideration is all the other projects ongoing and on the list.

[mawyatt]

I am also subscribing to interesting thread.

The images are beyond awesome.

Thanks.

I'll post more low resolution images and details if you and others want (many images I can't show though). Don't want to take up space unless folks are interested.

The history behind all this started way back in ~2003, when we wanted to get images of the chips we designed but wanted something other than the usual microscope type chip image.

Best,

[RoGeorge]

Yes, of course, please post more pics!

Also, in general there is a lack of information about how a certain transistor, stage or other schematic structure looks like in silicon (e.g. people asking to read a die photo similar with reading a schematic:  https://www.eevblog.com/forum/projects/die-shot-features-help/ ), so any other explanations or annotations will be appreciated.

[Dave]

Incredibly interesting thread. The pictures look surreal, almost like a fancy 3D render.

Speaking of DOF, would it not be possible to make it a bit deeper by reducing the aperture and extending the exposure time? Or are you already at that limit with the 28um example?
I understand that it would never achieve the same effect as focus stacking, but it would at least reduce the number of images required, making the precision/mechanical tolerances of the automated jig a bit looser.

[mawyatt]

Focus stacking in macro photography
...
Piezo Electric Stage Development.
...
Hopefully some folks will find this interesting.

Took the time to browse a few of the links, and wow!  Very nice projects.   
And the die photos, after focus stacking, are amazing.   

About the piezo stage project, does that model need pulses to move, similar with a stepper, or is it some sort of voltage to position actuator?  From your project logs it seems to me that your piezo stage is rather a piezo actuator than a piezo stepper.  If so, did you happened to experiment with a stepper piezostage, too?

I'm asking because one of my way too many bucket list projects is to experiment with a DIY nanostage (just a lump of piezo material for each axis, with an inertial weight attached at one side, and the free moving stage attached at the other side of the piezo lump, moving just by friction and inertia and a sawtooth waveform driving the piezo, no Archimedes screws, only free moving axis, similar with the idea from the inertial motor), and I will be very curious to learn how well a DIY piezo stage can perform.

To elaborate more on the Piezo Stage Technology. This is the same technology used in the semiconductor fabrication processes and involves a clever arrangement of precise cuts (laser) in a solid chunck of Stainless Steel, these cuts create a special mechanical type bearing called a Flexure. A long piezo element comprised of many smaller piezo sections is mounted into the SS structure by means of compression, under applied voltage the piezo elements expand and "push" against the SS structure to create movement. See more details on the PI site.

https://www.pi-usa.us/en/products/piezo-flexure-nanopositioners/?onl_goog_piezo_pos&gclid=Cj0KCQjw6PD3BRDPARIsAN8pHuENs--371LYn7jUgtgRU4uIro2KceY6ev7YfPrrAHESquNtg7WIUHwaArmJEALw_wcB

If you decide to go ahead with your DIY project, we've developed a couple DIY closed loop controllers for the PI stages (P601 and P603) that work with the Raspberry Pi. These PI stages sometimes show up on eBay for a reasonable price.

https://www.photomacrography.net/forum/viewtopic.php?f=25&t=40123&p=252513&hilit=VCM#p252513

Another interesting concept is using an ordinary speaker as a Voice Coil Motor, you can see some notes here.

https://www.photomacrography.net/forum/viewtopic.php?f=25&t=40123&p=252513&hilit=VCM#p252513

Best,

[mawyatt]

Incredibly interesting thread. The pictures look surreal, almost like a fancy 3D render.

Speaking of DOF, would it not be possible to make it a bit deeper by reducing the aperture and extending the exposure time? Or are you already at that limit with the 28um example?
I understand that it would never achieve the same effect as focus stacking, but it would at least reduce the number of images required, making the precision/mechanical tolerances of the automated jig a bit looser.

Thanks.

The effective aperture (EA) generally follows as Lens Aperture, or LA, as LM(1+M), where lens aperture is the normal "reading" on most cameras (Nikon shows the EA, not LA). When you get an EA that's beyond about F20 or so, diffraction begins to eat your lunch if your images are viewed at larger sizes. Normal lenses this doesn't matter, since shooting a bird at 100 feet or a portrait at 10 feet, M is very small, so M+1 ~ 1 and EA ~ LA. However when shooting macro this becomes a problem, example shooting at 1X the EA is twice the LA, so one would begin to see diffraction effects around LA of just f11 (in very high quality work this is below f8!!). So "stopping down" the aperture to increase the depth of focus has a limited range.

Some of the latest software tools attempt to reverse this situation, by using a de-convolution to partially remove some of the diffraction induced blurring. I haven't used any of the tools yet, so can't comment reliably, but others have shown good results.

Some the "benchmark" microscope objectives from Mitutoyo (5X, 10X, 20X) have an EA of ~18, so this is a good indication of where things begin to suffer from diffraction effects.

Depth of Focus = Lambda/(NA^2), where lambda is wavelength (550nm for green), NA is numerical aperture or M/(2*EA)

Best, 

[mawyatt]

Here's a few more very low resolution images that I can show. All of these are from long ago, so allowed to show. Note damage on 1st image

Best,

[mawyatt]

And a couple more.

The colorful chip is a new type (~8 years ago) ADC from USC in TSMC 65nm CMOS, called non-uniform sampling. It basically has a varying output data rate based upon the waveform characteristics, so it quantizes in amplitude and time and creates it's own antialiasing filter.

The other chip is an experimental Iridium Receiver in IBM SiGe 8WL 130nm BiCMOS.

Best,