Fake background blur on camera-phones

[brucehoult]

The issue with the fake DOF is that the camera often doesn't know the actual distances involved, so in a portrait, the face will be in focus, but the ears will be blurred, and will be blurred by the same amount as the wall behind them.

There was one good example on a manufacturers website where the person was stood on a bridge. The railings (which were probably a few meters away from the camera) were blurred, but the background in between the railings was pin sharp, but the [same] background above the railings was blurred!

The actual TOTL recent phones use multiple lenses with a (small!) displacement between them, which is presumably enough to get some depth information.

iPhone 12, 13, 14 also have LiDAR to directly get depth information in different parts of the image. Samsung's Galaxy S10 and S20 also have LiDAR, but apparently it's a cheaper implementation that doesn't work as well as Apple's and they're dropping it in newer models.

[Someone]

The issue with the fake DOF is that the camera often doesn't know the actual distances involved, so in a portrait, the face will be in focus, but the ears will be blurred, and will be blurred by the same amount as the wall behind them.

There was one good example on a manufacturers website where the person was stood on a bridge. The railings (which were probably a few meters away from the camera) were blurred, but the background in between the railings was pin sharp, but the [same] background above the railings was blurred!

The actual TOTL recent phones use multiple lenses with a (small!) displacement between them, which is presumably enough to get some depth information.

The other trick being that n cameras capturing the same scene from different positions can be combined to produce something approaching a lens of the same area, incoherent synthetic aperture. Add a couple of complementary methods on top of each other and the synthetic bokeh can be good enough for the unwashed masses.

[Mechatrommer]

i was the owner of Samsung KZoom, that was the closest thing to real bokeh and telephoto at smallest and lightest phone form factor that fits in a pocket. but since i need 2 SIM slots now, which is not available in KZoom, i have to buy newer phone, but sadly no more replacement (younger version) like KZoom, i guess not much sale and not trending. so i have to buy 3-pin-holes smartphone now. and then i found SONY DSC-QX10 (WHITE), wifi attachable to pin-hole smartphone. if i want real zoom and light for personal use (digital zoom is another older toy in existance in hunglow P&S camera and smartphone) thats the one i will grab. but sadly the DSC-QX10 type also extinct.

[magic]

Depth of field is determined by the inverse of the focal length squared, the distance to the subject (the plane of sharp focus) the aperture of the lens and an arbitrarily agreed upon circle-of-confusion or spot on the image that is not in focus but is not so blurry that it does not appear sharp in the final image ie sharp enough to be called sharp.

Another way of looking at depth of field is that it depends directly on the "film" or "sensor" size.
A "full-frame" DSLR sensor is 24 by 36 mm, just like a frame in a 35 mm film SLR, and will have a shallower depth of field than, say, 4.2 by 5.6 mm (IPhone 14) at a given f-number.
The exposure depends on that f number, the exposure time, and the actual speed (ASA/ISO) of the film or sensor.
When I started using an 8 x 10 inch camera to photograph flowers, I immediately noticed the shallow depth of field at a given f-number.

Another? This is the absolutely common way of looking at DoF which has already been posted by the guy above. It can be found on Wikipedia, and that's definite proof that it is misleading, inaccurate and of limited practical utility in ways that they never disclose.


I will show how to derive DoF from first principles.

Let θ be the maximum acceptable angular blur, or the demanded angular resolution or something like that. Let's say it's a small angle in units of radians so I can pretend that tgθ=θ. If U is the object distance then define C_o as the object-referred circle of confusion, i.e. C_o = U·θ.

By simple application of similar triangles, C_o/(½DoF) = D/U, where D is the entrance pupil diameter of the lens. See here.

Therefore,
2·C_o = D/U·DoF
DoF = 2·C_o·U/D
DoF = 2·U²·θ/D


For more practical convenience, you could replace θ with something derived from angle of view and expected resolution in lines per picture size. I didn't bother, to keep the calculations shorter.

Observe that neither focal length, nor sensor size, nor image-referred circle of confusion make any appearance in this formula. DoF is an object space phenomenon and what happens in the image space is simply not relevant. The only part of the camera which matters is the entrance pupil, and the only limitation is whether you can find a lens with given entrance pupil diameter for whatever image circle and angle of view you may want.

In practice, wide angle lenses for small sensors have short focal length, which puts an upper bound on D by the virtue of practical limitations on how small f/D ratios can be realized. Hence shallow DoF is not achievable on small sensor cameras, but large DoF can be achieved on large sensors because large f/D is not hard. Just stop down.


But if you must introduce the f-number, perhaps because your lens vendor didn't print the pupil diameter on the lens, it's trivial to plug it into the formula in place of D.
DoF = 2·U²·θ·N/f

And now a big lie, because the widely known formula simply doesn't work macro. But let's pretend that image distance behind the lens is just the focal length f. Then f/U is the magnification and the image-referred circle of confusion C_i becomes C_i = C_o·f/U = θ·f.

Then we get the familiar, difficult to understand, and subtly wrong version from Wikipedia. QED.
DoF = 2·U²·N·C_i/f²

[TimFox]

There are different equivalent equations for depth of field.
A problem I always had is determining the appropriate circle of confusion for a given application, since that needs to go into any calculation.
Past that, when comparing two different cameras, the variables known to the user are
--f-number, which also determines the exposure when combined with the exposure time, i.e. "reciprocity".
--Frame size:  the size of the image on the film or sensor.
--Distance from lens plane (technically, entrance principal plane to subject)
--Field size:  the total angle mapped onto the frame, which determines the required focal length for a given frame size.

My comment was based on my evolution with film photography:  starting with 35 mm, then 4x5 inch, and finally 8x10 inch, with some 6x9 cm along the way.
I quickly noticed, judging DOF visually, that I had to decrease the aperture (increase the f-number) for a given visual DOF with increasing frame size, requiring longer exposures.

Yes, macro focusing is different, since the equations for the above simplify when the lens-frame distance is close to the focal length (focused at infinity), while at 1:1 magnification, the object distance and film distance both equal twice the focal length.

[magic]

My comment was based on my evolution with film photography:  starting with 35 mm, then 4x5 inch, and finally 8x10 inch, with some 6x9 cm along the way.
I quickly noticed, judging DOF visually, that I had to decrease the aperture (increase the f-number) for a given visual DOF with increasing frame size, requiring longer exposures.

That's what everybody notices, except it's different digital sensor sizes with kids these days


The point of my rant is that there actually exists a simple (and easy to calculate) parameter of the lens, which pleasantly stays constant when you try to produce similar images with different formats. And that there are simple geometric reasons why it is so.

The same principle also applies to things like exposure, sensitivity and noise. ISO100 on a phone is a completely different thing than ISO100 on 35mm frame. And it frankly should be called something else than ISO100. People try to patch the traditional theory with concepts like "35mm equivalence", but IMO it's still a convoluted way of thinking, full of traps (when to apply crop factor scaling or not) and it surely doesn't help that "equivalence" is often applied selectively by manufacturers to a subset of specs which make their product look favorable compared to alternatives from other formats, but not to some other specs.

I find that for comparison of different formats it is usually possible to completely eliminate the format and parameters which are tied to it (the focal length / angle of view relationship, f-number, ISO speed) and replace them with objective quantities referring to the object, available light and the final output image. The above DoF formula was an example of such approach.

This doesn't seem to be nearly as obvious and well known as it should be. Many people still walk around:
a) using cookbook formulas without understanding
b) not knowing stupid limitations of their applicability (like that thing above about macro)
c) thinking that they need a small camera for more DoF
d) advising others to switch to a smaller camera for more DoF


There also seems to be little interest in promoting honest comparisons by manufacturers. I'm shocked.


BTW, it seems I still failed to account for the change of C_o with distance.
As written, my formula doesn't predict the possibility of hyperfocal focusing.

[TimFox]

"35 mm equivalence" is a buzzword for relating the focal length in a camera with a different frame size, commonly used for "crop-factor" sensors such as the popular "APS-C" frame that is roughly 2/3 the linear size of a "full-frame".

ASA/ISO "speeds" originated with photographic film, and essentially measure how much light does it take per unit area at the film to expose that area.
Ignoring depth of field for a moment, in normal (non-macro) operation, the exposure is a function of only the ASA number, f-number, and exposure ("shutter") time.

Technically (nobody follows this, however), with film, since the chemical development directly affects the slope of the exposure curve, the ASA number related to the "toe", where exposure rises from the background density.
"EI", "exposure index", measured in the same units, is useful for E-6 films (with standard development), and corresponds to the "middle" of the curve from zero to full exposure.
With film, you can manufacture it in very large sheets and cut it down to the desired size, and the ASA number remains constant.
The same measurement works for solid-state sensors, with reference to unit area in the sensor plane.

[magic]

"35mm equivalence" is used for more than focal length. People talk about "equivalent aperture", which means same DoF and light gathering power as some reference lens, angle-of-view-equivalent on 35mm frame. They may or may not realize that it simply means equal pupil diameter. There is apparently a concept of "equivalent ISO", which means similar noise level and same brightness if the other "equivalences" are met. It's a wholly developed theory.
https://photographylife.com/equivalence-also-includes-aperture-and-iso

Regarding commercial examples, I recall some Sony/Zeiss fixed lens camera being marketed in terms of 35mm equivalent focal length and aperture; these numbers were actually printed on the lens in place of real ones. I don't like it very much, but I understand the marketing value and appreciate the honesty, compared with phone manufacturers advertising equivalent focal length but true f-stop.

Olympus came up with "magnification equivalence" for their 4/3 SLR macro lenses.


Most of that Theory Of Equivalence wouldn't be necessary if different metrics were used in the first place.
1. Replace f-stops with pupil diameter. This gives DoF and light-gathering equality between formats.
2. Replace exposure value with total collected light: D²·FOV²·t. This gives important shot noise equality between formats and is consistent with 1.
3. Replace ISO with sensitivity per whole frame rather than per unit area, for consistency with 1&2.

In my world, smartphones would start at ISO-four-digits and have the level of noise expected from that. Lenses would need to be marked with angle of view for convenience. Not a big deal, they are married to a particular system anyway. Only users who adapt lenses on different formats would need to worry about crop factors (appearing in the FOV² part). Now, everybody needs to worry about crop factors when selecting format to meet particular DoF or SNR target. Or ignore it and almost surely end up misled by marketroids.

This system also seems to work better for macro, without needing kludges like "effective aperture". The tricky part now becomes knowing FOV, because it varies despite constant focal length. Arguably, FOV variation with magnification is something that already exists anyway and one must be aware of, no matter what approach is used to calculate exposure.


It's all pure madness, I know

[TimFox]

In my photographic usage, the most important thing is exposure.
For large format, I use a spot meter.
DOF matters, but not so important.
When switching lenses (and possibly films), the relevant parameters are f-number and conventionally-defined ASA.
I will get the same exposure from a given subject highlight at 1/125 sec and f/8 with a 50 mm lens or 135 mm lens, up to my 600 mm lens that covers 8x10.
Similarly, I will get the same exposure from a highlight for 35 mm or 8x10 inch ASA 100 film with those shutter times and aperture expressed as f-number.

For metering macro, I take the lens-to-film distance and use it in an inverse-square calculation (compared to the focal length) and set the ASA dial on my spot meter accordingly.

[Mechatrommer]

those EV meter + ASA/ISO chart are long gone imho, its now built-in and automatic. even if we have to be manual, all it takes now is 1 or few quick shots in a second and look at the built in histogram. thats why more and more people are not aware of them, even on the still currently used focal-length/f-number/shutter speed/iso combo in real pro dslr mode...

[TimFox]

Where's the sport in that?
Have you ever indulged in focusing a complicated subject on a ground glass with the help of a loupe, or is it all done for you?

On my office wall before I retired, I mounted a 16x20 inch Ilfochrome print made from a 4x5 transparency (Kodak 64T) that was carefully focused and exposed.
A side view of the workings of a steam locomotive in a railway museum, lit with tungsten lights.
I had a challenge to the junior employees to make a print with equal technical quality (artistic quality is in the eye of the beholder), spending less than ten times as much as the equipment I used to make the image.
No replies.

[Someone]

ASA/ISO "speeds" originated with photographic film, and essentially measure how much light does it take per unit area at the film to expose that area.
Ignoring depth of field for a moment, in normal (non-macro) operation, the exposure is a function of only the ASA number, f-number, and exposure ("shutter") time.

Technically (nobody follows this, however), with film, since the chemical development directly affects the slope of the exposure curve, the ASA number related to the "toe", where exposure rises from the background density.
"EI", "exposure index", measured in the same units, is useful for E-6 films (with standard development), and corresponds to the "middle" of the curve from zero to full exposure.
With film, you can manufacture it in very large sheets and cut it down to the desired size, and the ASA number remains constant.
The same measurement works for solid-state sensors, with reference to unit area in the sensor plane.

That last bit is where it all falls apart, exposure is relative to the resulting "intensity". For digital cameras there are a variety of ways to rate ISO (which the camera manufacturers usually dont advertise) but its not per unit area or per pixel if going by the intensity measure. Below is the PTC/noise plot for a canon R3 from public data (Review at www.the-digital-picture.com) across ISO levels, that camera has a linear response in the sensor with enough dynamic range they can use the level based Standard Output Sensitivity or Saturation method to rate the ISO speeds (and then round them to convenient steps). ISO just becomes gain in that situation (with it approaching shot noise limited for modern sensors).

Thus your traditional exposure calculations still work to maintain your expected 18% exposure point. Compared to film the noise characteristics are different and there is no knee in the exposure, but you can add that back in post with a high bit depth or raw workflow.

Dynamic range and noise based ISO ratings are dependent on the pixel size/area etc, and that is where some manufacturers cheat with "equivalents". But it is easy enough to find or make proper assessments of any specific camera and get around those marketing lies.

[magic]

On my office wall before I retired, I mounted a 16x20 inch Ilfochrome print made from a 4x5 transparency (Kodak 64T) that was carefully focused and exposed.
A side view of the workings of a steam locomotive in a railway museum, lit with tungsten lights.
I had a challenge to the junior employees to make a print with equal technical quality (artistic quality is in the eye of the beholder), spending less than ten times as much as the equipment I used to make the image.

You chose a static subject (I think?) so I imagine something could be attempted with a tripod and panorama tricks. Other than that, no clue.

I will get the same exposure from a given subject highlight at 1/125 sec and f/8 with a 50 mm lens or 135 mm lens, up to my 600 mm lens that covers 8x10.
Similarly, I will get the same exposure from a highlight for 35 mm or 8x10 inch ASA 100 film with those shutter times and aperture expressed as f-number.

Yeah, I can see how this can be convenient, particularly if calculations are done by hand. My suggestion of "obviously, changing to narrower FOV means you increase the pupil by the same ratio to compensate" may not be equally compelling, even if this is exactly what happens when you go from 50/8 to 600/8.

The purpose of 35mm equivalence and my format-agnostic framework is navigating the crazy world of digital, where exposure is a solved problem (easily and quickly done empirically even when you don't trust AE) and the greatest challenge facing humanity is penny pinching by getting away with the smallest and cheapest sensor you can buy. Or sell to somebody Either way, the result is that things now need to work closer to physical limits than in the past.

[steve30]

"35 mm equivalence" is a buzzword for relating the focal length in a camera with a different frame size, commonly used for "crop-factor" sensors such as the popular "APS-C" frame that is roughly 2/3 the linear size of a "full-frame".

My new video camera advertises itself as being "25mm Wide". Of course it is actually a 4.12mm lens.

[TimFox]

On my office wall before I retired, I mounted a 16x20 inch Ilfochrome print made from a 4x5 transparency (Kodak 64T) that was carefully focused and exposed.
A side view of the workings of a steam locomotive in a railway museum, lit with tungsten lights.
I had a challenge to the junior employees to make a print with equal technical quality (artistic quality is in the eye of the beholder), spending less than ten times as much as the equipment I used to make the image.

You chose a static subject (I think?) so I imagine something could be attempted with a tripod and panorama tricks. Other than that, no clue.

I will get the same exposure from a given subject highlight at 1/125 sec and f/8 with a 50 mm lens or 135 mm lens, up to my 600 mm lens that covers 8x10.
Similarly, I will get the same exposure from a highlight for 35 mm or 8x10 inch ASA 100 film with those shutter times and aperture expressed as f-number.

Yeah, I can see how this can be convenient, particularly if calculations are done by hand. My suggestion of "obviously, changing to narrower FOV means you increase the pupil by the same ratio to compensate" may not be equally compelling, even if this is exactly what happens when you go from 50/8 to 600/8.

The purpose of 35mm equivalence and my format-agnostic framework is navigating the crazy world of digital, where exposure is a solved problem (easily and quickly done empirically even when you don't trust AE) and the greatest challenge facing humanity is penny pinching by getting away with the smallest and cheapest sensor you can buy. Or sell to somebody Either way, the result is that things now need to work closer to physical limits than in the past.

The locomotive was stationary (for many years), and I used a simple 4x5 field camera, 135 mm/5.6 Nikkor-W, and a tripod.
I maybe spend $1200 total for that setup (in a previous century).
Again, I concentrate on exposure for my purposes, and my large-format work always requires long exposures. 
Keeping the highlight exposure constant is a reason why good zoom lenses (for DSLR and SLR cameras) have a constant f-number when zooming.
That is a non-trivial design lens design problem.
I am a curmudgeon, and remember the second half of the 20th Century, where technical progress resulted in better results (audio reproduction fidelity, photographic speed and resolution, etc.).
In the 21st Century, "best possible" has been replaced by "good enough", or as you put it, "penny pinching by getting away with...".

[TimFox]

"35 mm equivalence" is a buzzword for relating the focal length in a camera with a different frame size, commonly used for "crop-factor" sensors such as the popular "APS-C" frame that is roughly 2/3 the linear size of a "full-frame".

My new video camera advertises itself as being "25mm Wide". Of course it is actually a 4.12mm lens.

Another parameter to beware is when sensor size is quoted as, essentially, vidicon equivalent size.
When I was looking for the physical sensor size for the IPhone14, which is 4.2 by 5.6 mm active area, I found it specified as 1/1.28".
Literally, (1 inch)/(1.28) = 19.8 mm, which is much larger than the actual size:   the diagonal of a 4.2 by 5.6 rectangle is only 7 mm.
I had run into that specification before when looking at industrial sensors, and it has something to do with the outside (glass) diameter of an equivalent vidicon.
A 1" sensor is actually 8 by 13.2 mm.

[Mechatrommer]

Where's the sport in that?

The locomotive was stationary (for many years), and I used a simple 4x5 field camera, 135 mm/5.6 Nikkor-W, and a tripod.

sport photography dont have time to setup all that... still photography is only a fraction of photography discipline/business/hobby, which i dont do much the closest i do is landscape both during the day and night... where is the sport? doing much lightroom work (previously known as darkroom processes), HDR, color correction/enhancement, photos stitching etc... heck even HDR and stitching process are now AI'ed in smartphone. you know? fondling those dark area in your locomotive so in the end printouts just look like the way we see the locomotive with our own eyes... making it surreal while maintaining naturality, is another art... ymmv.

[TimFox]

Where's the sport in that?

The locomotive was stationary (for many years), and I used a simple 4x5 field camera, 135 mm/5.6 Nikkor-W, and a tripod.

sport photography dont have time to setup all that... still photography is only a fraction of photography discipline/business/hobby, which i dont do much the closest i do is landscape both during the day and night... where is the sport? doing much lightroom work (previously known as darkroom processes), HDR, color correction/enhancement, photos stitching etc... heck even HDR and stitching process are now AI'ed in smartphone. you know? fondling those dark area in your locomotive so in the end printouts just look like the way we see the locomotive with our own eyes... making it surreal while maintaining naturality, is another art... ymmv.

"Where's the sport in that?" is a joke, not a reference to sports photography.
Some of the greatest sports photos from the middle of the last century were done with 4x5 inch Speed Graphics.

[Mechatrommer]

"Where's the sport in that?" is a joke, not a reference to sports photography.

clearly you didnt read until the end i think btw i didnt have the chance to play with film back then because it was "not my age", except my first learning in 2nd school my uncle lent me his Olympus SLR, cant afford real full darkroom back then. let alone the chance to play with large/medium format size, even today descendant of large/medium format (digital) is very much unaffordable, so i have to do whatever it takes in small format 35mm, currently known as full frame sensor... but then, nothing we cant do in cropped format APS-C DSLR except a little bit of noisier picture in low level lighting.

[magic]

35mm is small format, medium format digital is expensive, and large format digital doesn't seem to be much of a thing.

You would pay for having half of a silicon wafer processed just for yourself.
If you are lucky and it works out, because defect probability increases proportionally to chip area...

[magic]

but then, nothing we cant do in cropped format APS-C DSLR except a little bit of noisier picture in low level lighting.

Noise is a fundamental problem for webcams and cellphones, astrophotographers and action shooters. If you have noise in other situations, you are probably doing it wrong and tripod and/or lighting is the best bang for he buck solution.

Resolution is where small formats will always lose to larger ones and mosaic/panorama stitching is the only trick that can get you out of this, provided that your subject/scene is perfectly still. If you try to fish out the same details out of a smaller image, you inevitably reach a point where you run into diffraction. Then you need a "faster" lens to push it away, and you inevitably reach a point when you run into optical aberrations. Then you're done for good.

And we come full circle back to the original topic, because overdone sharpening is another reason why phone images suck. They just can't resist the temptation

[AndyBeez]

The unsharp mask can never replace resolution lost from lossy compression. Even if it's turned up to 11. All you get is obvious artefacts.

On the subject of noise, some year's ago I tried cooling a MOS detector with a Peltier cooler for astro photography use. The noise floor certainly lowered, but frosting on the sensor made the imager somewhat useless after five minutes - exposures were for 10+ minutes. Today, a contemporary DSLR back and stacking images, will do in your backyard what was done on a mountaintop observatory in sheet film some 40 year's ago. Electronics needs electrons to get excited, but not too excited.

[steve30]

These two came up on my Flickr homepage today. I'd say they are pretty representative of the fake blurring problems we are seeing regularly at the moment.

https://flickr.com/photos/emoisland/52541085884/in/dateposted/
https://flickr.com/photos/emoisland/52540786156/in/dateposted/

On the first one, the actual blurred background looks OK, but you can clearly see where the blur has spilled over onto the person, and where it has not gone in quite far enough, thus creating a noticeable outline. It has also got the depth of field completely wrong with one of the legs.

On the second photo, it got the depth of field right for the leg, but not the wooden floor he is standing on. In fact, it looks like he has been cut out of another photo, and pasted on the background.

Both photos suffer excessive sharpening on the 'in focus' parts. Flickr says they are done on an iPhone XR at ISO 400 4.2mm f/1.8.

[AndyBeez]

Those are fake blurgrounds? 

Looks like someone was new to learning layers in Photoshop. ( having seen what some so called web graphic designers achieve after three year's at uni, that might be considered proficient ).

Disclosure: I learnt my photography looking down a Lupe and stressing over film grain. If you know Kodachrome 25/64, Fuji RDP or Ilford PanF and XP2, then you'll know what I'm talking about. There is a generation for whom grain is something they add in Lightroom, to give an image the feel of a bygone era.

[TimFox]

Many years ago, film photographers would use very high speed Kodak films intended for surveillance purposes to get exaggerated grain for artistic purposes.
Before that, there was a painting style called "pointillism".
A famous example, now at the Art Institute of Chicago, is Seurat's "Sunday on La Grande Jatte".  https://artsandculture.google.com/asset/a-sunday-on-la-grande-jatte/twGyqq52R-lYpA?hl=en&ms=%7B%22x%22%3A0.5%2C%22y%22%3A0.5%2C%22z%22%3A9.4555024148453%2C%22size%22%3A%7B%22width%22%3A1.616855466120035%2C%22height%22%3A1.2375000000000003%7D%7D
Perhaps, that is the only painting to be adapted into a musical comedy, "Sunday in the Park with George", by Sondheim et al.