Fake background blur on camera-phones

[Mechatrommer]

ps: this is partly in respond to magic's confusion on current vs previous HW vs SW tech in photography... and also partly related to the OP's topic... recently Topaz AI (SWs from Topaz Labs) got into my radar, its quite amazing SW at image recovery in general, and to my current definition, its a magic bunch of softwares, implementing AI model/database into otherwise classical method of image enhancement/enlargement/noise reduction... (the most surprising to me is Topaz Gigapixel AI at recovering blurry face into sharper and enlarged size... but thats off topic) the sample picture below was taken using Canon EOS 7D Mk2 @ ISO 6400 @ lens 17mm f/2.8... before.jpg is cropped portion of image, unscaled after what i guess as the best camera's built-in algorithm at reducing image's noise out of the 20Mpx APS-C sensor.. after.jpg is further enhancement using Topaz Denoise AI... the improvement imho is significant, although there are still imperfection to the AI algortihm, we still can see sort of aliasing somewhere... if you think noise problem is only for astronomers? try shooting an event in large ballroom with dark paint on every walls without your own artificial lighting and without full frame sensor at hand, high iso and less noisy pictures can really usefull!... my point is if such AI algorithm can be implemented real time in small sensor camera such as smartphones and even smaller sensor dedicated camera such as APS-C format, better (perceived) image quality or larger Mpx size can be achieved further... but this magic AI software has GBytes of installation and model files, so i believe currently its only for powerful PC and not real time. what i would like to see is Topaz "Fake Blur Backround" AI algorithm to get better fake-blur background but then currently there is no yet can beat real optic blur, hence i'm getting a used Canon 85mm f/1.2 because of that, after many years playing with less fast f/2.8 zoom lenses...

[magic]

Meh.
Didn't know that there are stars inside oscilloscopes

[steve30]

That's actually quite a good job of denoising.

[Mechatrommer]

That's actually quite a good job of denoising.

there is no free lunch, every denoise algorithms will kill away some details as magic described last year about the "flatten" sky, older point and shoot (no-denoiser, ie alot of random color noises, but no flattened sky) vs modern smartphone/camera (with denoiser built-in, less apparent noise, esp color noise, but with flattened sky), similar to when trying to recover original data from lossy compression, its impossible. modern AI algorithm seems capable of doing it, but imho its just an illusion, a try and guess and assume method of what it was. Topaz can produce some funny faces (not me and not who the people i know) when original file is excessively noisy or blurry than what the algorithm "expects"... normal users usually wouldnt notice until they compare the picture with more expensive larger sensor dedicated camera (dslr), no noise and sharper optics. ymmv.

[magic]

It's doing good job on large flat areas, but that's an easy job. And I guess it's also doing a good job of detecting such areas.

On areas with more busy detail it seems to switch to sharpening and only makes noise worse. The grilles of the equipment look ugly.

There is some sort of paper or cloth on the lower piece and it got some texture on it which is probably just noise amplified by the processing.

The text "keyboard" changed to "kθyboard", which would be a meaningful error in "serious" applications like text recognition or machine vision.
There is a blurry and noisy red text on a book or box in the bottom right corner, turned into meaningless curved lines. "Fake it and maybe you'll make it".

Overall contrast of the image was increased too. The white metal construction is brighter, I think parts of it are blown out, even.

edit
Next time you could also produce a longer exposure shot at lower ISO for reference, to compare the "enhanced" image against reality.

[Mechatrommer]

yes agreed, AI that is not clever enough, exactly this thread is all about.. i'm not going to do in-depth comparison in ee forum, and i should've setup a better controlled environment rather than a random mess. i know what it is, it is what it is. i just took a random picture, in the way mimicking the way i usually did in low light and challenging event. or pushing or knowing the limit of high iso high noise and how good today's denoise can do.. this AI still need human AI (actual intelligent) touch to improve or select the good part of it in photoshop and mask out the worse parts.. but it will need time... ymmv.

[Berni]

Yeah right now AI is best when guided by an actual human.

The benefit of AI in image editing is automating the tedious tasks. Like an easy example is "content aware fill" to remove something from a photo. Doing that by hand can be very time consuming to grab the correct parts of the background, duplicate them out in a way that doesn't make an obvious repeating pattern, align them well, blend it in to the rest..etc. The content aware fill simplifies that down to just selecting the thing you want gone and waiting a fraction of a second. Saves a massive amount of time.

[coppice]

Yeah right now AI is best when guided by an actual human.

Yeah, right now a novice human is best when guided by a skilled human,

[5U4GB]

Someone on IRC linked me to a major manufacturer's product page (I forget which) where they were promoting their shallow depth of field phone camera, and gave loads of example pictures. Some were actually quite good, but I'd say on half of them, it was blindingly obvious that it was faked.

Well, people with DSLRs buy fancy tilt-shift lenses to get the same effect, so someone probably thought "hey, lets emulate that in software!".  There are times when it's pretty useful... OK, not many times, but still some, and given the useless junk phone vendors are now putting in they've pretty much run out of ideas elsewhere.

[TimFox]

Someone on IRC linked me to a major manufacturer's product page (I forget which) where they were promoting their shallow depth of field phone camera, and gave loads of example pictures. Some were actually quite good, but I'd say on half of them, it was blindingly obvious that it was faked.

Well, people with DSLRs buy fancy tilt-shift lenses to get the same effect, so someone probably thought "hey, lets emulate that in software!".  There are times when it's pretty useful... OK, not many times, but still some, and given the useless junk phone vendors are now putting in they've pretty much run out of ideas elsewhere.

Once the image is formed at the sensor or film surface, it cannot be re-focused.
Of course, one can apply filters to the image data in order to, for example, boost the high-frequency content of the image after that, or smoosh the high frequencies to obtain blur.
The purpose of a tilt-shift lens is to obtain a limited effect, better done with a "view camera", to establish a plane of best focus (in object space) that is focused onto the image plane, by tilting the lens plane with respect to the image plane;  view cameras have far more tilt angle and shift displacement than a normal "tilt-shift" lens for an SLR or DSLR.
"Tilt" here means rotating the lens holder about a horizontal axis, and "swing" means rotating about a vertical axis, depending on the exact application, possibly of both.
For example, a typical Ansel Adams photograph would tilt the lens down so that the top of a distant mountain and the brook close to the camera are both in good focus, allowing the less interesting object space between them to go "soft". 
"Shift" means displacing the lens axis with respect to the center of the image.
This is useful, for example, when you make the lens and image planes vertical, to avoid convergence of vertical lines in the tall building you are imaging, and shift the lens upwards to reduce the amount of parking lot in the foreground.
For a detailed analysis, look up the "Scheimpflug principle":  https://www.opticsforhire.com/blog/scheimpflug-principle/