Author Topic: Digital camera sensors and the "film analogy"  (Read 1703 times)

0 Members and 1 Guest are viewing this topic.

Online paulcaTopic starter

  • Super Contributor
  • ***
  • Posts: 4362
  • Country: gb
Digital camera sensors and the "film analogy"
« on: July 14, 2021, 08:27:38 pm »
I keep finding myself making faces when photographers talk about things like ISO and shutter speed on digital sensors like they mean the same thing as they do in film.  I know that camera makers have been desperately trying to make that true also.  But it's just not. 

ISO is just analogue gain on the sensor.  The noise resulting from it, is not "digital noise", there is no such thing, it's the same analogue noise that was present on your sensor, just amplified.  More expensive sensors are less noisy so higher ISOs work better.

Shutter, is irrelevant.  Sensors have no accumulative effect like film when exposed.  They would only last one photo is they did.  People talk about them like the pixels are buckets that catch photons.  They don't.  They sample an instant.... or on a CMOS sensor a skewed slice across an instant in time.  To create a long exposure the DSP (Digital signal processor) in the camera, rapidly samples each instance in time, I assume as fast as it can or as much buffer/processing memory it has and ADDs them together like photo stacking in astrology.

I have seen advice to turn off "Long exposure NR" in my camera by several YouTube photographers.  However I feel it is likely the ONLY time NR can be done on long exposures is in the camera while those individual samples exist or are at least being stream processed.  Otherwise you will be adding together all the noise from all the samples and trying to remove all of it in post editing.  It is better to remove the noise while compositing the exposure rather than add it all together for the end surely?

In fairness the primary reason they give that advice is, after a long exposure your buffer/processing memory will be 100% full of samples and it will take, however long it takes to clear that buffer before it will shoot again.  On the A6100 this can be 10 seconds for example.

The other aspect would be the sensor sample rate.  On long exposures it can't just take an infinity number of samples per second and have the processing power to immediately composite them.  So there has to be a compromise and the longer the exposure the more it will be impacted by memory and processing limitations. 

It is however very likely HIGHLY optimised only adding differences and using ASIC electronics and "microcode" to process samples at incredible rates.

If I'm correct, sensors should have certain properties such as "Frequency" and "Response time" and other digitiser-IC-style parameters.  For example, a pixel stands zero chance of sampling an LED blinking at 10KHz if the sensor frequency is less than 20kHz.  You would effectively get a pseudo-random result long exposure shooting it.  I'm sure people like Nikon/Cannon/Sony et. al. have billions of IP invested into the real processing DSPs and ASICs to do this.  However, digital cameras with variable "shutter speed" have been around for decades in some forms, so the DSP required to do it must also be fairly old.

If I'm wrong, I probably need to look further into how exactly CMOS/CCD/* sensors manage to produce a cumulative electrical effect in the sensor over time and then reset it.  Optical-accumulative-electro-static?
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 8998
  • Country: us
  • Retired, now restoring antique test equipment
Re: Digital camera sensors and the "film analogy"
« Reply #1 on: July 14, 2021, 09:21:28 pm »
Technically, the ISO (or ASA) rating on a photographic film refers to the "toe", where the emulsion starts to expose.  Many transparency films, and proper users of the Zone System, refer instead to "EI", which is in the same units but measures the mid-point of the S curve from blank to exposed, but this depends on the development.  ISO is supposed to be a parameter of only the film.  Note that the photographic literature rarely discusses this.
Now, for a digital sensor, there is an inherent maximum level, where the charge on each pixel saturates as the diode forward-biases.  Typically, a camera sensor saturates at an exposure level that is typical of approximately ISO 100 to 200 film.  The very first line-sensor device from Fairchild that I encountered in the early 1970s was specified as equivalent (in that sense) to ISO 100.
The ISO setting in a digital camera is, in fact, the gain from the sensor to the ADC input, as you stated.  The quantization noise of the ADC remains constant with respect to the digital word put out, but as you crank up the gain the noise in the dark current of the sensor becomes larger with respect to the full-scale of the ADC and the image appears noisier.
Modern cameras may have built-in tricks to average multiple exposures by computation to improve the SNR, but this requires a constant object and a good tripod.
Exposure is the product of light input rate (determined by the object brightness and the f-number of the lens) and the exposure time (which in a SLR may be determined by a physical shutter).
"f-number" is merely the effective aperture diameter of the lens divided by its focal length, which is why it is written "f/16", where f is the focal length on the aperture scale.  Traditionally, a lens has click stops at "1/2 stop", where a "full stop" is a factor of 1.4:1, or an area factor of 2:1.  A traditional shutter has click stops at full stops (2:1 in time), and ISO settings on the light meter at "1/3 stops", or 1 dB (in terms of light energy).  Again, this last statement confuses some, since twice the light energy develops twice the charge in a digital camera, which is 6 dB at the ADC voltage input.
Another important difference is "reciprocity", as in "reciprocity failure".  With film at very long exposures, some of the charge on the photographic grains while the chemical state of a grain is still metastable, and not fully exposed, will drain off, so it takes more total light-induced charge on the grains to achieve the stable "latent image" chemical state.  I have not seen any good descriptions on any analogous conditions with photosensors at very long exposure times, but I'm sure the astronomers have carefully looked into this with their cooled sensors.
Once, when using my 8x10 inch view camera downtown, shooting a cityscape, a tourist asked me how many megapixels I had.  I did a mental calculation and replied "about 500".  Repeating that calculation at home, I found that was approximately correct, although the sampling is random with finite resolution, as opposed to periodic in a digital sensor.
 
The following users thanked this post: Kleinstein

Offline Sal Ammoniac

  • Super Contributor
  • ***
  • Posts: 1764
  • Country: us
Re: Digital camera sensors and the "film analogy"
« Reply #2 on: July 14, 2021, 10:37:09 pm »
Digital sensors don't have any reciprocity failure that I've been able to measure. I use a monochrome CMOS camera (with filters) for astrophotography and have never seen any effect that I could attribute to reciprocity failure like film has. My camera has an in-built Peltier cooler. Back in the bad old days of film, reciprocity failure was a big problem and people mitigated it to some extent in two ways: cooling the film (typically with dry ice) and by soaking the film in hydrogen gas before exposure.

Amateur astrophotography was horribly primitive back in the film days compared to today with digital cameras, stacking software like PixInsight, autoguiding, plate solving, etc. Nowadays I can tell my telescope what I want to image, push a button, go to bed, and come back in the morning to close everything up.
"That's not even wrong" -- Wolfgang Pauli
 

Offline eti

  • Super Contributor
  • ***
  • !
  • Posts: 1801
  • Country: gb
  • MOD: a.k.a Unlokia, glossywhite, iamwhoiam etc
Re: Digital camera sensors and the "film analogy"
« Reply #3 on: July 14, 2021, 10:42:02 pm »
Destin has all you need to know, in his usual, incredibly detailed, well-explained style:

 

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 8998
  • Country: us
  • Retired, now restoring antique test equipment
Re: Digital camera sensors and the "film analogy"
« Reply #4 on: July 14, 2021, 10:47:37 pm »
Reciprocity failure in film is physically interesting:  it indicates that more than one photon is required to "flip" a single grain from its initial state, through one or more metastable states, to the final stable state.  Clever photographers can use it to reduce the contrast between highlights and shadows (tricky).  Astronomers were the first to investigate it seriously with photographic emulsions, but I'm sure they are glad to avoid it now.  There should be some resistive leakage of charge during a long exposure, as well as biased-diode dark current (adding exposure to shadow regions), but the cooling reduces the dark current (and its noise) greatly.
« Last Edit: July 14, 2021, 11:07:48 pm by TimFox »
 

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5569
  • Country: us
Re: Digital camera sensors and the "film analogy"
« Reply #5 on: July 15, 2021, 12:11:51 am »
I have never dived into the details of consumer grade cameras, but some of the original posters assertions about digital sensors are wrong, at least when applied to all digital sensors.

At their core, all image sensors I am aware of feed the charge generated by a photosensor into a small capacitor assigned to that pixel.  Incoming photons are converted to electrons with some fixed efficiency (this is true for both photovoltaic and photoconductive modes of operation) and these are accumulated in this capacitor.  The accumulation time is called integration time and is chosen to prevent charging the capacitors to saturation (typically close to the supply voltage) on the brightest pixels in the array.  Integration time is also limited by the frame rate of the camera, in video sensors this is a modest fraction of a second while long exposure astronomical cameras may integrate for seconds or even hours.  At the end of the integration time the charge is dumped and the process restarts.

Integration time is analogous to shutter in film.  There are several noise terms in the sensor which will be proportional to the square root of integration time, while the signal is directly proportional to integration time.  Now it may be true that integration time is fixed in consumer grade cameras and/or that quantization noise is the dominant noise in this grade camera but in general "exposure time" or integration time matters. 

Some architectures may have very short integration times and sum the results from a series of samples.  The result is analogous to a single longer integration time.   Noise is reduce by the square root of the number of samples and signal increases linearly unless the implementation is incredibly inept.  So shutter matters. 

This isn't just theory.  I have observed it in low light conditions with my consumer digital cameras.  Short exposure times result in noisy images.  Noise is reduced on longer exposure times
 

Online paulcaTopic starter

  • Super Contributor
  • ***
  • Posts: 4362
  • Country: gb
Re: Digital camera sensors and the "film analogy"
« Reply #6 on: July 15, 2021, 12:49:13 am »
and these are accumulated in this capacitor. 

Best answer that explains my error.  I realised the alternative WAS a capacitive solution.  If you over expose, you .. over expose, the capacitors fill to capacity, so detail is lost, highlights a "blown out".

If that is true I do need to rethink my assumptions about CCD, CMOS sensors.  I thought they were instantiation transistor light sensitive semiconductors that where scanned or sampled. 

"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline Someone

  • Super Contributor
  • ***
  • Posts: 5155
  • Country: au
    • send complaints here
Re: Digital camera sensors and the "film analogy"
« Reply #7 on: July 15, 2021, 01:06:29 am »
If I'm correct,
You're not, so best to go back and learn how this all works before claiming such a silly position.
 

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5569
  • Country: us
Re: Digital camera sensors and the "film analogy"
« Reply #8 on: July 15, 2021, 02:02:00 am »
and these are accumulated in this capacitor. 

Best answer that explains my error.  I realised the alternative WAS a capacitive solution.  If you over expose, you .. over expose, the capacitors fill to capacity, so detail is lost, highlights a "blown out".

If that is true I do need to rethink my assumptions about CCD, CMOS sensors.  I thought they were instantiation transistor light sensitive semiconductors that where scanned or sampled.

First, if they were instantaneous no photons would have time to arrive.

CMOS and CCD are sensing the light the same way, just reading it out differently.
 

Offline twospoons

  • Frequent Contributor
  • **
  • Posts: 269
  • Country: nz
Re: Digital camera sensors and the "film analogy"
« Reply #9 on: July 15, 2021, 03:04:13 am »
There is another technology I came across while working with the photonics guys at ST on a custom CMOS image sensor. It was still in its infancy, but it was based on SPADs (single photon avalanche diode). A 'pixel' could detect and count individual photons using a digital accumulator.  They were aiming to use this combined with time-of-flight to create an image sensor that could record depth along with brightness at each pixel. In other words, a 3d image sensor.

This tech is not used in cameras yet, as far as I know.

FYI one of the more curious noise sources in image sensors is known as 'kTC reset noise' - essentially it means that when you reset the integrating capacitors there's a small random amount of charge left behind, which shows up as low level image noise. k= Boltzman's constant, T = temperature, C= capacitance, so you can see cooling the sensor will reduce the noise.

You also get 'fixed pattern noise' which results from tiny variances across the chip leading to a pattern thats always there. Being a fixed quantity, its pretty easy to remove digitally.
 

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 8998
  • Country: us
  • Retired, now restoring antique test equipment
Re: Digital camera sensors and the "film analogy"
« Reply #10 on: July 15, 2021, 03:41:29 am »
The interesting thing about "kT/C" noise is that it is independent of the ON resistance of the switch, although the resistance changes the time required to fully reset the capacitance.
« Last Edit: July 15, 2021, 03:43:51 am by TimFox »
 

Offline magic

  • Super Contributor
  • ***
  • Posts: 7453
  • Country: pl
Re: Digital camera sensors and the "film analogy"
« Reply #11 on: July 15, 2021, 06:30:12 am »
Some architectures may have very short integration times and sum the results from a series of samples.  The result is analogous to a single longer integration time.   Noise is reduce by the square root of the number of samples and signal increases linearly unless the implementation is incredibly inept.  So shutter matters. 

This isn't just theory.  I have observed it in low light conditions with my consumer digital cameras.  Short exposure times result in noisy images.  Noise is reduced on longer exposure times
Do you mean longer exposure at lower ISO in the same light or longer exposure at the same ISO in lower light?
Noise reduction in the former case seems explainable by more photons being captured and lees gain applied. My experience agrees.
In the latter seems it seems extremely dubious that any improvement could be had. My experience is that noise increases.
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 15151
  • Country: de
Re: Digital camera sensors and the "film analogy"
« Reply #12 on: July 15, 2021, 09:18:52 am »
The modern smart phones and consumer cameras are quite good in combining multiple pictures taken with a moderate exposure time (e.g. 20 ms) instead of a long exposure. The magic here is in also taking the movement of the camera into account to do a digital steady shot without a tripod even with an effective long exposure. So there can be some picture averaging for longer exposures, but usually only at the long end (e.g. > 100 ms).

The gain / iso setting is often also allowed higher than actually justified noise wise. So the highest ISO setting may be of limitd use. So a lower iso setting and longer exposure can really help, even without picture averaging.

For the noise, there is also shot noise than can become relevant: The number of photons hitting the pixel has an uncertainty the square root of the number. The analog sensro can't get better than photon counting, where the statistical error becomes obvious. AFAIK the modern sensors can in the low light range resolve some 50-100 electrons - so the shot noise can become relevant with more than 6-7 bit resolution for the brighter pixels. So it needs a certain amount of light for a really good picture. Amplification can't help here very much and the sensors are already quite good with the quantum efficiency.

AFAIK some CMOS sensors aready used coherent double sampling, so there is a way around the kTC reset noise. A good read-out circuit can be lower noise than kTC, though it is not easy. So the starting charge is also read and subtracted instead of assuming a fixed starting point.
 

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 9330
  • Country: fi
Re: Digital camera sensors and the "film analogy"
« Reply #13 on: July 15, 2021, 09:36:52 am »
You have incorrect understanding of the shutter.

CCD and CMOS sensors do have very real shutters, either electronic, mechanical, or combination thereof.

They do continuously collect photons, like film. There are no gaps in time, nothing digital, no DSP magic involved. (Of course, you can also digitally merge multiple shots but that doesn't usually make sense because doing a longer exposure on the "analog" level to begin with is simpler, and superior.)

Now I don't know if some modern camera chip primarily optimized for shooting video does use "virtual" longer exposure by combining multiple frames digitally, I don't know, but it doesn't need to be this way, fundamentally the sensor chips easily support arbitrary length exposures like film and this is how it classically is done.

A simplified model is; a photon is converted into electron; electrons (i.e., charge) are collected into buckets. A pixel is a bucket. The bucket can be emptied by the electronic "reset shutter"; another electronic shutter mechanism can prevent more charge accumulating in the bucket because readout of the buckets take time and you don't want to continue the exposure during that time. The latter electronic mechanism can be replaced with mechanical shutter which was common just a decade ago, because it doesn't waste the fill rate on the sensor. A very common failure mode on the digital pocket cameras of the 2000's is the failure of the mechanical shutter, which can be seen as striping as some lines of the sensor are read out later than others, and receive more light during readout.

Leakage currents may limit the maximum exposure time, but decent sensor chips can do minutes at room temperatures. Astrophotographers who need hours of exposure and prefer low noise levels benefit from sensor cooling.

Similarly to the leakage currents, film had reciprocity law failure, but modern film stocks of 2000's had all but got rid of that. This means that basically, if the time difference between photons hitting the same grain is too long, the chances of "missing" one of them gets higher. Thanks to modern (late 1990's to early 2000's) features like 2-electron sensitization, modern films can detect much smaller photon counts making them more sensitive / less grainy, and also making reciprocity law failure almost disappear.

I would hazard a guess that all good-brand sensor chips (or conversion ICs, on CCDs that may be a separate chip) have implemented correlated double sampling to get rid of reset noise at least for 15 years, my guess it's a concern of discussion, not a feature present in final products exactly because it's being taken care of. I'm saying this because I developed a CCD scanner module in 2010 and used a very low-cost commodity scanner ADC IC which did have this feature built-in.
« Last Edit: July 15, 2021, 09:45:08 am by Siwastaja »
 

Offline magic

  • Super Contributor
  • ***
  • Posts: 7453
  • Country: pl
Re: Digital camera sensors and the "film analogy"
« Reply #14 on: July 15, 2021, 10:39:05 am »
They do continuously collect photons, like film. There are no gaps in time, nothing digital, no DSP magic involved. (Of course, you can also digitally merge multiple shots but that doesn't usually make sense because doing a longer exposure on the "analog" level to begin with is simpler, and superior.)
As explained above, cellphones do it to eliminate shake blur in software. If you move your camera during a single 100ms exposure there is no recovery from that.
 
The following users thanked this post: Siwastaja


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf