How do videoscalars like retrotink and ossc work (technically)

[bzuidgeest]

Hello,

The last few years I have seen all kinds of video converters in the retro computer scene. Converting even the signals from ancient consoles to something better.

I under stand the basic idea behind them, first digitizing (often), then processing (scaling and other effects), then output in the new format either analog or digital.

A lot of hassle is made about keeping lag as low as possible. What I cannot seem to figure out and cannot find any documentation on is how the keep everything synced together. Most analog formats don't exactly have a pixel clock or something.

Does anyone here know of a good write up on how the "magic" is done? Not in generics, but with actual examples. Examples need not be harder then for example going from cga to vga with integerscaling in between and or things like moving from interlaced to progressive formats.

[dmills]

Not somewhere I have played, SDI generally being my starting point, but I can make some informed guesses.

Sync separator then lock a PLL to the input line rate to produce a pixel clock at whatever frequency is appropriate, some ability to slip the input pixel sampling phase to match whatever the video hardware is generating is probably desirable. This could probably be done with some magic involving an NCO and playing with the phase trimming given that a reasonable FPGA will go at way above the required pixel clock rate for 480p

If the input is composite then a well designed comb filter in the digital domain is likely better then trying to deal in the analogue space, and you will want to sort out the various options for chroma subsampling and assumed gamma, a quick enough ADC for this is reasonably trivial these days.

A good deinterlacer is actually one of the hardest things to write, because doing it really properly involves motion estimation, and that just eats gates (and memory bandwidth). Doing it low latency just adds to the joy. Getting below 1 frame of latency is basically impossible if doing it well, but for stuff like old console games there are options.

Scaling however is fairly easy, especially for the integer ratio cases, being a case of applying a 2d polyphase filter to produce the output samples, this can be done with only a few lines of latency depending obviously on the size of the filter kernel.
Memory bandwidth requirements can be reduced by doing it as a two 1D filters at the cost of some loss of accuracy that often does not matter. The hard thing here is making sure that the memory layout works to put the correct pixels on the bus when you need them, not trivial given the nature of DDR3 and friends really needing at least row length reads or writes (Fine for the horizontal processing, pain in the arse for the vertical bit).

Line doubling is of course easy, buffer 1 line, play it out twice at double speed, while buffering the next line, then just ping pong. Not how you write a GOOD scaler (Which is black magic), but very, very low latency.

[dave j]

There's also hoglet's Raspberry Pi Zero based RGB to HDMI converter. As well as the full source being available for it, there is extensive discussion of its development on this stardot.org.uk thread.

[tooki]

A lot of hassle is made about keeping lag as low as possible. What I cannot seem to figure out and cannot find any documentation on is how the keep everything synced together. Most analog formats don't exactly have a pixel clock or something.

Analog video doesn’t have a pixel clock because it doesn’t have pixels at all. It has lines, which are discrete, but within each line it’s a continuous signal. In interlaced video, like analog TV, a fixed number of lines form a field (even or odd) and two fields form a frame. In progressive video (like VGA or some analog HD formats), a fixed number of lines form a single frame.

The horizontal sync signals the beginning of a line, and vertical sync signals the beginning of a field or frame.

There are a few different synchronization schemes, but the upshot is that there’s always some mechanism for sync. Computer video connections like VGA usually have separate H and V sync lines. Some have one of those embedded in the green channel. Analog TV formats use complex voltage levels to encode the sync; it’s plainly visible in the waveform.

[bzuidgeest]

A lot of hassle is made about keeping lag as low as possible. What I cannot seem to figure out and cannot find any documentation on is how the keep everything synced together. Most analog formats don't exactly have a pixel clock or something.

Analog video doesn’t have a pixel clock because it doesn’t have pixels at all.

You clearly missed the sarcasm in the highlighted statement. It's one of the few things I actually seem to understand about this topic.

[Siwastaja]

While analog does not have pixels (and thus no sync for pixels), it has sync for lines, and sync for frames.

All that needs to be done is to decide number of pixels that go into a line, synchronize to the start of the line, and then digitize that many pixels. The more, the better.

If the analog material is originally "pixel perfect" digital art, and the number of output pixels is limited for technical reasons, then more sophisticated algorithms are needed for good results: something that tries to find the pixel boundaries of the original and map them to output pixels. LCD screens with VGA inputs try to do tricks like this.

[bzuidgeest]

@dmills, Thanks for replying, but pretend for a minute that I don't know how implementations of those steps work. Should not be to hard . Your reply (appreciated) is actually why I was asking the question. High level explanations there are a lot of, not so much in depth. Let me put it this way, making a good amplifier can get "complex", but I can pick up any textbook on electronics and find basic explanations, examples and experiments. Same for so many other topics. But not this one (as far as I could find.

@dave_J that is a usefull link, not exactly a textbook, but there is some useful information in there. I knew about the project, but not this thread. I even already looked at the code. The thing is, that there is a lot in there, so as a basic example, you would have to strip a lot away. So as a teaching aid it becomes usefull after one has some idea what he is looking at.
 

[bzuidgeest]

While analog does not have pixels (and thus no sync for pixels), it has sync for lines, and sync for frames.

All that needs to be done is to decide number of pixels that go into a line, synchronize to the start of the line, and then digitize that many pixels. The more, the better.

If the analog material is originally "pixel perfect" digital art, and the number of output pixels is limited for technical reasons, then more sophisticated algorithms are needed for good results: something that tries to find the pixel boundaries of the original and map them to output pixels. LCD screens with VGA inputs try to do tricks like this.

That sounds simple, just sync to the line. But surely lines, go faster in the digital signals then the old analogue ones. So you would have to look into the future to know what the slower signal is going to do. How do they keep up to each other? I might sound dumb here, but I was asking for basic explanation/tutorial stuff for a reason....

[tooki]

That sounds simple, just sync to the line. But surely lines, go faster in the digital signals then the old analogue ones. So you would have to look into the future to know what the slower signal is going to do. How do they keep up to each other? I might sound dumb here, but I was asking for basic explanation/tutorial stuff for a reason....

Huh? Just how exactly do you think a digitizer works?

I’m sure there are different methods, but I’d expect that the digitizer either digitizes the entire video signal continuously and then extracts the lines from it (either extracting the sync to a trigger from a composite signal, or from separate sync signals), or that it syncs first and then digitizes each line. Either way, the sampling rate is something the designer can choose. After that it’s all digital signal processing.

[dmills]

To line double you start at the beginning of the line sampling the data into a memory, at a multiple of the line rate, then when the line buffer becomes half full (so half a line of latency) you start reading out the buffer twice as fast as you wrote it. You will thus catch up with the writer just as you hit the end of the line buffer, and can start playing the line buffer a second time (for the doubled line). At the time the second line is played out, your input side will be half way along the line buffer so you can immediately go again.

The trick here is to configure the dual port ram such that if a simultaneous read and write occur, the read port gets the old data (Otherwise the first pixel on the doubled line gets stomped). 
 
Obviously higher multiples need slightly different timing to work, but the idea is the same.

It is probably worth using an actual PLL to produce the sampling clock to minimise jitter relative to the line sync pulse as this will translate directly into jitter in the output image, a bit of a pain as there are multiple different input formats.

[jonpaul]

see history of video codecs and compression.

Yeves FAROUDJA did the pioneering work in 1980s.

https://en.wikipedia.org/wiki/Faroudja

BRAVO to our old friend !


Jon

[nctnico]

That sounds simple, just sync to the line. But surely lines, go faster in the digital signals then the old analogue ones. So you would have to look into the future to know what the slower signal is going to do. How do they keep up to each other? I might sound dumb here, but I was asking for basic explanation/tutorial stuff for a reason....

Huh? Just how exactly do you think a digitizer works?

I’m sure there are different methods, but I’d expect that the digitizer either digitizes the entire video signal continuously and then extracts the lines from it (either extracting the sync to a trigger from a composite signal, or from separate sync signals), or that it syncs first and then digitizes each line. Either way, the sampling rate is something the designer can choose.

No, as Siwastaja already wrote: if the source of the analog signal is digital (which is the case for video consoles), then there will be a finite number of pixels in each horizontal line of the image. If you want the best results, then you'll need to find those pixel boundaries and lock the sampling to these boundaries so you are sampling the pixel values and not the crossover between pixels. You should try and hook up an LCD monitor to an analog VGA output and let the screen 'self adjust'. You'll see this locking process in action (and typically get horrible results when the sampling isn't locked to the pixel boundaries).

[tooki]

That sounds simple, just sync to the line. But surely lines, go faster in the digital signals then the old analogue ones. So you would have to look into the future to know what the slower signal is going to do. How do they keep up to each other? I might sound dumb here, but I was asking for basic explanation/tutorial stuff for a reason....

Huh? Just how exactly do you think a digitizer works?

I’m sure there are different methods, but I’d expect that the digitizer either digitizes the entire video signal continuously and then extracts the lines from it (either extracting the sync to a trigger from a composite signal, or from separate sync signals), or that it syncs first and then digitizes each line. Either way, the sampling rate is something the designer can choose.

No, as Siwastaja already wrote: if the source of the analog signal is digital (which is the case for video consoles), then there will be a finite number of pixels in each horizontal line of the image. If you want the best results, then you'll need to find those pixel boundaries and lock the sampling to these boundaries so you are sampling the pixel values and not the crossover between pixels. You should try and hook up an LCD monitor to an analog VGA output and let the screen 'self adjust'. You'll see this locking process in action (and typically get horrible results when the sampling isn't locked to the pixel boundaries).

I’m well familiar with that. But I didn’t overlook it, that falls under the “sync first and then digitize each line” approach I mentioned.

[m k]

Vertical is as equally important as is horizontal.
If it's up scaling it finally has caps to fill.

First there is a film camera.
There film is exposed to light, the whole picture at once.
Keyword here is time.

--A--B--C--D--

Picture taking time start from A and end to B.
Then is shutter time from B to C.
And next picture from C to D.
Object is moving all the time.

Computer picture has no time, if it is not added.
On the other hand, shutter time is lost for ever.

Frame rate adds a 3rd dimension.

So interpolating 2x 2x 2x and there is a single real pixel in the center of the cube.
And copying frames create a temporal disruption.

[tooki]

First there is a film camera.
There film is exposed to light, the whole picture at once.

FYI, that’s not necessarily true. SLRs and most other removable-lens 35mm film cameras use shutters with two shutter curtains that move independently of each other, from one edge to the other (mostly vertically, but earlier ones horizontally). This means that exposure begins at one edge of the film and then progressively moves to the other. One curtain is opening, the other closing shortly afterwards. For longer exposure times, the frame will be completely unblocked for some period of time, but for short shutter speeds, the opening curtain is quickly followed by the closing one, resulting in a moving slit, exposing the frame progressively. This can cause effects like a car or horse appearing slanted in the image. While it is continuous (no discrete lines like digital), it is not the whole picture exactly at once.