Understanding SDI video data format

[Yansi]

Hello,

I am having trouble understanding how the SDI video links work at the lowest layer. I have obtained a copy of the SMPTE-259M (SDTV SDI) specification. I understand, data is sent by means of 10bit symbols. I get it, that the serial bit stream is scrambled to increase bit entropy, that it is NRZI encoded so that clock can be recovered more easily.

But how the hell is one supposed to synchronize the transmitter and receiver?  So that the descrambler returns back the correct data at the correct time, that it won't fail from a random bit-slip or glitch, etc.

I just don't get it, how that is accomplished.  Or is one supposed to use a state machine to catch the SAV/EAV word triplet containing 3FF 000 000 positioned anywhere in the stream, that may come at any point in time to the interface input?

Is there any other SMPTE standard that provides explanation, or am I missing something completely?

Regards,
Y.

[Yansi]

Maybe I haven't uses the correct google-fu, I have just found this: https://www.cs.york.ac.uk/rts/docs/Xilinx-datasource-2003-q1/appnotes/xapp298.pdf

Looks promising, currently in the process of digesting this appnote.

//EDIT: Kchool! It seems to address all of my questions so far, including a nifty one about a parallel scrambler implementation.

[hamster_nz]

I"ve written an SDI RX and TX modules. IIRC The scrambler self-synchronizes - it just works.

[hamster_nz]

Oh, given that Xilinx AppNote I could write you some C code to play with if you like....

[helius]

At a higher level, there are many flavors of video and/or audio transmitted over SDI, so just implementing the lowest bit-packing level does not mean that devices are compatible. For standard-definition, there are both NTSC and PAL versions in both component video (D1) and composite (D2).

[Yansi]

I"ve written an SDI RX and TX modules. IIRC The scrambler self-synchronizes - it just works.

I am still having trouble understanding how a LFSR can self-sync. But maybe it just works. Somehow. 

I am yet to try to write an SDI TX or RX softcore.

At a higher level, there are many flavors of video and/or audio transmitted over SDI, so just implementing the lowest bit-packing level does not mean that devices are compatible. For standard-definition, there are both NTSC and PAL versions in both component video (D1) and composite (D2).

Sure, I know about that. Not currently sure where the difference is between the composite and component signalling (I am yet to do some google-fu into that), but mainly I am interested now in the ST259M-C (270Mbps), which I could potentialy interface with a lot of stuff directly.

//EDIT: It seems that SMPTE 125M and 267M are nowhere to be found on google. Or my google-fu is weak today. Shite!

[januszb]

I shouldn't worry too much about the composite variant of SDI.  At least here in Europe where PAL is used (I work in film/TV) the quasi-totality of equipment works with the component variant (the first paragraph of Annex B of ST259 states that as well).
Component signals separate each frame into Y (luma), and Cb Cr (chroma components - the blue and red colour differences).  YCbCr can be derived from RGB via a matrix + eventual gamma correction (see ITU Rec BT 601(SD) and 709(HD)) .  Composite signals combine both luma and chroma into one composite signal, with the chroma on a high frequency carrier and phase shifted per alternate line (PAL).  D2 is simply the digital representation of this signal, but is never used today.

Janusz

PS PM sent.

[Yansi]

Thank you,

I kind of understood the component representation, but still am not sure, how a digital composite video signal is represented. But, I will gladly skip this for now, as also as you write, this stuff is no longer used or supported and I will have enough trouble understanding other things I need. (Stupid question: Isn't it just the PAL composite signal sampled directly?)

I still have kind of confusion in my head about what the exact differences between YUV, YCbCr and YPbPr are, I should probably study more more about that in the ITU-R BT.601 spec. There is also an extensive article about this on Wikipedia, so I will give it a go.

Now I need to familiarize myself with the FPGA and start working on a PCB, so I can start experimenting.

I was thinking about using the cheaply Cyclone IV (EP4CE6E22C8N or EP4CE10..) to make a simple toy board, that should probably utilize the following:
Some 16b 100MHz+ SDRAM,
SDI 259M output,
SDI 259M input (likely two of those?)
CVBS input sampled using TVP5150 (have couple of those),
RGB output (probably DVI-I, have some chips to interface this too).
And some miscellaneous IO, whatever will left.

>=100MHz x16 SDRAM should have enough bandwidth for this application I guess (at x16 width, one video signal is just 13.5MTU/s)

Is this a good or bad idea or why? Any things to consider? Any interesting considerations for useful peripherals round the FPGA?

(After reading a couple of appnotes about SDI receivers, it seems they mostly use external VCOs + loop filters with a phase detector within the FPGA fabric to recover the incoming clock, so I am counting with this also).

[hamster_nz]

If you have any desire to store and read HD video from SDRAM the required memory bandwidth ramps up pretty quickly, esp if you convert 422 pixels to 444 (so each pixel has all its colour data).

[coppice]

I"ve written an SDI RX and TX modules. IIRC The scrambler self-synchronizes - it just works.

I am still having trouble understanding how a LFSR can self-sync. But maybe it just works. Somehow. 

Most LFSR scramblers used for whitening in comms protocols self synchronise.

[Yansi]

All I plan to have available are up to 3 video inputs and two outputs.

I do not think, I need to convert from 422 subsampling to 444 or even pure 24bit RGB, to attempt any kind of seamless video switching.

I intend the memory to only serve as framebuffer for synchronizing all inputs to a common output.  The most memory intensive thing I can currently imagine, is writing all 3 inputs simultaneously, while reading all 3 at once.   That means 6 streams of 13.5MTU/s, about ~81MTU/s.  Even cheap SDR-SDRAM can be clocked well above 100MHz, leaving probably still enough BW to accomodate for the overhead and bank switching.

Please mind you I am an FPGA and VHDL beginner and certainly won't be doing any fancy stuff with it soon, so every little bit will be a good success. 

All available video inputs will be 422 subsampled, the only SDI output will also be subsampled. I (currently to my best imagination) think I will not need to convert the color space nor the subsampling to full 444 samples.

[hamster_nz]

Are you sure you have those memory calculations are 'in the ballpark' right? They seem very low... Care to show your workings?

[Yansi]

One SDTV video is just 27M x 8 per second data in the active part of the video (13.5MHz luma+chroma sample pairs). I will only use 8bit sample resolution, for simplicity. Will not use full 10bit precision of SDI.
One outcoming video from the memory is the same bitrate.

I plan to have only 3 video inputs to the FPGA (2x SD-SDI, 1x CVBS analog through TVP5150, which provides the same 8-Bit YCbCr 4:2:2)

I need to put at least two of those video inputs into a long memory buffer, to be able to sync the video inputs together. So we have at least two streams writing to memory (likely all three).

I need at least two streams reading out of the memory: One for the main video output (SD-SDI encoder), one for what might end up as multi-view output.  Three output streams, if I'd want to mix/colorkey/whatever two outputs together.

Giving in total: 3x read stream, 3x write stream,  27MByte/s each.  Having a 16bit wide SDRAM, I need 13.5M transfers of data each second. Times 6 streams at once equals about ~81MTU/s or 162 MByte/s.

A cheap 2Mx16x4 SDRAM such as IS42S16400 can be clocked up to 166MHz for the -6 speed grade.  So either I could run it at 5x or 6x the video clock of 27MHz to remain synchronous with required video data rates, 135 or 162Mhz for the -7 or -6 speed grades respectively.

I know that determining the real data throughput of an SDRAM is a complex task, requiring precise knowledge of how it will be used. Keine Ahnung right now, but requiring only  162MByte/s maximum from a memory with theoretical bus speed of up to twice that (16bit wide bus) sounds reasonable, doesn't it?

[hamster_nz]

162MByte/s maximum from a memory with theoretical bus speed of up to twice that (16bit wide bus) sounds reasonable, doesn't it?

It does sound reasonable. I was fretting that you were targeting something other than SD-SDI which is 720x576@25Hz, or 270Mb/s.

For the SDRAM throughput calculations, you need to have refresh cycles, which will eat up about ~1% of bandwidth, and also random reads and writes are relatively slow on SDRAM - each time you move to a new row you have issue a "precharge" and the "active" command, taking about half a dozen cycles or so before you can issue the 'read' or 'write' command.

Because of this you have to batch writes that are in the same area of RAM to get good performance - optimal when streaming data would be writing whole columns at a time - giving 6 cycles of overhead over 256 cycles of data transfer, or about 2.5% overhead.

As a back of an envelope calculation, a well designed system might loose 10% of the bus bandwidth to overheads (six cycles of overhead for transferring 64 words). A poorly designed system, which does random single-word accesses might realize less than 20% of the data bus bandwidth.

Given that you are writing multiple streams into differing addresses, it would take a lot of design work (i.e. deep data FIFOs, bus arbiters, and optimally aligned transfers) to get really good results. I would plan on having 50% more bandwidth than you intend to use for your application. You will most likely need enough to have three frames for each source in the buffer (one being written to, one being read, and one either waiting to be filled or emptied). At 0.5MB per frame that needs 4.5MB of SDRAM - so that 64Mb part seems ideal.

The most challenging part would be what to do when the incoming streams are running at a few parts-per-million difference from the local clock. For the video you will either have to drop or duplicate frames, but that would upset the embedded audio streams...

[Yansi]

In the end I would sure like to attack the HD/3G-SDI standard, but small steps first. 

I am kind of familiar with SDRAM operation. I have never wrote any SDRAM driver in VHDL, but have worked with SDRAM controller equipped microcontrollers in the past, so the principles of the activation/precharging and working with banks is kind of familiar. This HW exercise will certainly be a beneficial extension of my practical knowledge.

Yes of course, the SDRAM would need to be read in bursts or even whole rows, while using the internal RAM blocks in the FPGA as a short buffer stage to smooth out those time where SDRAM can't provide data. But that will be some future project to hack together.

First I need to design the PCB for it, hence need to figure out all required stuff first. What I currently miss and is my huge knowledge gap, is how to recover the receiver clock from the incoming bit stream.

I have found a bunch of relevant appnotes from Xilinx, which shows either external CDR (pretty expensive and obsolete chip) or internal one, while using an external VCO and loop filter.



I guess this is the only reasonable way to get it done, without using unobtainium old ICs or hell expensive overkill ones.  What I don't see or understand, is how the phase detector is made inside the FPGA fabric.

Brief google-fu session brought me to "bang-bang" or "Alexander" phase detector, which seems to be suited for recovering  clock from a NRZI encoded streams:



However, I see quite an issue in the fact one of the flip flops needs a negative clock edge (almost impossible to do in an FPGA I guess), so not sure how to attack this one.

Neither do I see how the PLL can get synced to place the data sampling in the middle of the incoming data eye,  nor how the loop filter shall be designed (what bandwidth? etc..)


Most "cheap" video mixers/switchers usually don't work with genlocked SDI data sources. What is the common solution to the drift problem in these devices?

//EDIT: Or do I really need to recover the clock at all? I only need the recovered data. Here is an interesting reading: https://www.xilinx.com/support/documentation/application_notes/xapp224.pdf

[BrianHG]

The most challenging part would be what to do when the incoming streams are running at a few parts-per-million difference from the local clock. For the video you will either have to drop or duplicate frames, but that would upset the embedded audio streams...

I personally used a TI 27MHz XTAL PLL VCO input reference oscillator in my designs.  I tune it slowly with a phase detector on the VS from the chosen source video with my output video VS.  Never had trouble with switching sources while never breaking the output video.  Video blending is a different story.  I would just drop or add a fraction of an audio sample every VS to keep my reference output 48KHz digital audio clock.  Loosing/gaining a single field or frame once every few minutes or so is never noticed when blending 2 async sources.

[Psi]

I am still having trouble understanding how a LFSR can self-sync. But maybe it just works. Somehow. 

I know nothing about SDI but self syncing protocols can be pretty trivial to do.
It doesn't need to be super complicated.

At a basic level you can simply offset by 1 bit every time the frame structure is totally invalid.
So, if it gets out of sync, it automatically re-syncs after n increments.
This can be limited so it doesn't happen on a little random bit corruption but does happen if the entire structure makes no sense.

I have no idea if this is how SDI does it, i'm just saying that self syncing can be super simple.

[Yansi]

I would not like to dive into the math involved around the LFSR, I'd rather stay with "it just works" and leave it alone.

Currently I need to solve the mystery of data (and clock?) recovery. 

//EDIT: see below how SDI does the de/scrambling. There shall be a unique character for sync, 3FF 000 000 (10 bits high, 20 bits low).

[Yansi]

The most challenging part would be what to do when the incoming streams are running at a few parts-per-million difference from the local clock. For the video you will either have to drop or duplicate frames, but that would upset the embedded audio streams...

I personally used a TI 27MHz XTAL PLL VCO input reference oscillator in my designs.  I tune it slowly with a phase detector on the VS from the chosen source video with my output video VS.  Never had trouble with switching sources while never breaking the output video.  Video blending is a different story.  I would just drop or add a fraction of an audio sample every VS to keep my reference output 48KHz digital audio clock.  Loosing/gaining a single field or frame once every few minutes or so is never noticed when blending 2 async sources.

I understand that if you want to phase lock to a reference. But I need to extract data (and clock?) from the SDI stream first.  i am not sure how to extract the asynchronous serial data from the stream.

I see only two viable options:

1) Use a dedicated PLL (external loop filter + VCO) per each SDI input. (Not sure how to make the phase detector logic and it seems to me a bit clumsy, as I do not need the clock for anything else)

2) Oversample the input data by 4 (using a quadrature clock), which may also work, as per Xilinx XAPP224 and extract the data only (which is what can be done for multiple inputs, without dedicated PLLs). Bit more complicated, but no fiddling with PLLs.

[Yansi]

So this is what I was thinking about doing.

If you count the pins, it is likely all the FPGA (144pin LQFP) can offer.  Those few left will be brought out as generic IO, or fitted with some basic useful peripherals of the LED/button/switch type.

If you happen to have any tips on better modern cable equalizer or driver chips for SD-SDI, please let me know. CLC014 and CLC006 are quite old bastards and run from 5V, but can be obtained for $2. I would not like to use any more expensive than that. That 5V supply seems so far the only disadvantage which I can live with.

It is great that Texas offers a replacement LMH0074 for the CLC014, but priced at $18 they can shove them somewhere. One can get a pretty decent FPGA for that price, even couple of the modern 3GHz equalizer chips. Both CLC014 and 006 seem to be available from the eastern sources for about $2.

So, here is what I was thinking about:



//EDIT: I am also thinking about adding an I2S audio codec, if I happen to decide to play with embedded audio within the SDI data streams. Also adding one more SDI output will not hurt (CLC006 costs peanuts and will require just a bit board space for the extra BNC).

[Ice-Tea]

Is this for an actual product? The TVP5150 is obsolete. I've had decent results with ADV7280's or something from that series..

[coppice]

I would not like to dive into the math involved around the LFSR, I'd rather stay with "it just works" and leave it alone.

Note that ehe encode side feeds back, but the decode side only feeds forwards. Whatever goes into the "encoded video in" terminal has completely cleared from the "decoded video out" terminal after just 10 clock cycles. Therein lies its ability to self synchronise.

[BrianHG]

2) Oversample the input data by 4 (using a quadrature clock), which may also work, as per Xilinx XAPP224 and extract the data only (which is what can be done for multiple inputs, without dedicated PLLs). Bit more complicated, but no fiddling with PLLs.

If all you are worried about is 270mbit, 4x clock the input is doable.  Also, if you have multiple unused PLLs in the fpga, you can trick them into each running an SDI input even if they weren't designed for serial decoding.

Running a fixed 540mhz serial shift inputs, 2 in parallel per input, 1 on nclk and the other on pclk, basically a DDR ram input, will give you the equivilant to a 1080mhz clocking fpag even if the fpga max is only in the 540Mhz range.  You need additional logic to select which n or p input is currently in phase dynamically.

Also, the best solution, there exist generic deser clock recovery ICs which will take in SDI among many formats, output a CLK and 1,2, or 4 bits where you still need to deser the rest in the FPGA, but all the clocking PLL and really high speed signals are taken care of allowing you even higher speeds like 1.5gb and 3gb using only 2-5 FPGA inputs per SDI channel.  Same for HDMI, but the IP for that 1 costs big $.

[Yansi]

Yes, an actual product I will build and play with. I don't care if some of the parts are obsolete, all I care about is to get the most fun out of it while keeping the price low. It won't earn me any $, but gain experience and knowledge, which is why I am doing this exercise. I find the usual FPGA "blinkenlights projects" quite boring, hence came up with this, to make something more unique and probably also useful in the end.

TVP5150 does not seem obsolete: http://www.ti.com/lit/ds/symlink/tvp5150am1.pdf
Can be sourced from the usual sources from the east for less than $1, unlike the ADV7280, which costs more than I have payed for the FPGA.  But sure, the ADV7280 has 4 analog inputs and likely a better ADC, and can do YPbPr, which may be beneficial for some cases.

What I have in my stash is likely the "NRND" labeled TVP5150PBS. Not sure what the exact difference between TVPV5150-PBS and -AM1 is, likely only some minor software discrepancy between them.

This time I am just reusing what is already in my stash, which is both TVP5150 and the Chrontel CH7302A DVI transmitter, which I like for the reduced/multiplexed data bus, which in this case helps keeping the pin count low.

I do not want to complicate the HW unnecessarily, as my primary learning objective is the FPGA itself and would like to reuse parts I already happen to have.

As for the audio codec, I will use likely a WM8731L, also reusing some leftover stock from other projects.

[BrianHG]

2) Oversample the input data by 4 (using a quadrature clock), which may also work, as per Xilinx XAPP224 and extract the data only (which is what can be done for multiple inputs, without dedicated PLLs). Bit more complicated, but no fiddling with PLLs.

If all you are worried about is 270mbit, 4x clock the input is doable.  Also, if you have multiple unused PLLs in the fpga, you can trick them into each running an SDI input even if they weren't designed for serial decoding.

Running a fixed 540mhz serial shift inputs, 2 in parallel per input, 1 on nclk and the other on pclk, basically a DDR ram input, will give you the equivilant to a 1080mhz clocking fpag even if the fpga max is only in the 540Mhz range.  You need additional logic to select which n or p input is currently in phase dynamically.

Also, the best solution, there exist generic deser clock recovery ICs which will take in SDI among many formats, output a CLK and 1,2, or 4 bits where you still need to deser the rest in the FPGA, but all the clocking PLL and really high speed signals are taken care of allowing you even higher speeds like 1.5gb and 3gb using only 2-5 FPGA inputs per SDI channel.  Same for HDMI, but the IP for that 1 costs big $.

You are looking for ICs like these, maybe other ones better suited for SDI also exist: https://www.analog.com/en/products/clock-and-timing/clock-data-recoveryretiming.html  Most of these guys have a built in cable length equalizer as well.  The cheapest ones with any data clock rate, meaning SDI capable, are around 25$.