Cheap 250Mbs Link between Boads

[ali_asadzadeh]

Hi,
I want to design a new project for the customer, I want to send Data between boards with the least amount of wires, there are around 200 boards and they should send their data to the next Board, and the data rate is around 250Mbs, also the boards are around 50-60cm away from each other,
I was thinking of using RS485, But the data Rate is limited to 50Mb, so I was searching other options, Ethernet is out of question because of the cost, So I thought maybe Select IO from spartan 6 parts Like the LX9 part can come handy, is there a way to make a UART liked interface @250Mbs, what about the Physical layer? should I go with LVDS? any Ideas or hints would help a lot.

[laugensalm]

You might want to forget any asynchronous solution and use a 10b8b coding on the physical layer, as DVI does. If you're not stuck with Xilinx, you might want to look at the gear boxes in the mach XO[2-3] FPGAs, there are also neat ref designs that work over a Ethernet cable (without the Ethernet protocol). I've also abused DVI-alike protocols over SATA cables and HDMI (off standard) for such hacks with no issues. And not to forget some real time issues with deviating clocks on sender/receiver, but I assume you've thought about that already.

[glenenglish]

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

why would you even consider an Spartan6 , unsupported tools ?? , unsupported silicon
You are designing a product for a customer ! not a doorbell for your house...
There is plenty of QFP  FPGA around.... Trion T20  .... , Lattice XO2,

[BrianHG]

Even every single slowest worst lowest power Altera FPGAs can achieve those bit-rates on every pair of IO pins.

[glenenglish]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

[BrianHG]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

There is always fiber optic.... LOL, talk about overkill.

[langwadt]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

[SiliconWizard]

Cost of Ethernet? How so? Gigabit Ethernet PHY+mag jack can be pretty cheap. We don't know much about your system, does each board already have a FPGA? A MCU? A SoC?
Even a CH32V307 has a Gigabit MAC, so with an external PHY + mag jack, you'd get your (robust) data link for a couple extra bucks per board (and the CH32V307 also has a FMC that can be used to implement a FIFO to some other MCU or SoC. Of course other more powerful MCUs can support that too.

Otherwise yes, a modest FPGA and a few IOs set up in LVDS mode.

[BrianHG]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

It's a balanced signal with no continuous carrier clock, it's seen as weak spread-spectrum noise.

[langwadt]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

It's a balanced signal with no continuous carrier clock, it's seen as weak spread-spectrum noise.

and so can LVDS be ....

[ali_asadzadeh]

Cost of Ethernet? How so? Gigabit Ethernet PHY+mag jack can be pretty cheap. We don't know much about your system, does each board already have a FPGA? A MCU? A SoC?
Even a CH32V307 has a Gigabit MAC, so with an external PHY + mag jack, you'd get your (robust) data link for a couple extra bucks per board (and the CH32V307 also has a FMC that can be used to implement a FIFO to some other MCU or SoC. Of course other more powerful MCUs can support that too.

Otherwise yes, a modest FPGA and a few IOs set up in LVDS mode.

I was considering CH32V307, But Since I need two such ports, One receiver that gets it's data from last board in the chain and one transmitter that would add it's own data to the received one and resend it to next stage so I moved on it!

There is always fiber optic.... LOL, talk about overkill.

I have seen some lower speed parts,
https://www.broadcom.com/products/fiber-optic-modules-components/industrial/industrial-control-general-purpose/650nm
But since I need the system to be as cheap as possible, I forget about them.

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

Do you suggest any app note, code or open source project for that or You mean I try doing something like a SPI based system? the clock and Data! this seems not a bad option, But I prefer to use only two pairs of wires to make it even simpler.

You might want to forget any asynchronous solution and use a 10b8b coding on the physical layer, as DVI does. If you're not stuck with Xilinx, you might want to look at the gear boxes in the mach XO[2-3] FPGAs, there are also neat ref designs that work over a Ethernet cable (without the Ethernet protocol). I've also abused DVI-alike protocols over SATA cables and HDMI (off standard) for such hacks with no issues. And not to forget some real time issues with deviating clocks on sender/receiver, but I assume you've thought about that already.

I'm not stuck to AMD, I'm open to options, But I like AMD and Gowin parts, I have not done 10b8b coding before, So I'm a nobe in this area, any open source project or app note do you suggest?

[langwadt]

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

Do you suggest any app note, code or open source project for that or You mean I try doing something like a SPI based system? the clock and Data! this seems not a bad option, But I prefer to use only two pairs of wires to make it even simpler.

it is two pairs, one pair for data, one pair for clock

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

[glenenglish]

The gigabit ethernet is a controlled , band limited waveform.
it's not super fast hard edges like LVDS.
the LVDS drivers will do at 1000 MHz.... and with rise times to match.

even band limited, the clock on LVDS would need real shielding
HOWEVER, at 125 MHz LVDS (250 Mbps) , the FPGA could easily operate its own clock recovery, so you would not need to transmit clock, just data.....  so, just shielded CAT-6.
You could do DPLL clock recovery . Or you could transmit manchester.... but manchester would generate a clock line which you are trying to avoid ....
Glen.
RF engineer.

[mariush]

Some clarification would be helpful.... do you need that 250mbps because data has to reach the last device in the series in some reasonable time ...

I have a suspicion that you're making some RGB led tiles or something like that, for a big display, and you're trying to push 2k-4k pixels x a few hundred lines worth of information so many times a second.

Not sure why you think ethernet is too expensive and complicated... you can get used 48 port 100 mbps ethernet switches for something like 30-50$ , you'd only need 4 of those in a stack to control your 200 boards and you'd also be able to use ready made patch cables to limit the problems caused by making manual cables.

ethernet chips are like 1-1.5$ if you get 100+, one per card.

I'm assuming that your bandwidth would be much less than 100 mbps if you don't have to pass the data to the next board and so on... you could have a beefier fpga split the data into chunks for each board and do broadcast or push the data through udp and the switches would send only the data you want to individual boards.

At 100 mbps you could have the unused pairs repurposed for sending power to each board, like in passive poe  (one pair 12v , one ground, something like that)

[ali_asadzadeh]

thanks for the feedbacks,

Some clarification would be helpful.... do you need that 250mbps because data has to reach the last device in the series in some reasonable time ...

Yes, I need to reach this, Because every board should add it's data to the last part of the stream, and as the stream passes from the first board to the last board, it would get bigger, and they should do it in a limited time, since new data would arrive at fixed time, so The 250mbps is a must to meet all the required timings.

Not sure why you think ethernet is too expensive and complicated... you can get used 48 port 100 mbps ethernet switches for something like 30-50$ , you'd only need 4 of those in a stack to control your 200 boards and you'd also be able to use ready made patch cables to limit the problems caused by making manual cables.

Note that the space the boars suppose to have is limited, so I can not use ethernet cable for every board to a switch then get the final result, also money wise, it does not make sense either, since I need to use lot's of wire to run things.

HOWEVER, at 125 MHz LVDS (250 Mbps) , the FPGA could easily operate its own clock recovery, so you would not need to transmit clock, just data.....  so, just shielded CAT-6.
You could do DPLL clock recovery . Or you could transmit manchester.... but manchester would generate a clock line which you are trying to avoid ....
Glen.

Would you please explain more on how to recover clock, since I have not done related projects and I would be a noob there.

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

[BrianHG]

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

You have to go through each vendor's data sheets, read their LVDS transceiver's electrical section for wiring and loading the IO and then read and select the SERDES IP primitive which is right for your clocking and coding requirements.

Otherwise, we are at a point where you are asking us to do your homework/research and funnel you a bunch of vendor's .pdfs and page numbers.


You can also use single ended coax, like BNC or smaller RF connectors at 250mb for a good 100cm without cable-length equalization amplification as these solutions with proper termination will achieve above 3gbps.

Don't forget to diode protect your ports.

[langwadt]

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

oldschool 12Mbit USB got away with 4x sampling, either with a 48MHz SDR or 24MHz DDR, A spartan6 can definitely do much more then 100MHz DDR, though 500MHz DDR might b pushing it.

how are you going to wire it? why is limiting the number of pairs so important?  repurpose cables from something like HDMI, DP, USB-C, hell even RPi camera/display flatflex, and you'll get plenty of pairs cheap and easy

[glenenglish]

I think at this point, you probably need to read some books, do some tutorials,

maybe try this project you want at say 1/4 the bit rate here, say 62.5 Mbps, and learn about the process, then try 250 Mbps. This way you will learn and conquer each speed raising step and the associated challenges.

and even $4 Trion T8  FPGAs (can get in QFP) can reach 300 MHz clock and 400 Mbps LVDS ....and 250 Mbps  DDR  only needs 125 MHz clock at the edge, and internally, 8 bits wide, 31.25 MHz clock.

I know your blog signature says "I'm a Digital Expert from 8-bits to 64-bits" but, if this is a commercial project, I would suggest you employ a consultant to get you up to speed .

[mtwieg]

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

[glenenglish]

sata cables - that's a good idea.
yeah clock recovery for a 125 MHz clock using a DPLL should be a cinch for an experienced  fpga person.

otherwise.....1 gig ethernet could do it. or serdes. . the op can buy  5:1 serdes gigabit stream chips from TI  . (LVDS to serdes) . The are all higher latency, intensive overkill options.

but, I think given the op is a beginner, data and clock on two LVDS  shielded pairs would be the easiest. IE 1 x sata cable - cheap and cheap connectors as you pointed out mtwieg.  Use a Trion T8 (6 lvds in each direction per QFP, $5) .  Obviously bigger FPGA if more work or memory is required in the FPGA.
 

[langwadt]

sata cables - that's a good idea.
yeah clock recovery for a 125 MHz clock using a DPLL should be a cinch for an experienced  fpga person.

otherwise.....1 gig ethernet could do it. or serdes. . the op can buy  5:1 serdes gigabit stream chips from TI  . (LVDS to serdes) . The are all higher latency, intensive overkill options.

but, I think given the op is a beginner, data and clock on two LVDS  shielded pairs would be the easiest. IE 1 x sata cable - cheap and cheap connectors as you pointed out mtwieg.  Use a Trion T8 (6 lvds in each direction per QFP, $5) .  Obviously bigger FPGA if more work or memory is required in the FPGA.

I think if you are really penny pinching you could do SATA as edge connectors in the PCB, though I think you need slightly thinner than the usual 1.6mm PCB

[ali_asadzadeh]

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

I know I can use SERDES  to solve it all, the problem is the cost, I think I would go for XC6SLX9-2TQG144I part from AMD, since it's one of the cheapest Industrial parts available, unless you suggest something else with similar or better prices, (they are around 8$ @ 100 units)
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_AMD-XILINX-XC6SLX9-2TQG144I_C415799.html

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

Thanks for the suggestion, It seems a very good choice indeed.

[glenenglish]

why do you need industrial parts ?
sure you can use an old  obsolete part in a new customer product... but really, should you ?
(where there are plenty of good  an dmodern $4 parts like I gave examples of in QFP that are much faster than spartan 6 )

serdes is a ridiculous overkill. and might not run slow enough to work for you down your available cable....

[langwadt]

why do you need industrial parts ?
sure you can use an old  obsolete part in a new customer product... but really, should you ?
(where there are plenty of good  an dmodern $4 parts like I gave examples of in QFP that are much faster than spartan 6 )

serdes is a ridiculous overkill. and might not run slow enough to work for you down your available cable....

spartan6 has suspport until atleast 2030

serdes is just  a fancy shift register, you can run as slow as you like

[BrianHG]

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

I know I can use SERDES  to solve it all, the problem is the cost, I think I would go for XC6SLX9-2TQG144I part from AMD, since it's one of the cheapest Industrial parts available, unless you suggest something else with similar or better prices, (they are around 8$ @ 100 units)
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_AMD-XILINX-XC6SLX9-2TQG144I_C415799.html

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

Thanks for the suggestion, It seems a very good choice indeed.

All Xilinx, Lattice, Altera parts have built in SERDES capabilities in every one of their parts on almost on every one of their IOs.  Read the data sheets.  You do not need the expensive ones with the dedicated multi-gigabit dedicated SERDES pins.  The difference is that the cheaper IO pin's SERDES capabilities are limited to 500-900mbps.  Also, you can use the free dev tools when running the cheap 500-800mbps IO pins.

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.

[glenenglish]

serdes is just  a fancy shift register, you can run as slow as you like

I think we need to clarify : common terminology is that :

"SERDES"  or  "transceivers" are  a clock recovering high speed differential serial interface block, usually with 8b/10b or 64/65 etc encoding and pattern detection.   They are not in every fpga. They usually have a minimum bit rate that clock can be reocvered- the range of VCO frequencies supported. Usually about 500 Mbps

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

[langwadt]

serdes is just  a fancy shift register, you can run as slow as you like

I think we need to clarify : common terminology is that :

"SERDES"  or  "transceivers" are  a clock recovering high speed differential serial interface block, usually with 8b/10b or 64/65 etc encoding and pattern detection.   They are not in every fpga. They usually have a minimum bit rate that clock can be reocvered- the range of VCO frequencies supported. Usually about 500 Mbps

Xilinx call those GTP, https://docs.amd.com/v/u/en-US/ug482_7Series_GTP_Transceivers

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

Xilinx call those serdes, https://docs.amd.com/r/2021.1-English/ug953-vivado-7series-libraries/ISERDESE2

[Someone]

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

Xilinx and Altera both call that SERDES.

[hamster_nz]

Possible but a lot of work:

1. 8b10b coding (+25% overhead), 64b66b+scrambler coding (+3% overhead) or TMDS coding (25% overhead) on the link so you can recover framing

2. Oversample the input signal by 2x so you can identify the data edges and track them. Find for standard SERDES blocks on most FPGA I/O pins. No high speed transceiver needed.

3. Use the fine phase shift of the clock manager in a control loop to recover timing from the bit stream.

4. Use the TMDS_33 (aka HDMI) IO standard for your physical layer.

[BrianHG]

4. Use the TMDS_33 (aka HDMI) IO standard for your physical layer.

Nope, unless you are driving a TMDS terminated receiver, do not use TMDS_33.
There are other, lower power, easier to terminate differential balanced transmission line options exist compatible with your FPGA IOs which can run 500mb on a 100cm cable like you see in serial ATA HD cables.

Again, read your FPGA's electrical specifications data sheets.  They illustrate example LVDSIO<->LVDSIO wiring diagrams with example termination resistors, supported cable lengths and maximum bit-rates.

[ali_asadzadeh]

Thanks all for your input and feedback,
BrianHG I think you have lot's of exprince in this field,

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.

Would you please explain more, give some links ,app notes etc... I have seen the Select IO wizard IP block in Xilinx, and I think you might suggest this path, am I right?

[BrianHG]

Thanks all for your input and feedback,
BrianHG I think you have lot's of exprince in this field,

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.

Would you please explain more, give some links ,app notes etc... I have seen the Select IO wizard IP block in Xilinx, and I think you might suggest this path, am I right?

I can easily help you with Altera/Intel.  There are others here better suited for Xilinx.

You should start here: https://docs.amd.com/v/u/en-US/ug381
Read pages 33-37...  (Actually, just read the entire data sheet as it explains the SERDES IP as well...)
I was wrong, you can also use TMDS_33 with Xilinx.

However, you need to match the drive and termination wiring based on the type of cable you will be using.

For this, you will need to google the spec/impedance and wiring of shielded SATA cables and choose which IO wiring on page 34 of the Xilinx .pdf will get your data from FPGA to FPGA.

The you will need to choose how to clock your system.
SATA has 2 pairs of data signals.

If you are sending data in one direction, the easiest dumb solution is to have 1 pair transmit a 25 mhz clock to the next FPGA PLL reference input while the second pair runs serial data at 10x the clock.

The other solution is to embed a clock in your data.  This one is more a feature set function related to Xilinx's IP SERDES blocks.  I know you can use any differential IO pair for transmit, but for maximum clock recovery functions, you will need to consult the Xilinx data sheets.

My really old school extra dumb method for embedding a clock with Altera FPGAs would have been to transmit 1 bit high, 1 bit data, 1 bit low, 1 bit data, 1 bit high, 1 bit data.... at 500megabit, clocking the second FPGA with every second bit (my 1 bit high/low, a 125mhz clock) while having a 250mb data channel.  However, this required a little PLL trickery.

More modern embedded clock recovery techniques exist which should be available to you, but if you use a separate clock channel, things can be made to guarantee the dumb separate clock method.

I do not know the type of SATA wire being used, if it is twisted pair, or 2 individual shield cables.

If it is 2 individual shielded/coax connections, you could also use 1 cable as a 25mhz clock, the other as 250mb data, wiring everything in single ended instead of differential.

[ali_asadzadeh]

Dear BrianHG, thanks for clarifying things future

My really old school extra dumb method for embedding a clock with Altera FPGAs would have been to transmit 1 bit high, 1 bit data, 1 bit low, 1 bit data, 1 bit high, 1 bit data.... at 500megabit, clocking the second FPGA with every second bit (my 1 bit high/low, a 125mhz clock) while having a 250mb data channel.  However, this required a little PLL trickery.

I like your old school extra dumb method, can you share more on that, it sound a good solution to me, since  I'm only sending data in one direction.

[asmi]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

[langwadt]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

[Someone]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

Clean solution, but sadly violates the PLL input specs of most (at least all of the ones I'm familiar with) suitable FPGAs. 100 vs 110 would work for some and be relatively easy to close timing on.

[langwadt]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

Clean solution, but sadly violates the PLL input specs of most (at least all of the ones I'm familiar with) suitable FPGAs. 100 vs 110 would work for some and be relatively easy to close timing on.

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

[BrianHG]

Dear  ali_asadzadeh,

     Given all the above responses, a few possible routes exist for you.

     Xilinx is not my thing, but if I am to treat you as an absolute beginner, what I can help you with next is tying down your exact specification needs so you may choose which path forward you should go.

First if I were you , I would need to have answers for these questions before proceeding:

When you say 250mb, do you mean 25,000,000 bytes a second, or, 31,250,000 bytes a second?

For your connection, do you mean at least 250mb, with the data going on and off, or, do you mean it will be a nonstop stream at 250mb, never going slower or faster?

What is you source data, some FPGA core code running at 1/8 or 1/10th 250mhz, or code running slower, but you need a 250mb link to guarantee you can send every byte with a little headroom?

Can your destination FPGA's core run a little faster than 25/31.25 MHz to guarantee the core can receive every byte with the occasional blank space since the core is running a little faster than the data, or, do you need your destination FPGA to clock itself from the 250mb data so that the core runs in perfect sync?

These questions above will need to be answered so you may continue with in the right direction.

If all you wanted is parallel 8 bit in with a clock , through a serial port, then parallel out 8 bit data with a reconstructed clock, no FPGA, there exist dedicated IC serializers/deserializers with this function.  Being cheap and dirty, SDI serializer/deserializer will have all these functions build in with a 10 bit data bus, clocking at 27MHz.  (IE: 270mb serial stream, able to drive over 1 meter cables)

[BrianHG]

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

For 3 bit, you need 750mb.  So, for this trick to work, your PLL will need to run at 375 Mhz, driving the OSERDES2 internally in DDR mode to get that 750mb.  That 100/110 trick will create a 250mhz reference with a 33.3/66.6 duty cycle for the PLL input where you will need to run the PLL at 1.5x to get that 375mhz.

Does the Spartan6 have a manual override for the PLL loop bandwidth controls like what you see in Altera Cyclone PLLs?

Or, can you set the PLL to only trigger register exclusively on the rising edge of the input?  (IE: manually set the PLL controls input reference clock to divide by 2 so the PLL's phase comparator internally sees a 50/50 187.5MHz source.  (Lattice FPGA PLLs have a source clock divider option for their PLL's reference.))

Of, can you use your serial input tied to a single posedge clocked TFF, and use that toggling flip-flop's output as a perfect 50/50 187.5MHz PLL reference, running the PLL at 3x instead of 1.5x?  (This was my Altera serial embedded clock trick, and I did need to tune the PLL's phase to capture the middle bit error free.  I literally had my DFFIO tied to a GlobalClock PLL reference input with strict .sdc constraints.)

[ali_asadzadeh]

Thanks guys for the feedback,

When you say 250mb, do you mean 25,000,000 bytes a second, or, 31,250,000 bytes a second?

31,250,000 bytes a second or a 250mbps

For your connection, do you mean at least 250mb, with the data going on and off, or, do you mean it will be a nonstop stream at 250mb, never going slower or faster?

Each PCB has it's low speed ADC, so it would capture it's data and send it's data to next board, so the 250mb is for the last board in the chain, Basically each board capture it's data and will add it's data on top of the receiving packet from pervious board in the chain and resend it to the next board, so the last board should have at least 250mb speed so it can send it's data before new samples arrive.

What is you source data, some FPGA core code running at 1/8 or 1/10th 250mhz, or code running slower, but you need a 250mb link to guarantee you can send every byte with a little headroom?

The code inside the FPGA is simple enough and it would run at lower speed, I need a 250mb link to receive the pervious board data and a separate transmit line to send it to the next board.

Can your destination FPGA's core run a little faster than 25/31.25 MHz to guarantee the core can receive every byte with the occasional blank space since the core is running a little faster than the data, or, do you need your destination FPGA to clock itself from the 250mb data so that the core runs in perfect sync?

I have not decided yet, But I think I have explained enough what the boards will be doing, so I think running the actual Core around 100Mhz is doable inside the Spartan6, and it's fast enough and would give a lot of head room, the problem is the links for receiving and transmitting the stream of data.

[glenenglish]

Why use a Spartan 6 ?  They'll be EOL and unobtanium soon. Yes I know Xilinx said 2030, but AMD own it now.

[langwadt]

Why use a Spartan 6 ?  They'll be EOL and unobtanium soon. Yes I know Xilinx said 2030, but AMD own it now.

it was AMD before the extension, from the spartan6 front page:

"AMD takes our commitment to long lifecycles very seriously. We are pleased to announce that support is formally being extended for AMD Spartan™ 6 devices until at least 2030."

but I agree that you probably shouldn't start a new product design with spartan6 unless it is something where you can jsut buy all the parts you'll ever need right now

[langwadt]

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

For 3 bit, you need 750mb.  So, for this trick to work, your PLL will need to run at 375 Mhz, driving the OSERDES2 internally in DDR mode to get that 750mb.  That 100/110 trick will create a 250mhz reference with a 33.3/66.6 duty cycle for the PLL input where you will need to run the PLL at 1.5x to get that 375mhz.

Does the Spartan6 have a manual override for the PLL loop bandwidth controls like what you see in Altera Cyclone PLLs?

Or, can you set the PLL to only trigger register exclusively on the rising edge of the input?  (IE: manually set the PLL controls input reference clock to divide by 2 so the PLL's phase comparator internally sees a 50/50 187.5MHz source.  (Lattice FPGA PLLs have a source clock divider option for their PLL's reference.))

Of, can you use your serial input tied to a single posedge clocked TFF, and use that toggling flip-flop's output as a perfect 50/50 187.5MHz PLL reference, running the PLL at 3x instead of 1.5x?  (This was my Altera serial embedded clock trick, and I did need to tune the PLL's phase to capture the middle bit error free.  I literally had my DFFIO tied to a GlobalClock PLL reference input with strict .sdc constraints.)

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

[BrianHG]

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

Well, if the op can sacrifice 1-2 byte every fixed period, and it would help if he could sacrifice a 255 or a 0, this would help.  The other choice is to transmit 9 or 10 bits instead of 8 bits, giving him a initial start, index and stop bit.

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

I'm assuming he needs to clock his sampler at a fixed speed for clean sampling, some fixed divider of the 250mb clock.  I do not know how many of these chained samplers he will have running down the chain of boards, but it sounds like each boars samples an 8 bit number, like down at 112.1875khz.  The first boards adds a byte to the serial stream once every 256 bytes.  All concurrent boards sample at the same time, each adding their own additional byte to the stream, feeding out a copy of the incoming serial stream, adding their new byte at the end.  So, once you max out at 256 boards, you will have a continuous filled stream with 256x8 bit samples, 11187.5 times a second. 

@ali_asadzadeh, is this correct?
What is your individual sampler sample speed?
How many board will you be chaining?
Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

It also appears like the OP is using only 1 channel of each SATA cable.  I know ASMI said that the 2 pairs timing aren't guaranteed by the set standards, but with 2x 6gigabit channels and a plug with parallel clamped cables, having a second transmitter for a side data channel which doesn't need such precision phase timing is also at play.

[BrianHG]

Just for reference: https://datasheet.octopart.com/XC6SLX9-2TQG144C-Xilinx-datasheet-10025886.pdf

Page 17 -> OSERDES2 using a speed grade -2, for 3 bit, you can go up to 750Mb/s.  For 4-8bit, you get a little more allowing you 1050Mb/s.  I do not know if this means you are allowed 6bit at 1050Mb/s, but if it does, it can be leveraged for the extra bandwidth, or you will need to code 6 bit chunks sliding into an 8 bit pattern which is easy enough to do.

  If you must use a 4:1 or an 8:1, then you may generate a modified version of the 3 bit pattern 1d0/1d0 into a 4 bit pattern 1dd0/1dd0 where the [dd] represents a single bit of data being fattened for easier capture by the receiver.

The above datasheet also describes the limitation of the PLL and DCM (beginning page 49).  Using the DCM with the input divide by 2, then having it mult by 4, running the 4 bit fat 1dd0/1dd0 pattern would create your clock embedded 1Gs/s stream containing a synchronous 250Mb/s serial channel.

[langwadt]

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

Well, if the op can sacrifice 1-2 byte every fixed period, and it would help if he could sacrifice a 255 or a 0, this would help.  The other choice is to transmit 9 or 10 bits instead of 8 bits, giving him a initial start, index and stop bit.

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

I'm assuming he needs to clock his sampler at a fixed speed for clean sampling, some fixed divider of the 250mb clock.  I do not know how many of these chained samplers he will have running down the chain of boards, but it sounds like each boars samples an 8 bit number, like down at 112.1875khz.  The first boards adds a byte to the serial stream once every 256 bytes.  All concurrent boards sample at the same time, each adding their own additional byte to the stream, feeding out a copy of the incoming serial stream, adding their new byte at the end.  So, once you max out at 256 boards, you will have a continuous filled stream with 256x8 bit samples, 11187.5 times a second. 

@ali_asadzadeh, is this correct?
What is your individual sampler sample speed?
How many board will you be chaining?
Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

It also appears like the OP is using only 1 channel of each SATA cable.  I know ASMI said that the 2 pairs timing aren't guaranteed by the set standards, but with 2x 6gigabit channels and a plug with parallel clamped cables, having a second transmitter for a side data channel which doesn't need such precision phase timing is also at play.

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronized load of all the registers

[BrianHG]

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronized load of all the registers

Yes.
In [1b0] 750mb mode, all you need is 1 sync byte.
Or, in [1b0] 1000mb mode, you can blindly embed start-stop & sync as well as incorporate a parity bit to verify you are sampling the bit in the sweet-spot over every 2 bytes.

The second channel may then be used for reverse direction communications.

[glenenglish]

Brian, sounds llike you miss the days of SDI.
 given these cables are short (< 1m) , I think the use of a transmitted clock on another pair (like the SATA connector) would seem to be a no brainer. For 250 Mbps, I would stick to a transmitted 125 MHz LVDS clock... Perhaps 136 MHz  (272 Mbps) to stay out of the aircraft band...
Its a slow enough data rate that having the clock at the data rate will eliminate the need to resolve the ambiguity.

Of course, using a really slow clock would enable using the edge of the slow clock as a frame / posiiton marker, perhaps.

Ths leaves the requirement for sample rate conversion from local clock domains into the passed packet steam, unless the whole show is synchronous with the beginning of the chain source clock.

Might be useful to do a cascaded jitter analyssssis of a passes through clock, depending on the number of segments, there might become a problem there.

 

[ali_asadzadeh]

Guys thanks for the feedback,
BrianHG thanks for the inputs, I think I'm total noobe in this area, I know PLLs would get an input clock and have a multiplier and a divider to generate another clock in it's simplest from, But I do not get the points  regarding this

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

Please note that nothing is written on stone and I should suggest the customer the solution, so if anything would make sense to me, I would suggest it.

What is your individual sampler sample speed?

Each board would sample around 10Ksps and it has around 125bits of data, so if we need more bandwidth to add additional bytes or info to add frame sync etc... We can go higher in speed, since the IO can go to 750Mbps easily.

How many board will you be chaining?

200 boards

Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

No, the actual sampling speed is around 10Khz, But each board have some ADC and different Sensors, so the total data is around 125bits, if adding additional bit's would help, we would add them.

In [1b0] 750mb mode, all you need is 1 sync byte.
Or, in [1b0] 1000mb mode, you can blindly embed start-stop & sync as well as incorporate a parity bit to verify you are sampling the bit in the sweet-spot over every 2 bytes.

I do not understand these too, I'm too noobe

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronised load of all the registers

I like the Idea of seeing it as giant shift-register, But how? since all the boards should be synced in sampling time.

[BrianHG]

When we say '1b0', what we are saying is if you look as the serial output on a scope, it means you will see:


...HxLHxLHxLHxLH.... and so on...

The 'x' will be set high or low according to the data you wish to transmit.

The 'H' will always be high and the 'L' will always be low.  Every transition L->H looks like a positive edge clock.  If you were to lock your scope on this rising edge, you will see a clean data bit right after 'H'.

So, if we choose a 24bit parallel to serial 1 bit SERDES transmitter, the 24 bit on the parallel input will look like:

"1x01x01x01x01x01x01x01x0", where X would be an 8 bit word...

But, the receiving end needs to know which 'x' is the true first bit to begin.

For the DLL/PLL clock input, we will use the DLL where at every '01' transition, it will divide that by 2 making a clean 125MHz reference clock as the 'x' data bit after the 1 will be random noise.  A simple PLL (Spartain 6 has one of those) can create a single or multiple clock outputs by multiplying and dividing the source clock by a set of fixed integers, like 1,2,3,4,5,6,7,8,9...1023 and also deliver you optional multiple output offset phases.  I'm assuming the Spartain 6 DLL is a bit simpler than it's PLL as it can only multiply by integers of 1,2,4,8,16 and offer something like 4 or 8 optional different phases.

Ok, back to you bandwidth requirements:
125 bits x 10 000 x 200 = 250 000 000.  No room for anything.

Let's say we go for 130 bits, 1 start bit, 125 data bits and 4 stop bits.
(130*10000*200)= 260 000 000 baud, x4bits = 1040mbaud.  The speed limit of the OSERDES2 for the Spartain6.  (I'm using x4 instead of x3)  This means H[xx]LH[xx]LH[xx]LH...  The [xx] need to be the same, IE 1 bit value, they are just twice as wide instead of using the x3 pattern.  This means compatibility with the DLL saving your PLL for something else if needed.

Every time you get a start bit, start your next sample, while feeding through the previous data appending your previous sample to the end of the stream being fed from your previous board to the next board.
Will this work for you?

Each board will sample in parallel with an approximate 5-10ns delay + cable length from each other since they need to decode the serial stream looking for that first 'start/go' bit.  So, board #200's sample will be delayed by ~100-200ns.  Though, with additional coding, you can counteract that 10ns by predicting the start bit's arrival because of it's perfectly repetitive nature.  Basically your internal 10lhz clock will be set to begin sampling early by the 4-8 clocks on the 260MHz side it takes to see the 'start/go' bit.  I'm not sure how you will deal with the cable length, but with a 10khz sample rate, I don't think a global 200ns offset can be interpreted.

If everything is ok, these are my recommended next steps:

Design a SystemVerilog test-bench which will synthesize your master board's serial data chain.  Then when you begin coding for your Spartan 6, you will add that Spartan 6 code in your testbench, feeding it your custom 125bits serial input and see if your FPGA will lock onto your clock data and create a new internal clock from it while decoding and passing all the data though.

Then you can append your own Spartan 6 temporary dummy data onto the stream.

Then you can modify your Spartan 6 code to synthesize it's own master serial data option to replace your test-bench's beginner stream, basically clocking that Spartan 6 from a regular crystal with an IO pin set high or low to define whether it will run as a slave serial input, or run as the first master board from a crystal oscillator input.

Next, add multiple boards of your Spartan 6 code to your testbench, chained together as if wired in real life to verify each board adds it's own data into the stream without errors or missing bits and verify the phase of your internal generated 10khz sampling clock.

Then you may add you data acquisition sampling IOs to feed the true data into the chain.  (This will be a separate testbench just to verify you sampler connections as you already did the com, then you may merge the 200 board setup with the sampling IO version if you like.)

The goal it to create your entire 200 board system in something like ModelSim (So long as Xilinx has it's DLL and OSERDES models for ModelSim, or, whichever simulator Xilinx uses.) and see the entire board-boars system power-up and function.

You want to test everything before even creating a schematic so you know what you build will work.

[langwadt]

When we say '1b0', what we are saying is if you look as the serial output on a scope, it means you will see:


...HxLHxLHxLHxLH.... and so on...

The 'x' will be set high or low according to the data you wish to transmit.

The 'H' will always be high and the 'L' will always be low.  Every transition L->H looks like a positive edge clock.  If you were to lock your scope on this rising edge, you will see a clean data bit right after 'H'.

So, if we choose a 24bit parallel to serial 1 bit SERDES transmitter, the 24 bit on the parallel input will look like:

"1x01x01x01x01x01x01x01x0", where X would be an 8 bit word...

But, the receiving end needs to know which 'x' is the true first bit to begin.

For the DLL/PLL clock input, we will use the DLL where at every '01' transition, it will divide that by 2 making a clean 125MHz reference clock as the 'x' data bit after the 1 will be random noise.  A simple PLL (Spartain 6 has one of those) can create a single or multiple clock outputs by multiplying and dividing the source clock by a set of fixed integers, like 1,2,3,4,5,6,7,8,9...1023 and also deliver you optional multiple output offset phases.  I'm assuming the Spartain 6 DLL is a bit simpler than it's PLL as it can only multiply by integers of 1,2,4,8,16 and offer something like 4 or 8 optional different phases.

Ok, back to you bandwidth requirements:
125 bits x 10 000 x 200 = 250 000 000.  No room for anything.

Let's say we go for 130 bits, 1 start bit, 125 data bits and 4 stop bits.
(130*10000*200)= 260 000 000 baud, x4bits = 1040mbaud.  The speed limit of the OSERDES2 for the Spartain6.  (I'm using x4 instead of x3)  This means H[xx]LH[xx]LH[xx]LH...  The [xx] need to be the same, IE 1 bit value, they are just twice as wide instead of using the x3 pattern.  This means compatibility with the DLL saving your PLL for something else if needed.

Every time you get a start bit, start your next sample, while feeding through the previous data appending your previous sample to the end of the stream being fed from your previous board to the next board.
Will this work for you?

Each board will sample in parallel with an approximate 5-10ns delay + cable length from each other since they need to decode the serial stream looking for that first 'start/go' bit.  So, board #200's sample will be delayed by ~100-200ns.  Though, with additional coding, you can counteract that 10ns by predicting the start bit's arrival because of it's perfectly repetitive nature.  Basically your internal 10lhz clock will be set to begin sampling early by the 4-8 clocks on the 260MHz side it takes to see the 'start/go' bit.  I'm not sure how you will deal with the cable length, but with a 10khz sample rate, I don't think a global 200ns offset can be interpreted.

If everything is ok, these are my recommended next steps:

Design a SystemVerilog test-bench which will synthesize your master board's serial data chain.  Then when you begin coding for your Spartan 6, you will add that Spartan 6 code in your testbench, feeding it your custom 125bits serial input and see if your FPGA will lock onto your clock data and create a new internal clock from it while decoding and passing all the data though.

Then you can append your own Spartan 6 temporary dummy data onto the stream.

Then you can modify your Spartan 6 code to synthesize it's own master serial data option to replace your test-bench's beginner stream, basically clocking that Spartan 6 from a regular crystal with an IO pin set high or low to define whether it will run as a slave serial input, or run as the first master board from a crystal oscillator input.

Next, add multiple boards of your Spartan 6 code to your testbench, chained together as if wired in real life to verify each board adds it's own data into the stream without errors or missing bits and verify the phase of your internal generated 10khz sampling clock.

Then you may add you data acquisition sampling IOs to feed the true data into the chain.  (This will be a separate testbench just to verify you sampler connections as you already did the com, then you may merge the 200 board setup with the sampling IO version if you like.)

The goal it to create your entire 200 board system in something like ModelSim (So long as Xilinx has it's DLL and OSERDES models for ModelSim, or, whichever simulator Xilinx uses.) and see the entire board-boars system power-up and function.

You want to test everything before even creating a schematic so you know what you build will work.

afaict using the start bit to trigger sampling would make the last device sample 200 "frames" later than the first, unless you make each device aware of it's position so it can delay sampling N bits depending on position

[BrianHG]

  The maximum for the -2 Spartan 6 is 950mb, not 1050mb.

So, we need to do a few modifications:

Let's say we go for 136 bits, 1 start bit, 125 data bits and 10 stop bits.
(136*10000*200)= 272 000 000 baud, x3bits = 816mbaud.

But, we will need to use the PLL to divide our input clock by 2, the multiply it by 3 to get 408mhz reference clock while running the OSERRDES2 in 4:1 mode making it's parallel 4 bit IO port run at 204MHz.

136 bits TIMES (1 data bit + 2 clock bits) = 408 bit word frame.
408 bit word frame / 4 bit OSERDES2 = 102 -> 4 bit byte packets.  (Good, no ugly odd numbers half 4 bit parallel cycles to deal with)

With all 200 boards installed, how at the end, how will you know which was the first board's sample?
I guess now that we have an extra 10 stop bits, use 1 or 2 bits to signify an index and a final board in the list.

[BrianHG]

afaict using the start bit to trigger sampling would make the last device sample 200 "frames" later than the first, unless you make each device aware of it's position so it can delay sampling N bits depending on position

Each board sampled ahead by 1 clock from the beginning of the first/index start bit, while presenting that stored result data, the FPGA should have at the same time begun the sample for the next packet.  So, all samples will be delayed by 1 clock, all acquisitions beginning in parallel, but the data coming out throughout the 200 board packet.

[langwadt]

The maximum for the -2 Spartan 6 is 950mb, not 1050mb.

So, we need to do a few modifications:

Let's say we go for 136 bits, 1 start bit, 125 data bits and 10 stop bits.
(136*10000*200)= 272 000 000 baud, x3bits = 816mbaud.

But, we will need to use the PLL to divide our input clock by 2, the multiply it by 3 to get 408mhz reference clock while running the OSERRDES2 in 4:1 mode making it's parallel 4 bit IO port run at 204MHz.

136 bits TIMES (1 data bit + 2 clock bits) = 408 bit word frame.
408 bit word frame / 4 bit OSERDES2 = 102 -> 4 bit byte packets.  (Good, no ugly odd numbers half 4 bit parallel cycles to deal with)

With all 200 boards installed, how at the end, how will you know which was the first board's sample?
I guess now that we have an extra 10 stop bits, use 1 or 2 bits to signify an index and a final board in the list.

yeh, much much simpler with a clock on a separate pair, they could trigger the sampling with a pause in the clock

[BrianHG]

yeh, much much simpler with a clock on a separate pair, they could trigger the sampling with a pause in the clock

We now have the extra bits to embed it all.

I will not help ali_asadzadeh any further if he doesn't do a real test bench and fully test his chained design there, so we will see if everything clocks and syncs up before attempting to build anything.

That is I believe that you can simulate a Xilinx Spartan 6's PLL and OSERDES2, right?  (Otherwise, I made the right choice using Altera as you can simulated all their FPGA's IP primitives.)

[langwadt]

yeh, much much simpler with a clock on a separate pair, they could trigger the sampling with a pause in the clock

We now have the extra bits to embed it all.

I will not help ali_asadzadeh any further if he doesn't do a real test bench and fully test his chained design there, so we will see if everything clocks and syncs up before attempting to build anything.

That is I believe that you can simulate a Xilinx Spartan 6's PLL and OSERDES2, right?  (Otherwise, I made the right choice using Altera as you can simulated all their FPGA's IP primitives.)

yes you can simulate it all in ISIM that comes with ISE

(or in any other simulator once you figure out how to point to all the right libs)

[BrianHG]

https://docs.amd.com/v/u/en-US/spartan6_hdl
DLL 'DCM_SP', page 83.

Yes, you can use a multiple of 3 or 6 for the CLKFX clock output multiplier.

If you can use the CLKFX to drive the OSERDES2 transceiver, then you are clear to go with my 816mb calculation so long as you can clock the rest of the FPGA logic from a CLKFX/2 output.  Maybe use the CLKDV output set to divide by 2 if it's source is the CLKFX.

Use the switch clock input to select the external serial bus clock VS the PLL's local reference 204MHz clock.

Page 234, PLL_BASE, the normal full PLL.  Connect to a 12MHz or 24MHz crystal oscillator to generate the 204MHz core clock when you need it for the first master acquisition PCB.

The you might need to do more in the PLL.

The full Spartan 6 clocking datasheet: https://docs.amd.com/v/u/en-US/ug382

[langwadt]

https://docs.amd.com/v/u/en-US/spartan6_hdl
DLL 'DCM_SP', page 83.

Yes, you can use a multiple of 3 or 6 for the CLKFX clock output multiplier.

If you can use the CLKFX to drive the OSERDES2 transceiver, then you are clear to go with my 816mb calculation so long as you can clock the rest of the FPGA logic from a CLKFX/2 output.  Maybe use the CLKDV output set to divide by 2 if it's source is the CLKFX.

Use the switch clock input to select the external serial bus clock VS the PLL's local reference 204MHz clock.

Page 234, PLL_BASE, the normal full PLL.  Connect to a 12MHz or 24MHz crystal oscillator to generate the 204MHz core clock when you need it for the first master acquisition PCB.

The you might need to do more in the PLL.

The full Spartan 6 clocking datasheet: https://docs.amd.com/v/u/en-US/ug382

and before that make it all work with sampling and synchronization with just a continuous running separate clock and data

[BrianHG]

https://docs.amd.com/v/u/en-US/spartan6_hdl
DLL 'DCM_SP', page 83.

Yes, you can use a multiple of 3 or 6 for the CLKFX clock output multiplier.

If you can use the CLKFX to drive the OSERDES2 transceiver, then you are clear to go with my 816mb calculation so long as you can clock the rest of the FPGA logic from a CLKFX/2 output.  Maybe use the CLKDV output set to divide by 2 if it's source is the CLKFX.

Use the switch clock input to select the external serial bus clock VS the PLL's local reference 204MHz clock.

Page 234, PLL_BASE, the normal full PLL.  Connect to a 12MHz or 24MHz crystal oscillator to generate the 204MHz core clock when you need it for the first master acquisition PCB.

The you might need to do more in the PLL.

The full Spartan 6 clocking datasheet: https://docs.amd.com/v/u/en-US/ug382

and before that make it all work with sampling and synchronization with just a continuous running separate clock and data

He wanted a single serial link.
Having his testbench create a '1x0' pattern at a specific speed to feed his fpga, then having that FPGA's DLL lock onto the pattern where the 'x' is random junk is the first basic thing to achieve.  Seeing the simulator output a stable locked clock on a dummy IO or signal tap from the Spartan 6 code is all you want to see here.

Reading and sorting the junk with a connected OSERDES2 is part 2.

Making the testbench replace the random 'x' data with something useful, a dummy 136 bit packet and spaced to 10khz is next.

Having his FPGA now reading stuff which is no longer junk, and sorting that stuff into a 136bit serial register is next.  The start bit will begin the 36bit register and the index 10khz clock should be the next after the start bit.  Everything else is the 125 bit packet.

After receiving the first 125bit packet, the next once should already be ready to merge into the output OSERDES2.

If you are having trouble with this, then this project will be too tough to debug in the field.

Take these small steps, 1 at a time is all you need to do to move towards success.

[ali_asadzadeh]

Thanks BrianHG for your feedback, I will try to make a test project in Xilinx ISE and make some simulation,
But I need help on setting up the IP,


First We need a transmitter Block, Please check if I'm selecting the right choices in here
Page 1 of the IP wizard

Page 2 of the IP wizard

Page 3 of the IP wizard

Page 4 of the IP wizard

Page 5 of the IP wizard

Page 6 of the IP wizard


Then we need a receiver part, so here are the pages for the receiver side IP
Page 1 of the IP wizard

Page 2 of the IP wizard

Page 3 of the IP wizard

Page 4 of the IP wizard

Page 5 of the IP wizard

Page 6 of the IP wizard


Please check if I made the right choices in the wizard, also these questions comes to my mind, for the transmitter part the clk_in is the output of a pll that has multiplied for example the onboard 50Mhz clock to 1040MHz?
Also for the reciver part there is clk_in , where should I connect it? and I assume that I should send data to the transmitter and connect the diffoutputs to the reciver to recive the data.

[BrianHG]

Thanks BrianHG for your feedback, I will try to make a test project in Xilinx ISE and make some simulation,
But I need help on setting up the IP,

Wait, you went ahead 1 step too quickly...
1 last thing first....

Option 1:
You need to decide whether to use 1 twisted pair to transmit data and clock.
Basically an ~1ghz link using our 3 bit trick using OSERDES2 & a special DLL setup.

Or,

Option 2:
Use the 2 balanced lines in the SATA cable to transmit data and clock separately.
Basically an ~300mb link with an ~150mhz clock using DDRIO buffers as a 2:1 serializers.


Here are the pros and cons:
Option 1 means you will generate code which will be heavily tied down to Spartan 6 specific DLL and PLL.
The important data bit will be a 1ns wide pulse in the signal, meaning with IO tolerances, looking at an ~ +/- 250ps jitter at the IO pins, to read capture this bit, your PCB wiring will need to be good enough to operate in an ~ +/-250ps wide read window.
This method will be a little tricky to get working right.

Option 2, (langwadt recommended and my preferred method)
Means you will generate code which will work not only on Spartan6, but, it will also work on Efinix, Lattice, Altera, and any other FPGA which has IO with a general purpose IO DDR capable pins.
The jitter at the IO pins will still be ~ +/-250ps, however, the data bits will now be 3ns long, meaning you serial capture window will be around +/- 1250ps.  Far easier for your PCB routing and cable tolerances.
Your core clock will operate around 150mhz, fairly easy on modern FPGA, even slow ones.
No special DLL/PLL timing considerations, all vendors FPGAs can handle a 150Mhz 1:1 clock in, clock out.
Much easier to code everything.


(I know ASMI said that the SATA spec doesn't specify twisted pair timing matching, but come on, a +/-1250ps read window between both pairs in the cable means to mess this up, you will need 1 pair in the cable to be over 10 inches longer or shorter than the adjacent pair and looking at the crimped SATA ribbon cable connector, I do not think that is mechanically possible within the laws of our universe.  If you were doing the speeds of Option 1 with a 250ps/bit timing window, then yes, things would be a lot more critical here.)


Which option will you choose 1 or 2?

And, if you do not have the experience handling 1gbit routing on a pcb or how to order impedance controlled PCBs, then to be safe, I would recommend only going with option #2.  Though, if you need to send data in the opposite direction, depending on how much bandwidth you need, you might be stuck with option #1 with a new problem, you cannot exactly use the embedded clock as precisely the data going in the opposite direction at the same speed.  Otherwise, you PCB routing and coding will be a nightmare as now each cable length matters, you, you will need to use 2x DLL / PLLs in each FPGA to custom clock each direction.

[BrianHG]

Let me put it this way, do you even have a scope to inspect a 1GHz serial link?
This means at least 1GHz bandwidth, 10gsps for 1 channel, 5gsps with 2 channels on.
Proper 1GHz probes?
Preferably J-Fet amplified for low capacitance loads when reading data?

If you are not properly equipped, you might be forced into option #2, otherwise, you will be operating completely in the blind, relying purely on the FPGA internal digital signal spying utilities.

[langwadt]

with the two pairs of SATA you could also split the string in two, every other device on one pair, the other devices on the other pair. That would half the bandwidth requirement for each pair

[BrianHG]

with the two pairs of SATA you could also split the string in two, every other device on one pair, the other devices on the other pair. That would half the bandwidth requirement for each pair

Once you try to embed a clock somewhere on one pair or both, your screwed up to the higher bandwidth.

For 320mb, you need a 160mhz channel for a clock with a 160mhz bandwidth for the 320mb serial stream.
Remember, at 320mb, 1 bit high, the next low, and back equals a 160mhz clock.

(Yes, I'm bumping up the bitrate so there is room for 224 packets, 142 bits each, instead of 200x125.  The gives a huge 24 packet break in between every 200 packet chunk for the option of a reverse direction message, optional checksum, while the extra 17 bits from the 125 offers initial start, message direction and board number signature IDs for each of the 200 sequential boards.  IE: sequential boards may now be placed out of sequence and for the final FPGA, the will each be able to write directly into dual port memory to a correct address instead of a dumb address increment counter where the chain will always be in order.)

 (224*142+192)*10000) = 320 mbits per second.  The absolute worst FPGAs DDRIOs can do this 320mb.
The dedicated clock line is nothing more than that, a pure 160.000 MHz clock.  The stupidest FPGA, even ones without a PLL can pass this one through.

Operating like this, ali_asadzadeh can potentially go to a bottom end 2-3$ Lattice FPGA, or a 2$ PLD with 512 macrocells, though your better off with the FPGA for cell density.


(Note: (224*142+192)*10000) = 320mbps.  / 16 = cheap 20MHz reference crystal, that 192 extra cycles are dumb empty filler to make the 10khz sample clock and packet size a perfect multiple of 20/40/80/160MHz.  Think of it as preamble and a guaranteed dead zone which cannot be filled for sync alignment.)

[hamster_nz]

Just to chime in... a simple encoding system like 8b10b would:

- Keep bit rate to 300Mb/s - (Only a 25% coding over head rather than 200%)

- Provide the timing information as edge transitions are present in the bit stream

- Ensure that the signals are DC balanced, could be AC coupled.

- You would have the K symbols to allow for framing of data (e.g, you could wrap the data blocks with chosen "Start of Frame" and End Of Frame" symbols

- Would allow a crude for of error detection though invalid symbols.

- Could be implemented with a IDDR and ODDR privative and a 150MHz clock domain, rather than the work needed to get SERDES blocks running.

[ali_asadzadeh]

Which option will you choose 1 or 2?

I have designed high speed PCB's before and I would choose Option 1, also I noticed something if we shoot for the 1Ghz link and only 4X serializer, the actual core should run @ 250Mhz, which I think the best speed I could get from spartan 6 is around 100Mhz.

Just to chime in... a simple encoding system like 8b10b would:

- Keep bit rate to 300Mb/s - (Only a 25% coding over head rather than 200%)

- Provide the timing information as edge transitions are present in the bit stream

- Ensure that the signals are DC balanced, could be AC coupled.

- You would have the K symbols to allow for framing of data (e.g, you could wrap the data blocks with chosen "Start of Frame" and End Of Frame" symbols

- Would allow a crude for of error detection though invalid symbols.

- Could be implemented with a IDDR and ODDR privative and a 150MHz clock domain, rather than the work needed to get SERDES blocks running.

I'm not familiar with 8b10b encoding, would you explain more, consider me a total noobe in this area

[hamster_nz]

8b10b is really neat. Originally discovered/Invented by IBM the patent has now expired. It is a great entry into coding theory.

Much of this is from memory, so you really want to refer to the Wikipedia page - https://en.wikipedia.org/wiki/8b/10b_encoding

At a very simple level, you replace every 8-bit byte with a 10-bit symbol and transmit that instead. The symbols are special.

The symbol set is 256 'D' (data) symbols, and 13 'K' symbols (Control Symbols - which don't represent data bytes).

Simplicity

To generate the codes the data byte is split into 5 bits and 3 bits, giving two numbers- one between 0 and 31, the other between 0 and 7. These parts is mapped to 6 bits and 4 bits to build the final symbol - so (almost) two simple lookup tables.

This gives the naming system of D.x.y or K.x.y - e.g. D.10.7 is the data symbol for 01010111 ('01010' = 10 dec, '111' = 7 dec).

Traditionally it was implemented using a relatively small number of logic gates, but of course you use lookup tables in FPGAs to do  the same mapping.

DC Balanced

Each D or K symbol has either five 1s and five 0s, or there are two encodings for the symbol, one with four 1s, one with six 1s. You chose between the two encodings based on the running disparity of the symbols before it, so the disparity jumps between +1 and -1.

This ensures that on any stream you can have a equal number of 1s and 0s (to within +/-1 bit). As the resulting stream has a long term average of 0.5, and can be AC coupled.

Clock recovery
Because of the way the symbols are constructed there will a large number of transitions between 1s and 0s. This allows the receiver to clearly identify bit boundaries and recover the data clock. You will never transmit "011111111111111111110" so the receiver will never be left wondering if there are 19 or 20 bits in that long run of 1s.

Synchronization

If the K.28.7 symbol (0011111000 or 1100000111) is not used , then the sequence 00111110 or 11000001 will never appear in a 8b10b stream, other then in the K.28.1 (0011111001 or 1100000110) and K.28.5 (0011111010/1100000101) symbols. This is neat because if you use this as your 'idle' symbol then you look for that pattern and recover symbol alignment.

The other K symbols can be used as non-date symbols. In your case assigning a 'start of frame' and 'end of frame' K symbol might be a good idea.

Efficiency
Because every 8 bits is mapped to 10-bit symbols, the overhead is fixed at 20%. This is the same as RS232 (8-data bits, one start bit, one stop bit), but is much better then schemes like Manchester coding.

[BrianHG]

8b10b is really neat. Originally discovered/Invented by IBM the patent has now expired. It is a great entry into coding theory.

Much of this is from memory, so you really want to refer to the Wikipedia page - https://en.wikipedia.org/wiki/8b/10b_encoding

At a very simple level, you replace every 8-bit byte with a 10-bit symbol and transmit that instead. The symbols are special.

The symbol set is 256 'D' (data) symbols, and 13 'K' symbols (Control Symbols - which don't represent data bytes).

Simplicity

To generate the codes the data byte is split into 5 bits and 3 bits, giving two numbers- one between 0 and 31, the other between 0 and 7. These parts is mapped to 6 bits and 4 bits to build the final symbol - so (almost) two simple lookup tables.

This gives the naming system of D.x.y or K.x.y - e.g. D.10.7 is the data symbol for 01010111 ('01010' = 10 dec, '111' = 7 dec).

Traditionally it was implemented using a relatively small number of logic gates, but of course you use lookup tables in FPGAs to do  the same mapping.

DC Balanced

Each D or K symbol has either five 1s and five 0s, or there are two encodings for the symbol, one with four 1s, one with six 1s. You chose between the two encodings based on the running disparity of the symbols before it, so the disparity jumps between +1 and -1.

This ensures that on any stream you can have a equal number of 1s and 0s (to within +/-1 bit). As the resulting stream has a long term average of 0.5, and can be AC coupled.

Clock recovery
Because of the way the symbols are constructed there will a large number of transitions between 1s and 0s. This allows the receiver to clearly identify bit boundaries and recover the data clock. You will never transmit "011111111111111111110" so the receiver will never be left wondering if there are 19 or 20 bits in that long run of 1s.

Synchronization

If the K.28.7 symbol (0011111000 or 1100000111) is not used , then the sequence 00111110 or 11000001 will never appear in a 8b10b stream, other then in the K.28.1 (0011111001 or 1100000110) and K.28.5 (0011111010/1100000101) symbols. This is neat because if you use this as your 'idle' symbol then you look for that pattern and recover symbol alignment.

The other K symbols can be used as non-date symbols. In your case assigning a 'start of frame' and 'end of frame' K symbol might be a good idea.

Efficiency
Because every 8 bits is mapped to 10-bit symbols, the overhead is fixed at 20%. This is the same as RS232 (8-data bits, one start bit, one stop bit), but is much better then schemes like Manchester coding.

@hamster_nz, I don't think the OP has room for any ansyc clock recovery in his com.  At least according to the way the project was described.  If you may get the com working, but what happens to the perfect sync required for all the 10khz samplers.  What if one slightly slips ahead, where does the extra bit of 125 bits go?

On the other hand, I'll let you explain to the OP how to program his clock recovery system and how to make a new synchronous global PLL from it for a clean sampler clock.

The only reason for the original option#1, was to make a pure PLL locked parallel shared system clock between boards with data on 1 cable with the SERDES capabilities built into a Spartan 6.  I'll let you 2 work out how to do this without an embedded clock and how to simulate it verifying proper error free functionality.

I know with 3x or 4x oversampling, you can do software clock realignment seeing the slipping edge bit.  But this means each board-to-board will not have a dead perfect acquisition 10khz clock.  The will all have a go signal and skew the start of their 10khz samplers either a few clocks early or late.

The '1x0' trick was the same as a 3x oversampling clock, ( '1xx0' basically a 4x oversampling equivalent ) but you have a new guaranteed timing bit at the beginning of each transmitted bit.  If fact, the pattern is so small and repetitive, you can now PLL lock onto the rising edge of that signal so long as it is broadcasted continuously.  This trick doesn't require any SERDES trickery, you literally have bits 0 and 3 hard tied high and low while bits 1 and 2 are tied together to transmit a 125+extra serial shift register containing your package.

If the op want to continue with a clock embedded into every transmitted bit going out, then I need him to verify if the OSERDES2 can do 6:1 instead of 4:1 or 8:1.

No matter where he goes, DDR oversampling give him a maximum bitrate of 750mb while OSERDES's QDR oversampling gives him 950mb on a -2 speed grade Spartan.  A -3 speed grade Spartan can go to 1080mb, but that one might not be available in industrial temperature range which the OP needs, unless the OP isn't afraid of overclocking.


Also, the OP's original break neck 125 bits by 200 packets exactly without room for an extended break might be too close for comfort, though I see how he may want to originally minimize bandwidth.

[BrianHG]

Also, dont forget with the async design, you repeating a message coming in from the previous board in the chain.  If you reference crystal is ever so slightly slower than the previous board, you may clean up and soft-reclock capturing the data coming in correctly, but your repeat transmission will be going out ever so slightly slower meaning your 4-8bit buffer may over flow breaking your transmission.

I'm not going to help the OP deal with all those implications as I though all he wanted was simple bit pass through synchronous pipe so it can fit into the cheapest tiny FPGA without any grand memory buffering and message re-timing/re-pacing code.  I was thinking about a complete com section being about 1 page of very dumb in->out verilog code with a 2-4bit shift register to transfer in to out with the other 125bit shift register to pipe through the current board's stored acquisition sample.

[langwadt]

The '1x0' trick was the same as a 3x oversampling clock, ( '1xx0' basically a 4x oversampling equivalent ) but you have a new guaranteed timing bit at the beginning of each transmitted bit.  If fact, the pattern is so small and repetitive, you can now PLL lock onto the rising edge of that signal so long as it is broadcasted continuously.  This trick doesn't require any SERDES trickery, you literally have bits 0 and 3 hard tied high and low while bits 1 and 2 are tied together to transmit a 125+extra serial shift register containing your package.

If the op want to continue with a clock embedded into every transmitted bit going out, then I need him to verify if the OSERDES2 can do 6:1 instead of 4:1 or 8:1.

I had a play with it in simulation and OSERDES will do 3:1, but I'd say forget it, it works in simulation but getting clocking and routing just right to meet timing  will likely take lots of work work

I had a much more naughty idea that seems it could work at 200MHz, so two strings would do 400Mbit

use a PLL to (re)generate 200MHz from the 200Mbit data, same PLL can generate 200MHz with 70% duty cycle, and 200MHz with 40% duty cycle

then use mux controlled by tx data to switch between the two clocks, should be possible to do glitch free since the switch always happen in the high period



 

[BrianHG]

I had a play with it in simulation and OSERDES will do 3:1, but I'd say forget it, it works in simulation but getting clocking and routing just right to meet timing  will likely take lots of work work

I had a much more naughty idea that seems it could work at 200MHz, so two strings would do 400Mbit

use a PLL to (re)generate 200MHz from the 200Mbit data, same PLL can generate 200MHz with 70% duty cycle, and 200MHz with 40% duty cycle

then use mux controlled by tx data to switch between the two clocks, should be possible to do glitch free since the switch always happen in the high period

Your sim is not showing the DLL status.
Was the DLL already locked in and stable so soon in your sim?
Did you set the DLL input to divide by 2?
Why didn't you show your DLL's output 150MHz clock in your sim VS source data?

On the receive end, if tuned right, the 6:1 deserialize should show a parallel output of 6 bits 6'b{1,data,0,1,data,0} if tuned properly on every 150mhz clock.

How did your home made auto-phase lock adjustment tuning algorithm do?
How many phase steps / degrees of valid data window did the deserializer output the proper mask 6'b{1,x,0,x,1,x,0} ?

A 6:1 OSERDES2 running at 150Mhz should give you 900mb.
Since there are 2 data bits running at 150Mhz, you should have a 300megabit link.

Remember, you cannot use the PLL_BASE, it does not have that crucial divide by 2 on the reference clock input.
The PLL's phase comparator will get all confused with the noisy bit.

Note that there is still an async way to do this with a 300mb straight serial data, but the OP wanted a synced clock.

[glenenglish]

It would be fine at 125 MHz/ 250Mbps, or even double that if you used a T20 or a S7.....
but why why why restrict yourself to Spartan 6 ? A part from 15 years ago ?
There are plenty of easy to use modern parts...

Why design a product for a customer using obsolete parts ? Obsolete parts and obsolete + unsupported tools? this isnt doing any favours for the customer....

Modern parts will easy do 400 MHz / 800 Mbps in even the lowest speed grade.

[BrianHG]

Modern parts will easy do 400 MHz / 800 Mbps in even the lowest speed grade.

900mb for the slowest spartan 6, and we need it anyway we approach the project.  But yes, I wouldn't use such an old part.

1280mb RX, ie 4x over sampling a 320mb serial com would make a simple async solution.
960mb RX, ie 3x oversampling a 320mb serial com would make a async solution where you have to shift an odd number of bits on the RX end.  Just a bit more annoying to work the logic.

350mb would give us breathing room.
Above, I'm speaking in old fashioned 8N1 serial characters.

[langwadt]

I had a play with it in simulation and OSERDES will do 3:1, but I'd say forget it, it works in simulation but getting clocking and routing just right to meet timing  will likely take lots of work work

I had a much more naughty idea that seems it could work at 200MHz, so two strings would do 400Mbit

use a PLL to (re)generate 200MHz from the 200Mbit data, same PLL can generate 200MHz with 70% duty cycle, and 200MHz with 40% duty cycle

then use mux controlled by tx data to switch between the two clocks, should be possible to do glitch free since the switch always happen in the high period

Your sim is not showing the DLL status.
Was the DLL already locked in and stable so soon in your sim?
Did you set the DLL input to divide by 2?
Why didn't you show your DLL's output 150MHz clock in your sim VS source data?

On the receive end, if tuned right, the 6:1 deserialize should show a parallel output of 6 bits 6'b{1,data,0,1,data,0} if tuned properly on every 150mhz clock.

How did your home made auto-phase lock adjustment tuning algorithm do?
How many phase steps / degrees of valid data window did the deserializer output the proper mask 6'b{1,x,0,x,1,x,0} ?

A 6:1 OSERDES2 running at 150Mhz should give you 900mb.
Since there are 2 data bits running at 150Mhz, you should have a 300megabit link.

Remember, you cannot use the PLL_BASE, it does not have that crucial divide by 2 on the reference clock input.
The PLL's phase comparator will get all confused with the noisy bit.

Note that there is still an async way to do this with a 300mb straight serial data, but the OP wanted a synced clock.

the duty cycle is (barely) within limits for 200MHz, the PLL only uses rising edge why else would it have such a wide allowable duty cycle range?
but you can divide input before the pll


 

[BrianHG]

With 2 channel com, he could have gone with a $1.50 Efinix FPGA.  (800mb peak LVDS)
Using a HDMI cable, he would have had 4 twisted pairs, 2 for TX, another 2 for RX allowing for a sync clock plus additional.  Though, I am not certain on the consistency of available HDMI cables, though running at a slow 300mb should make the worst cable AOK.  Also, with HDMI cables, you also get power pins plus another 5 wires to share between PCBs.  SATA is exclusive 2 twisted pairs and GND.

In Async mode, it is that slipping bit you need to keep track of which is why I recommend at least 3x oversampling when running in async mode.  (Async having no clock embedded in the serial stream.  You are literally re-adjusting the read data position based on every detected transition in the RX stream in real time, so the transmitter runs at 300mb, but the receiver reads with a local crystal oscillator at 900mb, slipping the read position to the next adjacent bit at every detected input transition.  You would want something like a 1-8N2, or 1-16N2 serial pattern to allow slippage up or down. 1 start bit, 2 stop bits.)

[BrianHG]

the duty cycle is (barely) within limits for 200MHz, the PLL only uses rising edge why else would it have such a wide allowable duty cycle range?
but you can divide input before the pll

The DLL has this divide by 2 on it's input built into it's circuitry.  So, no noise & phase error based on using a logic cell to divide the input by 2, then feed the PLL.  It was specifically designed for this purpose giving us that guaranteed 50:50 duty cycle.  It is literally a hard parameter switch for the DLL separate of the normal divide and multiply parameters where the compiler may bypass your divide by 2 in the m.n settings to optimize the PLL's performance.

[BrianHG]

https://docs.amd.com/v/u/en-US/spartan6_hdl
DLL 'DCM_SP', see parameters page 86.

Parameter:
CLKIN_DIVIDE_BY_2 -  TRUE or FALSE.

What do you think this was designed for?

Remember, this is separate of parameters:
CLKDV_DIVIDE
CLKFX_DIVIDE
CLKFX_MULTIPLY

[langwadt]

https://docs.amd.com/v/u/en-US/spartan6_hdl
DLL 'DCM_SP', see parameters page 86.

Parameter:
CLKIN_DIVIDE_BY_2 -  TRUE or FALSE.

What do you think this was designed for?

Remember, this is separate of parameters:
CLKDV_DIVIDE
CLKFX_DIVIDE
CLKFX_MULTIPLY

I didn't use DCM_SP, I used PLL_BASE (optionally feed by a clock input buffer that can divide)

[ali_asadzadeh]

Guys thanks for the feedbacks.

If the op want to continue with a clock embedded into every transmitted bit going out, then I need him to verify if the OSERDES2 can do 6:1 instead of 4:1 or 8:1.

Yes they can work.

With 2 channel com, he could have gone with a $1.50 Efinix FPGA.  (800mb peak LVDS)

I'm all ears? what part do you suggest?

but why why why restrict yourself to Spartan 6 ? A part from 15 years ago ?
There are plenty of easy to use modern parts...

Why design a product for a customer using obsolete parts ? Obsolete parts and obsolete + unsupported tools? this isnt doing any favours for the customer....

Modern parts will easy do 400 MHz / 800 Mbps in even the lowest speed grade.

I'm all ears on selecting another parts,

https://www.digikey.com/en/products/detail/efinix-inc/T8Q144I4/11591372
This one from efinix is comparable to the XC6slX9 part from xilinx,  But It has higher price, since I can get the AMD part cheaper in china, also Please note that I prefer TQFP Because of the ease of assembly over BGA, since it would cost less and can be done by hand very easily.

[BrianHG]

https://www.digikey.com/en/products/detail/efinix-inc/T8Q144I4/11591372
This one from efinix is comparable to the XC6slX9 part from xilinx,  But It has higher price, since I can get the AMD part cheaper in china, also Please note that I prefer TQFP Because of the ease of assembly over BGA, since it would cost less and can be done by hand very easily.

You are comparing a real price for the Efinix to a LCSC priced Xilinx.  A brand new version of your Xilinx part at Digikey is actually 53$.   
Read the 2 adjacent posts here regarding LCSC:
https://www.eevblog.com/forum/chat/lcsc-in-china-is-much-cheaper-than-normal-franchised-distributors/msg5604233/#msg5604233

BTW, I find 256pin BGA (0.5mm) easier to hand solder than 144pin TQPF.

[BrianHG]

You want cheap LCSC, here's $4.52:  110kbit ram, 2000lc.
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_Intel-Altera-10M02SCE144C8G_C78764.html

No bootprom needed.  It has a built in bootproms with 2 switchable boot-up profiles.
No multi-power-supply.  It can run off a single 3.3v or 2.5v supply.

[glenenglish]

arrghh! there's always an Altera person lurking

Ali, if you are going to hand assemble, surely this means a very low quantity and so FPGA price does not really matter at all.

[BrianHG]

If you could squish you project into this Intel/Altera 5M240ZT100C5N, 100 pin QFP, $2.15 each for 200pcs at  LCSC.

304mb per pin. 

But the 240 logic cells might be a constraint.
Though, you only need to buffer 125 bits + say another 8 bits.

I believe it also has built in flash.
Also, it is more akin to a PLD, not a full fledged FPGA like the MAX10 series.

[BrianHG]

arrghh! there's always an Altera person lurking

Ali, if you are going to hand assemble, surely this means a very low quantity and so FPGA price does not really matter at all.

Read his posts.  Each 'product' requires 200 serial linked acquisition boards, that's two hundred daughter boards, each with the cheap FPGA.  If he saves 5$ in bootprom and FPGA combined, that is a 1k$ savings for each product.  If he sells 10 'products', that's a 10k$ savings.  If he already set his price or already quoted a final amount to his clients, for 10 installations, it is an additional 10k$ in ali_asadzadeh pocket.  If he sells 100 setups, that is an additional 100k$ in his pocket.  At this level, it is worth it to put in the effort to find the cheapest solution.

I'm sure the final processing unit will have some more modern FPGA which is a lot beefier.

Making everything work on a $2.15 FPGA instead of a 7$ one plus it's bootprom price in the OP ali_asadzadeh eyes may make proper business sense.  And if the Altera 5M240ZT100C5N can be made to work, he may thank me for finding it as it will save him at least 1000$ for the first unit with 200 daughter boards.

For prototyping, he may only assemble 3-5 units for testing, but once the PCBs are verified error free, he will have them assembled by the PCB manufacturer.

Even the MAX10 device I listed at $4.52, which can LVDS over 650mb (the 5M240 part maxes out at 304mb) and has a full PLL will still save ali_asadzadeh over 600$ on each setup as he no longer need a boot flash, or VCC core voltage regulator to use that device.  A device which has full real-time operation debug monitoring through the JTAG, PLL and a reasonable blockram unlike the cheaper altera 5M240ZT100C5N where everything need to be programed to the bone.

[technix]

RGMII? Then stick something like RTL8211 and a Magjack and you get gigabit Ethernet. However many devices you have just design an appropriate sized network switching architecture.

[glenenglish]

some other FLASH options (Ali just how many are you going to build )

Brian has given you  some fairly good "PLD like/ small fpga " options for you at low cost.  (Brian, I thought this was one system of 200)

other options
How about IGLOO. they are FLASH. and they have emulated differential signalling ..... and certainly fast enough.
----
If ALi needs cheap, one of these ForgeFLASH Renesas FPGAs might do it , *might* be (just) fast enough.
like 50 cents.
https://www.renesas.com/us/en/document/dst/slg47910-datasheet?r=25546631
24QFN- can be soldered. probably enough I/O - cant remember. and is non volatile.

No LVDS but for 29c you can put LVDS buffers on it like DS90LT1 DS90LV DS90LV018 etc etc series  (SOT23)
---
otherwise T4 or T8 Trion Efinix $1.45 / $1.70  . plenty of logic and 400 Mbps LVDS

[ali_asadzadeh]

You want cheap LCSC, here's $4.52:  110kbit ram, 2000lc.
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_Intel-Altera-10M02SCE144C8G_C78764.html

No bootprom needed.  It has a built in bootproms with 2 switchable boot-up profiles.
No multi-power-supply.  It can run off a single 3.3v or 2.5v supply.

Thanks for the feedback, But I think 2K logic elements would be very limiting, Because there are some ADC's with SPI and they have a fair bit of registers that need to be configured, also There are some other Sensors to sample that have a bit of complexity in them, so I think 8K LE is the minimum I would be ok with.

[asmi]

TI also still makes SERDES chips so even if your FPGA can't do high-speed serial you can use these devices to handle transmission/receiving with CDR and everything, and you get a nice parallel interface on both ends.

[BrianHG]

You want cheap LCSC, here's $4.52:  110kbit ram, 2000lc.
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_Intel-Altera-10M02SCE144C8G_C78764.html

No bootprom needed.  It has a built in bootproms with 2 switchable boot-up profiles.
No multi-power-supply.  It can run off a single 3.3v or 2.5v supply.

Thanks for the feedback, But I think 2K logic elements would be very limiting, Because there are some ADC's with SPI and they have a fair bit of registers that need to be configured, also There are some other Sensors to sample that have a bit of complexity in them, so I think 8K LE is the minimum I would be ok with.

Setting I2C registers aside which can be stored as a LUT in the 100kbit blockram, such complexity in clocking and shifting an SPI bus to acquire 125bits to be transmitted?  You must also be doing data compression in each FPGA before transmitting the data stream.

BTW, a little secret, when programming a 10M02 FPGA in quartus, just select the 10M04 fpga and upgrade your projects FPGA for free.  Both FPGAs are the same die and selecting the 10M02 is nothing more than a pseudo software on the chips capabilities.

Actually, the same thing be said for the first 2 bottom sizes of most Xilinx and Lattice FPGAs.
You need to google around, but someone has generated a table of many of the available FPGA sizes which are actually identical dies.

Another trick is to search in the datasheets for the raw uncompressed bootprom image file sizes for each FPGA in a set family.  Any 2 or 3 sizes which match the number of bytes, you know the core die is identical.

[BrianHG]

It looks like the XC6SLX4 might have the same core die as the XC6SLX9.  The bitstream has 10 thousand extra bits out of 2 million bits.

Maybe you could save money using the XC6SLX4, try programming it as a XC6SLX9 and see if it works.

[Someone]

It looks like the XC6SLX4 might have the same core die as the XC6SLX9.  The bitstream has 10 thousand extra bits out of 2 million bits.

Maybe you could save money using the XC6SLX4, try programming it as a XC6SLX9 and see if it works.

I was sure I documented that on this forum, but cant seem to find it 

[BrianHG]

It looks like the XC6SLX4 might have the same core die as the XC6SLX9.  The bitstream has 10 thousand extra bits out of 2 million bits.

Maybe you could save money using the XC6SLX4, try programming it as a XC6SLX9 and see if it works.

I was sure I documented that on this forum, but cant seem to find it 

Here: https://www.eevblog.com/forum/fpga/quartus-prime-18-1-cant-fit-design-in-device/msg3179010/#msg3179010

It is a shame since before the pandemic, the Lattice  LFE5U-12F series was around 7$ each for 10 pcs.
It happens to be the same die as the LFE5U-25F, that's a 24k logic cell chip with 1 megabit blockram for 7$ !!!

Today, the LFE5U-12F and LFE5U-25F are 15$, then 20$.  The 12f doubled in price while the 25f went up by around 7$.

[Someone]

It looks like the XC6SLX4 might have the same core die as the XC6SLX9.  The bitstream has 10 thousand extra bits out of 2 million bits.

Maybe you could save money using the XC6SLX4, try programming it as a XC6SLX9 and see if it works.

I was sure I documented that on this forum, but cant seem to find it 

Here: https://www.eevblog.com/forum/fpga/quartus-prime-18-1-cant-fit-design-in-device/msg3179010/#msg3179010

https://www.eevblog.com/forum/microcontrollers/artix-7-only-has-three-different-die-sizes/msg837185/#msg837185
There is it, thanks. Bizarrely it doesn't come up in site searches for keywords and needs a lot of context on google to hit it.

a little bitstream twiddling, and some extensive per device testing (or just faith/dumb luck) gets some extra bang for the buck.

[glenenglish]

just look at Vcc CORE static current.... that tells you what die is what....

Now, just because they're the same die, does not guarentee the whole die works. They do do binning.  (do do?).  Usually the smaller devices in each mask set get released later after the mfr figures out yields, what works, what does not....

@ Ali, any resistance to using Efinix T4, T8 ? they are the highest performance, lowest cost, modern,  and package friendly device that you can choose from.  Just wondering ? I just wonder why you wouldnt consider them when they are very clearly, the best fit (speed, IO, price, tools)  for your project.

[ali_asadzadeh]

Thanks for the feedback

@ Ali, any resistance to using Efinix T4, T8 ? they are the highest performance, lowest cost, modern,  and package friendly device that you can choose from.  Just wondering ? I just wonder why you wouldnt consider them when they are very clearly, the best fit (speed, IO, price, tools)  for your project.

I'm not resisting anything, what prices can I buy them? do you have a source with good prices? I can get XC6SLX9 around 6$ from my Chinese supplier

[BrianHG]

https://www.questcomp.com/part/4/t4f81c2x/435178692?utm_source=findchips&utm_medium=industry-cpc&utm_term=t4f81c2x&utm_content=standardpricing&utm_campaign=aktype0

$3.60 x200

https://www.digikey.com/en/products/detail/efinix-inc/T4F81I2X/16162570

$3.61 x1500  ***industrial temp range version, -40°C ~ 100°C (TJ)

[ali_asadzadeh]

https://www.questcomp.com/part/4/t4f81c2x/435178692?utm_source=findchips&utm_medium=industry-cpc&utm_term=t4f81c2x&utm_content=standardpricing&utm_campaign=aktype0

$3.60 x200

https://www.digikey.com/en/products/detail/efinix-inc/T4F81I2X/16162570

$3.61 x1500  ***industrial temp range version, -40°C ~ 100°C (TJ)

Thanks for the update, what about the software? is it free too? what about the programming tool, does Generic FT2232 work as well?

[BrianHG]

If you do not need the IO count and only need 4klc, this one is a $1.70 in quantity:
https://www.digikey.com/en/products/detail/efinix-inc/T4F49C2/11591374
T4F49I versions exist.

The effinix T4 series only goes up to 400mb in sdr mode only.

The effinix T8 series supports DDR on their IOs for their 144pin qfp, up to 800mbps:
https://www.digikey.com/en/products/detail/efinix-inc/T8Q144C4/11591363
T8Q144I versions exist.
Contact Digikey for quantity pricing...

[glenenglish]

Ali
contact the maufacturers representative, you'll always get half to one quarter  the price of digikey....
T4 is too cut down, suggest T8 or T13 or  T20
https://www.efinixinc.com/products-trion.html

QFP100 is available with on chip FLASH in T13, T20 size (you'll pay something for the flash)
otherwise QFP144  T8/T20
I would be using a T13 or T20  in a 0.65mm 169 ball BGA. easy.

go read the site.