Cheap 250Mbs Link between Boads

[glenenglish]

serdes is just  a fancy shift register, you can run as slow as you like

I think we need to clarify : common terminology is that :

"SERDES"  or  "transceivers" are  a clock recovering high speed differential serial interface block, usually with 8b/10b or 64/65 etc encoding and pattern detection.   They are not in every fpga. They usually have a minimum bit rate that clock can be reocvered- the range of VCO frequencies supported. Usually about 500 Mbps

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

[langwadt]

serdes is just  a fancy shift register, you can run as slow as you like

I think we need to clarify : common terminology is that :

"SERDES"  or  "transceivers" are  a clock recovering high speed differential serial interface block, usually with 8b/10b or 64/65 etc encoding and pattern detection.   They are not in every fpga. They usually have a minimum bit rate that clock can be reocvered- the range of VCO frequencies supported. Usually about 500 Mbps

Xilinx call those GTP, https://docs.amd.com/v/u/en-US/ug482_7Series_GTP_Transceivers

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

Xilinx call those serdes, https://docs.amd.com/r/2021.1-English/ug953-vivado-7series-libraries/ISERDESE2

[Someone]

Serializers/deserializer for LVDS are  rising and falling edge  flipflop/shift parallel to serial registers that require clock. Yes they are in almost every fpga.

Xilinx and Altera both call that SERDES.

[hamster_nz]

Possible but a lot of work:

1. 8b10b coding (+25% overhead), 64b66b+scrambler coding (+3% overhead) or TMDS coding (25% overhead) on the link so you can recover framing

2. Oversample the input signal by 2x so you can identify the data edges and track them. Find for standard SERDES blocks on most FPGA I/O pins. No high speed transceiver needed.

3. Use the fine phase shift of the clock manager in a control loop to recover timing from the bit stream.

4. Use the TMDS_33 (aka HDMI) IO standard for your physical layer.

[BrianHG]

4. Use the TMDS_33 (aka HDMI) IO standard for your physical layer.

Nope, unless you are driving a TMDS terminated receiver, do not use TMDS_33.
There are other, lower power, easier to terminate differential balanced transmission line options exist compatible with your FPGA IOs which can run 500mb on a 100cm cable like you see in serial ATA HD cables.

Again, read your FPGA's electrical specifications data sheets.  They illustrate example LVDSIO<->LVDSIO wiring diagrams with example termination resistors, supported cable lengths and maximum bit-rates.

[ali_asadzadeh]

Thanks all for your input and feedback,
BrianHG I think you have lot's of exprince in this field,

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.

Would you please explain more, give some links ,app notes etc... I have seen the Select IO wizard IP block in Xilinx, and I think you might suggest this path, am I right?

[BrianHG]

Thanks all for your input and feedback,
BrianHG I think you have lot's of exprince in this field,

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.

Would you please explain more, give some links ,app notes etc... I have seen the Select IO wizard IP block in Xilinx, and I think you might suggest this path, am I right?

I can easily help you with Altera/Intel.  There are others here better suited for Xilinx.

You should start here: https://docs.amd.com/v/u/en-US/ug381
Read pages 33-37...  (Actually, just read the entire data sheet as it explains the SERDES IP as well...)
I was wrong, you can also use TMDS_33 with Xilinx.

However, you need to match the drive and termination wiring based on the type of cable you will be using.

For this, you will need to google the spec/impedance and wiring of shielded SATA cables and choose which IO wiring on page 34 of the Xilinx .pdf will get your data from FPGA to FPGA.

The you will need to choose how to clock your system.
SATA has 2 pairs of data signals.

If you are sending data in one direction, the easiest dumb solution is to have 1 pair transmit a 25 mhz clock to the next FPGA PLL reference input while the second pair runs serial data at 10x the clock.

The other solution is to embed a clock in your data.  This one is more a feature set function related to Xilinx's IP SERDES blocks.  I know you can use any differential IO pair for transmit, but for maximum clock recovery functions, you will need to consult the Xilinx data sheets.

My really old school extra dumb method for embedding a clock with Altera FPGAs would have been to transmit 1 bit high, 1 bit data, 1 bit low, 1 bit data, 1 bit high, 1 bit data.... at 500megabit, clocking the second FPGA with every second bit (my 1 bit high/low, a 125mhz clock) while having a 250mb data channel.  However, this required a little PLL trickery.

More modern embedded clock recovery techniques exist which should be available to you, but if you use a separate clock channel, things can be made to guarantee the dumb separate clock method.

I do not know the type of SATA wire being used, if it is twisted pair, or 2 individual shield cables.

If it is 2 individual shielded/coax connections, you could also use 1 cable as a 25mhz clock, the other as 250mb data, wiring everything in single ended instead of differential.

[ali_asadzadeh]

Dear BrianHG, thanks for clarifying things future

My really old school extra dumb method for embedding a clock with Altera FPGAs would have been to transmit 1 bit high, 1 bit data, 1 bit low, 1 bit data, 1 bit high, 1 bit data.... at 500megabit, clocking the second FPGA with every second bit (my 1 bit high/low, a 125mhz clock) while having a 250mb data channel.  However, this required a little PLL trickery.

I like your old school extra dumb method, can you share more on that, it sound a good solution to me, since  I'm only sending data in one direction.

[asmi]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

[langwadt]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

[Someone]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

Clean solution, but sadly violates the PLL input specs of most (at least all of the ones I'm familiar with) suitable FPGAs. 100 vs 110 would work for some and be relatively easy to close timing on.

[langwadt]

SATA cables don't have a requirement to match delays between differential pairs as they are intended for serial data streams with embedded clock. You would typically need a CDR to receive such signals, unless you can oversample. Xilinx provides an app note on how to receive 1.25 Gbps stream using 4x oversampling for 7 series FPGA because their IO tile design allows that, but there is a limit on how slow this can go because they are using delay blocks for alignment, and these have limited max delay.

I think it could be simply done something similar to the WS2812 et.al. LEDs, i.e.

at 2X datarate send for zero 1000, for one send 1110 using DDR. At the receiving end run the data into a pll x1, the pll only uses the rising edge and has duty cycle correction

though at 250Mbit, it might be on the edge for both frequency and PLL input duty cycle

Clean solution, but sadly violates the PLL input specs of most (at least all of the ones I'm familiar with) suitable FPGAs. 100 vs 110 would work for some and be relatively easy to close timing on.

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

[BrianHG]

Dear  ali_asadzadeh,

     Given all the above responses, a few possible routes exist for you.

     Xilinx is not my thing, but if I am to treat you as an absolute beginner, what I can help you with next is tying down your exact specification needs so you may choose which path forward you should go.

First if I were you , I would need to have answers for these questions before proceeding:

When you say 250mb, do you mean 25,000,000 bytes a second, or, 31,250,000 bytes a second?

For your connection, do you mean at least 250mb, with the data going on and off, or, do you mean it will be a nonstop stream at 250mb, never going slower or faster?

What is you source data, some FPGA core code running at 1/8 or 1/10th 250mhz, or code running slower, but you need a 250mb link to guarantee you can send every byte with a little headroom?

Can your destination FPGA's core run a little faster than 25/31.25 MHz to guarantee the core can receive every byte with the occasional blank space since the core is running a little faster than the data, or, do you need your destination FPGA to clock itself from the 250mb data so that the core runs in perfect sync?

These questions above will need to be answered so you may continue with in the right direction.

If all you wanted is parallel 8 bit in with a clock , through a serial port, then parallel out 8 bit data with a reconstructed clock, no FPGA, there exist dedicated IC serializers/deserializers with this function.  Being cheap and dirty, SDI serializer/deserializer will have all these functions build in with a 10 bit data bus, clocking at 27MHz.  (IE: 270mb serial stream, able to drive over 1 meter cables)

[BrianHG]

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

For 3 bit, you need 750mb.  So, for this trick to work, your PLL will need to run at 375 Mhz, driving the OSERDES2 internally in DDR mode to get that 750mb.  That 100/110 trick will create a 250mhz reference with a 33.3/66.6 duty cycle for the PLL input where you will need to run the PLL at 1.5x to get that 375mhz.

Does the Spartan6 have a manual override for the PLL loop bandwidth controls like what you see in Altera Cyclone PLLs?

Or, can you set the PLL to only trigger register exclusively on the rising edge of the input?  (IE: manually set the PLL controls input reference clock to divide by 2 so the PLL's phase comparator internally sees a 50/50 187.5MHz source.  (Lattice FPGA PLLs have a source clock divider option for their PLL's reference.))

Of, can you use your serial input tied to a single posedge clocked TFF, and use that toggling flip-flop's output as a perfect 50/50 187.5MHz PLL reference, running the PLL at 3x instead of 1.5x?  (This was my Altera serial embedded clock trick, and I did need to tune the PLL's phase to capture the middle bit error free.  I literally had my DFFIO tied to a GlobalClock PLL reference input with strict .sdc constraints.)

[ali_asadzadeh]

Thanks guys for the feedback,

When you say 250mb, do you mean 25,000,000 bytes a second, or, 31,250,000 bytes a second?

31,250,000 bytes a second or a 250mbps

For your connection, do you mean at least 250mb, with the data going on and off, or, do you mean it will be a nonstop stream at 250mb, never going slower or faster?

Each PCB has it's low speed ADC, so it would capture it's data and send it's data to next board, so the 250mb is for the last board in the chain, Basically each board capture it's data and will add it's data on top of the receiving packet from pervious board in the chain and resend it to the next board, so the last board should have at least 250mb speed so it can send it's data before new samples arrive.

What is you source data, some FPGA core code running at 1/8 or 1/10th 250mhz, or code running slower, but you need a 250mb link to guarantee you can send every byte with a little headroom?

The code inside the FPGA is simple enough and it would run at lower speed, I need a 250mb link to receive the pervious board data and a separate transmit line to send it to the next board.

Can your destination FPGA's core run a little faster than 25/31.25 MHz to guarantee the core can receive every byte with the occasional blank space since the core is running a little faster than the data, or, do you need your destination FPGA to clock itself from the 250mb data so that the core runs in perfect sync?

I have not decided yet, But I think I have explained enough what the boards will be doing, so I think running the actual Core around 100Mhz is doable inside the Spartan6, and it's fast enough and would give a lot of head room, the problem is the links for receiving and transmitting the stream of data.

[glenenglish]

Why use a Spartan 6 ?  They'll be EOL and unobtanium soon. Yes I know Xilinx said 2030, but AMD own it now.

[langwadt]

Why use a Spartan 6 ?  They'll be EOL and unobtanium soon. Yes I know Xilinx said 2030, but AMD own it now.

it was AMD before the extension, from the spartan6 front page:

"AMD takes our commitment to long lifecycles very seriously. We are pleased to announce that support is formally being extended for AMD Spartan™ 6 devices until at least 2030."

but I agree that you probably shouldn't start a new product design with spartan6 unless it is something where you can jsut buy all the parts you'll ever need right now

[langwadt]

100/110 would be easy enough using 3bit OSERDES instead of just DDR, it would still be outside duty cycle spec for a spartan6 at 250MHz(max 35/65% @ 200-299MHz), but not at <200MHz (max 25/75% @19-199MHz)

For 3 bit, you need 750mb.  So, for this trick to work, your PLL will need to run at 375 Mhz, driving the OSERDES2 internally in DDR mode to get that 750mb.  That 100/110 trick will create a 250mhz reference with a 33.3/66.6 duty cycle for the PLL input where you will need to run the PLL at 1.5x to get that 375mhz.

Does the Spartan6 have a manual override for the PLL loop bandwidth controls like what you see in Altera Cyclone PLLs?

Or, can you set the PLL to only trigger register exclusively on the rising edge of the input?  (IE: manually set the PLL controls input reference clock to divide by 2 so the PLL's phase comparator internally sees a 50/50 187.5MHz source.  (Lattice FPGA PLLs have a source clock divider option for their PLL's reference.))

Of, can you use your serial input tied to a single posedge clocked TFF, and use that toggling flip-flop's output as a perfect 50/50 187.5MHz PLL reference, running the PLL at 3x instead of 1.5x?  (This was my Altera serial embedded clock trick, and I did need to tune the PLL's phase to capture the middle bit error free.  I literally had my DFFIO tied to a GlobalClock PLL reference input with strict .sdc constraints.)

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

[BrianHG]

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

Well, if the op can sacrifice 1-2 byte every fixed period, and it would help if he could sacrifice a 255 or a 0, this would help.  The other choice is to transmit 9 or 10 bits instead of 8 bits, giving him a initial start, index and stop bit.

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

I'm assuming he needs to clock his sampler at a fixed speed for clean sampling, some fixed divider of the 250mb clock.  I do not know how many of these chained samplers he will have running down the chain of boards, but it sounds like each boars samples an 8 bit number, like down at 112.1875khz.  The first boards adds a byte to the serial stream once every 256 bytes.  All concurrent boards sample at the same time, each adding their own additional byte to the stream, feeding out a copy of the incoming serial stream, adding their new byte at the end.  So, once you max out at 256 boards, you will have a continuous filled stream with 256x8 bit samples, 11187.5 times a second. 

@ali_asadzadeh, is this correct?
What is your individual sampler sample speed?
How many board will you be chaining?
Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

It also appears like the OP is using only 1 channel of each SATA cable.  I know ASMI said that the 2 pairs timing aren't guaranteed by the set standards, but with 2x 6gigabit channels and a plug with parallel clamped cables, having a second transmitter for a side data channel which doesn't need such precision phase timing is also at play.

[BrianHG]

Just for reference: https://datasheet.octopart.com/XC6SLX9-2TQG144C-Xilinx-datasheet-10025886.pdf

Page 17 -> OSERDES2 using a speed grade -2, for 3 bit, you can go up to 750Mb/s.  For 4-8bit, you get a little more allowing you 1050Mb/s.  I do not know if this means you are allowed 6bit at 1050Mb/s, but if it does, it can be leveraged for the extra bandwidth, or you will need to code 6 bit chunks sliding into an 8 bit pattern which is easy enough to do.

  If you must use a 4:1 or an 8:1, then you may generate a modified version of the 3 bit pattern 1d0/1d0 into a 4 bit pattern 1dd0/1dd0 where the [dd] represents a single bit of data being fattened for easier capture by the receiver.

The above datasheet also describes the limitation of the PLL and DCM (beginning page 49).  Using the DCM with the input divide by 2, then having it mult by 4, running the 4 bit fat 1dd0/1dd0 pattern would create your clock embedded 1Gs/s stream containing a synchronous 250Mb/s serial channel.

[langwadt]

the Xilinx DCM/PLL only uses the rising edge. The DMC/PLL has an optional div2 on the reference clock and in the manual it is actually mentioned that it to easier meet the duty-cycle requirement, the DCM already outputs both CLK and CLK2x

so I'm sure it could work, but it only a tiny part of OP's project. If 250Mb is actually needed it needs to run faster, some overhead will be need for synchronization and framing

Well, if the op can sacrifice 1-2 byte every fixed period, and it would help if he could sacrifice a 255 or a 0, this would help.  The other choice is to transmit 9 or 10 bits instead of 8 bits, giving him a initial start, index and stop bit.

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

I'm assuming he needs to clock his sampler at a fixed speed for clean sampling, some fixed divider of the 250mb clock.  I do not know how many of these chained samplers he will have running down the chain of boards, but it sounds like each boars samples an 8 bit number, like down at 112.1875khz.  The first boards adds a byte to the serial stream once every 256 bytes.  All concurrent boards sample at the same time, each adding their own additional byte to the stream, feeding out a copy of the incoming serial stream, adding their new byte at the end.  So, once you max out at 256 boards, you will have a continuous filled stream with 256x8 bit samples, 11187.5 times a second. 

@ali_asadzadeh, is this correct?
What is your individual sampler sample speed?
How many board will you be chaining?
Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

It also appears like the OP is using only 1 channel of each SATA cable.  I know ASMI said that the 2 pairs timing aren't guaranteed by the set standards, but with 2x 6gigabit channels and a plug with parallel clamped cables, having a second transmitter for a side data channel which doesn't need such precision phase timing is also at play.

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronized load of all the registers

[BrianHG]

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronized load of all the registers

Yes.
In [1b0] 750mb mode, all you need is 1 sync byte.
Or, in [1b0] 1000mb mode, you can blindly embed start-stop & sync as well as incorporate a parity bit to verify you are sampling the bit in the sweet-spot over every 2 bytes.

The second channel may then be used for reverse direction communications.

[glenenglish]

Brian, sounds llike you miss the days of SDI.
 given these cables are short (< 1m) , I think the use of a transmitted clock on another pair (like the SATA connector) would seem to be a no brainer. For 250 Mbps, I would stick to a transmitted 125 MHz LVDS clock... Perhaps 136 MHz  (272 Mbps) to stay out of the aircraft band...
Its a slow enough data rate that having the clock at the data rate will eliminate the need to resolve the ambiguity.

Of course, using a really slow clock would enable using the edge of the slow clock as a frame / posiiton marker, perhaps.

Ths leaves the requirement for sample rate conversion from local clock domains into the passed packet steam, unless the whole show is synchronous with the beginning of the chain source clock.

Might be useful to do a cascaded jitter analyssssis of a passes through clock, depending on the number of segments, there might become a problem there.

 

[ali_asadzadeh]

Guys thanks for the feedback,
BrianHG thanks for the inputs, I think I'm total noobe in this area, I know PLLs would get an input clock and have a multiplier and a divider to generate another clock in it's simplest from, But I do not get the points  regarding this

We just don't know enough, but having the option to PLL on rising edge only, setting the PLL 3:1, or if you divide by 2 on the PLL input and set the PLL to 6:2, or mult by 2, the got through a general PLL again at 3:1, the op will get his clock.

Please note that nothing is written on stone and I should suggest the customer the solution, so if anything would make sense to me, I would suggest it.

What is your individual sampler sample speed?

Each board would sample around 10Ksps and it has around 125bits of data, so if we need more bandwidth to add additional bytes or info to add frame sync etc... We can go higher in speed, since the IO can go to 750Mbps easily.

How many board will you be chaining?

200 boards

Is the sampler actually 8 bits, or are you just saying 8 bits but it is really 16 bits and you are feeding 2x8bit numbers?
Do you have additional bits to send other than the sync frame?

No, the actual sampling speed is around 10Khz, But each board have some ADC and different Sensors, so the total data is around 125bits, if adding additional bit's would help, we would add them.

In [1b0] 750mb mode, all you need is 1 sync byte.
Or, in [1b0] 1000mb mode, you can blindly embed start-stop & sync as well as incorporate a parity bit to verify you are sampling the bit in the sweet-spot over every 2 bytes.

I do not understand these too, I'm too noobe

with a side channel it would be easier, no need to keep track of number of bytes etc. the whole string could just be a string of shift registers and the side channel trigger a synchronised load of all the registers

I like the Idea of seeing it as giant shift-register, But how? since all the boards should be synced in sampling time.

[BrianHG]

When we say '1b0', what we are saying is if you look as the serial output on a scope, it means you will see:


...HxLHxLHxLHxLH.... and so on...

The 'x' will be set high or low according to the data you wish to transmit.

The 'H' will always be high and the 'L' will always be low.  Every transition L->H looks like a positive edge clock.  If you were to lock your scope on this rising edge, you will see a clean data bit right after 'H'.

So, if we choose a 24bit parallel to serial 1 bit SERDES transmitter, the 24 bit on the parallel input will look like:

"1x01x01x01x01x01x01x01x0", where X would be an 8 bit word...

But, the receiving end needs to know which 'x' is the true first bit to begin.

For the DLL/PLL clock input, we will use the DLL where at every '01' transition, it will divide that by 2 making a clean 125MHz reference clock as the 'x' data bit after the 1 will be random noise.  A simple PLL (Spartain 6 has one of those) can create a single or multiple clock outputs by multiplying and dividing the source clock by a set of fixed integers, like 1,2,3,4,5,6,7,8,9...1023 and also deliver you optional multiple output offset phases.  I'm assuming the Spartain 6 DLL is a bit simpler than it's PLL as it can only multiply by integers of 1,2,4,8,16 and offer something like 4 or 8 optional different phases.

Ok, back to you bandwidth requirements:
125 bits x 10 000 x 200 = 250 000 000.  No room for anything.

Let's say we go for 130 bits, 1 start bit, 125 data bits and 4 stop bits.
(130*10000*200)= 260 000 000 baud, x4bits = 1040mbaud.  The speed limit of the OSERDES2 for the Spartain6.  (I'm using x4 instead of x3)  This means H[xx]LH[xx]LH[xx]LH...  The [xx] need to be the same, IE 1 bit value, they are just twice as wide instead of using the x3 pattern.  This means compatibility with the DLL saving your PLL for something else if needed.

Every time you get a start bit, start your next sample, while feeding through the previous data appending your previous sample to the end of the stream being fed from your previous board to the next board.
Will this work for you?

Each board will sample in parallel with an approximate 5-10ns delay + cable length from each other since they need to decode the serial stream looking for that first 'start/go' bit.  So, board #200's sample will be delayed by ~100-200ns.  Though, with additional coding, you can counteract that 10ns by predicting the start bit's arrival because of it's perfectly repetitive nature.  Basically your internal 10lhz clock will be set to begin sampling early by the 4-8 clocks on the 260MHz side it takes to see the 'start/go' bit.  I'm not sure how you will deal with the cable length, but with a 10khz sample rate, I don't think a global 200ns offset can be interpreted.

If everything is ok, these are my recommended next steps:

Design a SystemVerilog test-bench which will synthesize your master board's serial data chain.  Then when you begin coding for your Spartan 6, you will add that Spartan 6 code in your testbench, feeding it your custom 125bits serial input and see if your FPGA will lock onto your clock data and create a new internal clock from it while decoding and passing all the data though.

Then you can append your own Spartan 6 temporary dummy data onto the stream.

Then you can modify your Spartan 6 code to synthesize it's own master serial data option to replace your test-bench's beginner stream, basically clocking that Spartan 6 from a regular crystal with an IO pin set high or low to define whether it will run as a slave serial input, or run as the first master board from a crystal oscillator input.

Next, add multiple boards of your Spartan 6 code to your testbench, chained together as if wired in real life to verify each board adds it's own data into the stream without errors or missing bits and verify the phase of your internal generated 10khz sampling clock.

Then you may add you data acquisition sampling IOs to feed the true data into the chain.  (This will be a separate testbench just to verify you sampler connections as you already did the com, then you may merge the 200 board setup with the sampling IO version if you like.)

The goal it to create your entire 200 board system in something like ModelSim (So long as Xilinx has it's DLL and OSERDES models for ModelSim, or, whichever simulator Xilinx uses.) and see the entire board-boars system power-up and function.

You want to test everything before even creating a schematic so you know what you build will work.