FPGA 10gigabit Ethernet

[Thomas83]

Hello,
I am using OpenVPX board based on the Xilinx Zynq from model 5953 from PENTEK.
I need 10 gigabit ethernet communication.
My question is the following, can we use 10 gigabit ethernet instead of the PCIe Gen 3 x8 (8 differential pair Tx and 8 differential pair Rx) bus.
If we can, do we need an external ethernet PHY or not?

Thanks in advance,

[Daixiwen]

You will need a PCB with an SFP+ socket connected to the GTY lines on the FPGA. The you can put any SFP+ module in there to have your 10 gigabit Ethernet connection.

[Thomas83]

Hi Daixiwen,
Thank you for your reply.

If I understood,
I can use the PCIe (GTY) bus which comes out of the P2 connector (Cf. image attached in my previous message) to make 10GbE (fiber or RJ45 cooper)?

[asmi]

Why don't you just insert a 10G network card into PCIE slot and call it a day?

[Daixiwen]

I can use the PCIe (GTY) bus which comes out of the P2 connector (Cf. image attached in my previous message) to make 10GbE (fiber or RJ45 cooper)?

It looks like it's P1 from what I'm seeing (or the VITA connector) but yes basically route a tx ad a rx pair to the SFP+ module. You will also need a small bunch of low frequency signals (i2c, rate select, status...)
The samtec connectors that you show are for board to board connection with your own protocol. It works too, but it's not 10GbE.

[Thomas83]

"You will also need a small bunch of low frequency signals (i2c, rate select, status...)" : what do mean by low frequency signals? Are they necessary for 10GbE?
In the P1 connector pinout , Only 8 paires of PCIE_TX and 8 pairs of PCIE_RX are connected to GTY.

[jeremy]

You will need a low jitter 156.25MHz oscillator connected to the MGT reference clock input, but then yes you can just repurpose the PCIE lanes for SFP modules. I don’t think you can use 10G and PCIE at the same time unless you are careful about which transceiver pins you are using as some will share clock circuitry, and PCIe doesn’t use a 156.25MHz clock.

These modules also have a TX_DISABLE pin which needs to be driven low or the optical transmitter will not be enabled. There is also a rate select pin as mentioned before, but what this does depends on the module. I2C is needed if you want to read out the diagnostic information, but it is not strictly necessary (or at least I have never seen a module where it is required)

[Thomas83]

Hi Jeremy,

I do not know if you have ssen the block diagram of 5953 Pentek Module (Cf. image attached in my previous messages),
We can program the FPGA but the module has PCIe signals and 100GbE fiber that are connected to GTY.

I need 10GbE (because I have 10Gb recorder), It is possible to use PCIe lanes (8 TX and 8 RX)?
do we need an external ethernet PHY ? or we can connected PCIe lanes to SFP+ module and then implement ethernet MAC in the FPGA?
We have the possibility to remap I/O pins of the FPGA.

Thanks in advance,

[jeremy]

Yes, I did see the pic. I have done exactly what you are describing with a 7 series FPGA (I can’t afford the one you have  )

Using 1 TX lane and 1 RX lane of GTY you can do 10GigE with an SFP+ module. You don’t need an external PHY (this is half done in the module, half in logic). You literally just wire them to the module. You also need to tie the other pins high or low as I mentioned and provide power to the module.

You then use the Xilinx 10G Ethernet PCS/PMA and you now have a working XGMII interface in your logic. If you have a license for it, the 10G ethernet subsystem IP also includes a MAC and some sort of AXI memory map to packet conversion.

Only catch is that you must have a low jitter clock, and at least for the 7 series devices, it must be 156.25MHz. I’m not familiar with the clocking structures in the Ultrascale devices.

Also if you have 100GigE QSFP cages and they are unused, why not just use a QSFP+ to SFP+ adapter?

[Thomas83]

Hi Jeremy

1) I have dual 100GigE fiber (VITA 66.4), I think it is 4lanes of 25Gigabit/s.
My  samples recorder accepts 20Gigabit/s max.

2) If I undestood, for example I can connect 1 TX lane and 1 RX lane to SFP module. DO you have any refernce of SFP module?

Tomorrow, I will see if the Ultrascale has 156.25MHz reference Clock and keep you informed.

If you have some schematics I will be happy.

Thank you very much.

[jeremy]

See here for some breakout boards I have used as a reference: http://shop.sysmocom.de/t/development-boards/sfp

SFP+ is the same, just with faster data rates.

[Thomas83]

Sorry for the delayed response,

Thanks to all for your responses.

I was inspired by the schematic of of the KCU105 board.(cf. image)


thank you once again for your help.