Electronics > FPGA
Some thoughts on PCI on 3.3V FPGAs
Forty-Bot:
The other thread about PCI reminded me of my recent thoughts on this subject. I'm going to write some of them down before I forget.
PCI has some nice features. It's slow enough that you don't need serdes or other fast I/Os, so it can be used on cheap/slow/FOSS FPGAs. You can use it on a regular PC, no need to add a soft-cpu to control things. And there's a ton of old motherboards with PCI slots. Unfortunately, there are some problems
* First, while there's a 3.3 V version of the bus, almost no motherboards implement it. So you're stuck using 5V. This means either using a 5 V FPGA (which don't get made any more and have outdated tooling) or level shifting. The main problem with level shifters is that the delay will affect your timing. You have 11 ns from when the clock goes high to when your output has to be valid, and 7 ns before the clock goes high when inputs are valid. Depending on your FPGA, you will lose several ns just waiting for your signals to get off-chip (or on-chip). This can be offset a little using PLLs, but then you start to eat into your input setup time. So ideally, you want very fast level shifters, preferably with sub-5 ns propagation delays. Fortunately, even faster level shifters exist. "FET Bus switches" like the CBT or CB3T families (and several others from different vendors) connect the in/outs directly to the source/drain of a FET, with the gate held at a constant voltage. By keeping the gate at 4.3 V or so, you can prevent either side from ever getting above 3.3 V, which is a logic high for 5 V TTL. These level shifters typically have propagation delays on the level of 250 ps, due to RC delay, which is effectively instant. We end up with around 10ns to decide what to do.
* Unfortunately, there's another problem with level shifting. You can have a maximum of 10 pF on any input pin. Around half of that is going to be eaten up by your FPGA. Unfortunately, those bus switches from earlier have a capacitance of 10-15 pF when on (aka functioning), which eats up the other three halves. On top of this, reflections on the bus can cause signals to spike briefly to 11 V or -5.5 V, requiring diodes for protection (the level shifters will typically have clamping diodes, but you need something more sturdy). You can find diodes with suitably low capacitance, like the 1PS302 or BAT63-07W, but they don't usually come in bulk packages, so you'd have to have one per signal. Fortunately, the capacitance is per-slot, so you can always use a multi-slot motherboard and leave the other slot empty.
* The last electrical problem is R_ON. Although not explicitly spelled out in the spec, you'll want to have reasonable R_ON in order to meet the driver requirements, especially since bus switches provide no amplification. Unfortunately, R_ON depends on V_GS, which is capped by the need to perform level shifting. The best I found was around 8 Ohms (e.g. TC7MPB9307FK).
* There are also some trace length limits (<1.5" for signals, and =2.5" for the clock). Presumably these are specified to ensure proper timing, but I'm not sure how everything interacts.
* In terms of timing, while you can take a few cycles for address decoding, and you can wait up to 8 cycles between transfers (and up to 16 on the first cycle), you still have to respond to inputs on the previous cycle for IRDY/TRDY handshaking. This isn't terribly onerous, but you'll certainly have better throughput if you can respond faster.So you can do everything with level-shifting, but it won't completely be in-spec (due to capacitance). What are some other options?
* You can add a PCI-to-PCI bridge, or better yet a PCIe-to-PCI bridge. This lets you work at 3.3 V and skip the level shifter and protection diodes. This is probably the most practical way.
* You can use a PCIe PIPE phy, like the XIO1100. This implements around 2/3 of the PCIe physical layer, so you'd need to do the other 1/3 of the physical layer, the data link layer, and the transaction layer handling on the FPGA. There are some projects implementing the transaction layer, but I couldn't find anything doing the data link layer. PIPE is a 16 bit bus running at 125 MHz, which is much faster than PCI, but not much faster than the 10 ns (aka 100 MHz) you'd need to work at anyway. There aren't any back-to-back responses, and the XIO1100 will align the clock to the center of the data, so it's not all that bad.
DiTBho:
--- Quote from: Forty-Bot on January 22, 2023, 06:37:13 am ---while there's a 3.3 V version of the bus, almost no motherboards implement it. So you're stuck using 5V
--- End quote ---
on my HP RISC workstations
--- Code: ---PCI-32/66 66 MHz 32-bit 266 MB/s 3.3 V
PCI-64/33 33 MHz 64-bit 266 MB/s 3.3 V/5 V
PCI-64/66 66 MHz 64-bit 533 MB/s 3.3 V
PCI-X 100 MHz 64-bit 800 MB/s 3.3 V
--- End code ---
my MIPS router has miniPCI, 32-bit 3.3V data @33Mhz.
PCI-64 3.3V is perfect: it can be downgraded to 32-bit ;D
DiTBho:
--- Quote from: Forty-Bot on January 22, 2023, 06:37:13 am ---The other thread about PCI
--- End quote ---
my topic :o :o :o
berke:
--- Quote from: Forty-Bot on January 22, 2023, 06:37:13 am ---You can add a PCI-to-PCI bridge, or better yet a PCIe-to-PCI bridge. This lets you work at 3.3 V and skip the level shifter and protection diodes. This is probably the most practical way.
--- End quote ---
That's quite interesting, do you have a link to a PCIe-to-PCI adapter that works at 3.3V? It would be surprising if those are readily available if 3.3V PCI boards were a very tiny minority.
But it would be a very neat solution for all the cases where you want to pipe a data stream from a cheap/FOSS FPGA to something with enough RAM and with good reliability. The signals are also slow enough that you have a chance at scoping or analyzing them without having to sell any kidneys.
Forty-Bot:
--- Quote from: DiTBho on January 22, 2023, 04:20:35 pm ---
--- Quote from: Forty-Bot on January 22, 2023, 06:37:13 am ---while there's a 3.3 V version of the bus, almost no motherboards implement it. So you're stuck using 5V
--- End quote ---
on my HP RISC workstations
--- Code: ---PCI-32/66 66 MHz 32-bit 266 MB/s 3.3 V
PCI-64/33 33 MHz 64-bit 266 MB/s 3.3 V/5 V
PCI-64/66 66 MHz 64-bit 533 MB/s 3.3 V
PCI-X 100 MHz 64-bit 800 MB/s 3.3 V
--- End code ---
my MIPS router has miniPCI, 32-bit 3.3V data @33Mhz.
PCI-64 3.3V is perfect: it can be downgraded to 32-bit ;D
--- End quote ---
Yeah, that's certainly the way to go if you have it. But it's much less common on PCs, as they all want to support old 5V-only cards. For example, a quick search shows no motherboards on the first page with 3.3V PCI slots.
--- Quote ---That's quite interesting, do you have a link to a PCIe-to-PCI adapter that works at 3.3V? It would be surprising if those are readily available if 3.3V PCI boards were a very tiny minority.
--- End quote ---
Most bridges will have a dedicated input rail for the I/O supply, so it's pretty easy to select 3.3V or 5V as you please. For example, PEX8311-AA66BCF, PI7C9X111SL, and XIO2001 all support 3.3V PCI busses.
Navigation
[0] Message Index
[#] Next page
Go to full version