Picking a "core frequency" for a FPGA

[NiHaoMike]

What's the process for picking a "core frequency" when starting a FPGA design from scratch? My understanding is that going lower makes it easier to meet timing constraints, while going higher allows more to be done with the FPGA.

If it helps, in my application (solar power inverter) there are 2 important frequencies:
* IGBT PWM, about 15kHz. That is somewhat flexible so unlikely to be a determining factor.
* ADC sample rate, 768kHz, with an interface that runs at 64x the sample rate for 49.152MHz. The ADC (multiple channels) is used both for voltage/current feedback and (with a downconverter) PLC signal receiver.

The FPGA I'm planning to use is a Spartan 6 LX100-3. If I understand the datasheet correctly, the DSP cores are limited to 390MHz and the block RAM to 320MHz. Am I correct that such a high frequency is hard to meet timing on with other logic and therefore the general "core clock" would be some submultiple of that?

[dmendesf]

No, they run "up to" these frequencies (more on that later). You can run them at 1Hz if you wish. Don't get too cosy with these big numbers unless you plan to heavily pipeline your design. Look at the diagrams of these blocks: they have lots of pipeline registers. You can only hit the maximum frequency of you use All of them.

[james_s]

It's just like designing with discrete logic, the faster you clock everything, the faster it does whatever it does, but at some point you start to hit limits and won't be able to meet the timing constraints. The max frequency you can clock a given FPGA depends very heavily on the design you are implementing within it. Each bit of logic a signal routes through introduces a small delay, these delays are cumulative on a signal passing through multiple blocks just as they are in a discrete design built out of 74xx chips.

[NiHaoMike]

I would like to pick a frequency that both allows good utilization of the FPGA and is reasonably easy to meet timing constraints on, in particular because I would like to avoid having to change it (and then have to adjust the blocks already implemented) as I progress in the design.

The key (high level) blocks I'll probably have to implement are:
* PWM controller, with 6 or 7 channels (depending on whether or not I use the FPGA for the buck/boost converter control) each having its own voltage and current feedback and accepting limits from other blocks.
* Grid tie control block, mostly just applies sanity checking. (E.g. make sure it only operates if the voltage and frequency are in range, do a running average on the locally calculated power measurements to make sure the zero export condition stays met.)
* PLC receiver, takes the digitized IF signal from one or more of the ADC channels and decodes the FM signal to get the real time grid current readings. It also decodes some subcarriers to get the phase current imbalance signal and digitally encoded power measurements. (The digital power measurement decoding might be performed on the Raspberry Pi, but I would prefer to put as much logic into the FPGA as would be practical, partly for the learning experience and partly to reduce CPU load.)
* 3 phase motor inverter control, to operate a compressor motor for thermal storage. Might be as simple as V/Hz control or as complex as FOC.
* Single phase UPS, just a simple sine wave generator with limited V/Hz scaling.
* Communication with Raspberry Pi, to implement less timing critical features that would not make sense to implement in the FPGA, for example networking. Most likely would be implemented as a SPI interface exposing a register file to the Pi. Also likely to contain a watchdog timer to handle communication timeouts.
* BMS communication using optoisolated UART. (Might be better to implement in software on the Pi, since critical faults will cause the BMS to switch off the battery contactors.)
* J1772 PWM generator to control EV charging current in response to power system state, future expansion if the need arises. (How fast do EVs respond to changes in duty cycle anyways? Might be better to implement the EV charging station as a networked smart device and let the Pi manage it like any other smart device.)

[james_s]

You're not limited to just one clock either, most FPGAs have a handful of clock input pins and even old ones have at least two PLL's, larger parts have four or more. Each PLL can be driven from any of the clock input pins and can output other frequencies. The PLL's in older FPGAs are somewhat limited in multiplier/divide ratios but more modern ones are extremely flexible.

[Berni]

When running things fast you actually have to run your design trough the timing analyzer feature of your IDE. It will calculate all the round trip signal delays for the logic and determine at how fast of a clock speed you can safely run at.

But as a general rule of thumb pretty much everything will run fine at 1MHz, nearly all sensible designs (No absurdly long strings of combinational logic) will run at 10 MHz, most of the common stuff done on FPGAs run on 50 MHz fine. Things that involve simple logic(Compact logic with particularly short paths) can run at 100 MHz fairly easily. But anything above 100MHz involves carefully designed heavily pipelined designs.

But as others have said you typically won't run the whole design at a single clock speed. Things like serial I/O interfaces might run at high clock speeds to get the required bandwidth. For example once that fast SPI bus is decoded the result might be a 8bit or 16bit parallel bus that only needs to run at 1/16 the clock speed.

[Bassman59]

What's the process for picking a "core frequency" when starting a FPGA design from scratch?

Usually, there is something in the whole design that gives you a clock or otherwise has timing requirements you must meet. For example, on my current design I have an external transceiver that runs on a 100 MHz clock, so it is necessary to run the part of the FPGA that talks to that device on that same clock. It is convenient to run other, not necessarily related, logic on that clock, too. The design also has ADCs that run on a 25 MHz clock with DDR serial output at 150 MHz, so after deserialization those data are written to buffers on 25 MHz. The data are read out on the other side on the global 100 MHz clock.

Or, consider a digital audio device that talks to converters. A common modulator clock for converters is 24.576 MHz with bit clock and L/R clock divided from that. So it is convenient to run your processing on that same 24.576 MHz clock. The cool thing about FPGAs is that if the 24.576 MHz clock is not fast enough for your processing (say your filter has a lot of taps that need calculation in one sample time) you can use an internal PLL to double or quadruple the clock frequency inside the FPGA.

If there are no externally-imposed constraints, consider what you are trying to accomplish. Does it "need" to run at 200 MHz, or does 50 MHz suffice? Put another way, what is the slowest clock frequency necessary for the design to function? Remember, slower means reduced power consumption.

[filssavi]

It depends on the particular FPGA series you are targeting, for example for ice 40 going much past 50 MHz will be very painful, on Xilinx series 7 100MHz is my base goto frequency.

What I will tell you is, if you foresee the need to go fast in the future (even distant one) plan to pipeline the design from the start, as doing it at a later Date is much much more painful (and likely to fail nonetheless)

[BrianHG]

We had this problem with the Z80 VGA project :
https://www.eevblog.com/forum/fpga/fpga-vga-controller-for-8-bit-computer/msg3096900/#msg3096900

I wanted to address up to 500 million pixels per second even though we were only drawing 25 million per second to allow random addressing and multiple addressed transparency window layers over each pixel.  However, for the CycloneIV 250Mhz let alone 500Mhz would not be possible to fit on this FPGA.  So, what I did was make the entire core a safe easy to fit 125MHz and just for the true dual port ram, I ran that ram at it's 250MHz speed limit and had a 2 bank A/B switch MUX at the address/data/we inputs to the ram, and another A/B mux at the data outputs making a true 125MHz, 4 port read/write ram.

So, if you want a 320MHz dsp stream tied to internal ram, or lower that to 300MHz, a safe core frequency would be 1/2 at 150Mhz.  If you want functionality under multiple speed of the FPGA and want wide temperature functionality, I would lower these 2 clocks a little further.  Unless you want to deal with multiple clock domain, using x2 for a similar strategy to mine is probably be the easiest way to go as a single FPGA pll will be used to generate both outputs in perfect sync.

[filssavi]

You're not limited to just one clock either, most FPGAs have a handful of clock input pins and even old ones have at least two PLL's, larger parts have four or more. Each PLL can be driven from any of the clock input pins and can output other frequencies. The PLL's in older FPGAs are somewhat limited in multiplier/divide ratios but more modern ones are extremely flexible.

That is true you can do mutiple clock domains, if you really need, however I would stay as far from it as humanely possible, as for any non trivial design, clock domain crossing is extremely painful for several reasons:

1) it is a royal PITA to get working right 100% of the times with a fixed clock ratio, if that is variable god help you

2) a non trivial CDC will need a full three way handshake, introducing at least 3 clock cycles latency (probably more once you factor in pipelining registers). That is fine for some applications, e.g. video processing, where you are mostly with throughput and the pipeline filling time is just a tiny fraction of your processing time. It is kind of undesirable in other applications, e.g. power electronics where low latency is the name of the game,  there the processing time for the vast majority of algorithms will be comparable to the time it takes to fully fill the pipeline and do the CDC, thus having separate clocks can basically half your maximum loop frequency (not that critical seen how fast the clocks are compared to the switching frequency);  and give you a higher latency on both control and safety input to output paths, this can either mean worse performance (especially for load transients and ripple/noise rejection) or even the difference between a safety shutdown and a rapid unplanned transistor decapping (depending on the device itself)

P.S. regarding the networking/raspberry side, have you taken into consideration the xilinx Zynq line (or intel cyclone SoC if you are in that camp) they have both processor cores capable of running linux and the fpga section, bonus points, the processing cores are tied with a much higher performance bus (AXI) so you can actually implement PWM, safety, heavy DSP signal processing in fabric and run the control laws on the processor (which is much easier than doing everything discretely in an FPGA). Being the processors multicore you can use a core with linux for the networking (so you dont have to either implement the network stack yourself or use a crappy embedded one) and use the other core to run hard real time bare metal code for the control.

[tggzzz]

What's the process for picking a "core frequency" when starting a FPGA design from scratch? My understanding is that going lower makes it easier to meet timing constraints, while going higher allows more to be done with the FPGA.

In many applications the clock frequency is dictated by the i/o clock rates, either directly or indirectly as a multiple or fraction. That's what the various PLL clock primitives are for.

You're not limited to just one clock either, most FPGAs have a handful of clock input pins and even old ones have at least two PLL's, larger parts have four or more. Each PLL can be driven from any of the clock input pins and can output other frequencies. The PLL's in older FPGAs are somewhat limited in multiplier/divide ratios but more modern ones are extremely flexible.

That is true you can do mutiple clock domains, if you really need, however I would stay as far from it as humanely possible,

Usually you can't get away with it completely.

If external devices have their own clock domains, you will have to cross at least one clock domain. That's true even if they are nominally the same frequency, since the phase would be indeterminate and the frequency not quite the same (i.e. variable in your statement below).

And then there are the large FPGAs, where the cross-chip delay is comparable with the gate delay.

as for any non trivial design, clock domain crossing is extremely painful for several reasons:
1) it is a royal PITA to get working right 100% of the times with a fixed clock ratio, if that is variable god help you

2) a non trivial CDC will need a full three way handshake, introducing at least 3 clock cycles latency (probably more once you factor in pipelining registers).

It is best to plan on having to bite the bullet somewhere, and then choosing the least inconvenient place to do it.

Rules of thumb when assessing unfamiliar designs and proposals...

With software systems, ask for a definition of the scope and lifetime of "addresses" and "identifiers". With hardware systems, ask where the clock domain boundaries are, and how signals cross them.

The (lack of decent) answers to those fundamental questions is often a good starting point for probing deeper.

[nctnico]

I would like to pick a frequency that both allows good utilization of the FPGA and is reasonably easy to meet timing constraints on, in particular because I would like to avoid having to change it (and then have to adjust the blocks already implemented) as I progress in the design.

Based on my experience with the Spartan6 and FPGAs in general: Around 50MHz is a good ball-park for the Spartan6. If you need faster clocks then use the internal PLL to create multiples. The software will automatically make sure the crossing between related clocks is handled well in the timing constraints.

Using unrelated clocks isn't a big problem but it takes extra constraints to tell the place & route about the timing relation between the clocks (besides having clock domain crossing circuitry).

And then there are the large FPGAs, where the cross-chip delay is comparable with the gate delay.

No. The correct phrase is:  And then there are the large FPGAs, where the cross-chip delay is comparable with far exceeds the gate delay. Or put differently: it is possible to choose an FPGA which is so oversized that meeting timing is difficult due to the cross chip delays alone even with the design being clocked at around 65MHz.

[tggzzz]

And then there are the large FPGAs, where the cross-chip delay is comparable with the gate delay.

No. The correct phrase is:  And then there are the large FPGAs, where the cross-chip delay is comparable with far exceeds the gate delay. Or put differently: it is possible to choose an FPGA which is so oversized that meeting timing is difficult due to the cross chip delays alone even with the design being clocked at around 65MHz.

Snort! I wouldn't argue with that, except to nit-pick that such a large FPGA might not be oversized.

[filssavi]

I am well aware that there are occasions where I/O will decide the clock speed, however I don't think that is the OP case:

- Anything Power electronics related can easily be dealt with by clock enable signals, since the frequencies are three orders of magnitude lower
- For the ADC sampling, The frequency is high enough, and the design is simple enough (FOC is not that resource intensive on a modern FPGA with dozens of DSP blocks) that I would run the entire design at that frequency
- The big question is the PLC communication, more info are needed on that, however I would argue that using a ready made module would save loads of time, and would be worth the hassle in the end, as you don't need to worry about all the EMI/safety testing needed, moreover if price is your concerns you have bigger fishes to fry, namely instead of an FPGA, you would have gone with a midrange MCU targeted at power converters (TI delfinos and concetos C2000, NXP kinetis k5x, NXP mcp5600/5700 etc) that will result in a much cheaper and simpler system (just 1/2 digital supplies with modest requirements, no external flash, cheaper to begin with, etc). And anyway if you really want to roll your own you should be able to just use a regular 1-3 MSPS SPI ADC and still be within the nyquist limit, in order even do the demodulation in digital (I'm assuming your bandwidth is somewhere in the tens/hundreds of kHz, if it is higher than a Mhz than that might indeed be an issue)

[rstofer]

It looks to me like the ADC drives the design.  Is there a timing advantage to using multiple ADCs in parallel?

I have never used the Spartan 6 and, given that the current flavor is the  7 family (Spartan-7, Artix-7, Kinex-7, Virtex-7), I'm not sure I would without talking to Xilinx Marketing.  Some devices in the Spartan 6 family are hitting End Of Life.

Worse yet, the Vivado toolchain doesn't support the obsolete Spartan 6 family and you would be stuck using ISE 14.7 which is no longer supported.  Moving forward, well, there is no way forward...  The device is obsolete, the toolchain is obsolete and your design is unsupportable as soon as it is complete.

[rstofer]

I am well aware that there are occasions where I/O will decide the clock speed, however I don't think that is the OP case:

- Anything Power electronics related can easily be dealt with by clock enable signals, since the frequencies are three orders of
magnitude lower

I was going to make the same comment but I wasn't confident that I knew anything about the application.  That sample rate seems high for power control.  Given that the ADC clock is many times higher than the sample rate, the clock rate gets out of control pretty quick.  If the sample rate is driven because the analog input is multiplexed, maybe multiple ADCs will slow things down.

I'm thinking 10 MHz, not 100 MHz and I'm thinking PIC, not ARM (not seriously...).  There are some pretty fast ARM devices, maybe the Teensy 4.1 is fast enough at 600 MHz.  It uses the NXP IMXRT1062 chip.  These specs are shared over the Teensy 4.0 and Teensy 4.1:

https://www.pjrc.com/store/teensy40.html

[filssavi]

I am well aware that there are occasions where I/O will decide the clock speed, however I don't think that is the OP case:

- Anything Power electronics related can easily be dealt with by clock enable signals, since the frequencies are three orders of
magnitude lower

I was going to make the same comment but I wasn't confident that I knew anything about the application.  That sample rate seems high for power control.  Given that the ADC clock is many times higher than the sample rate, the clock rate gets out of control pretty quick.  If the sample rate is driven because the analog input is multiplexed, maybe multiple ADCs will slow things down.

I'm thinking 10 MHz, not 100 MHz and I'm thinking PIC, not ARM (not seriously...).  There are some pretty fast ARM devices, maybe the Teensy 4.1 is fast enough at 600 MHz.  It uses the NXP IMXRT1062 chip.  These specs are shared over the Teensy 4.0 and Teensy 4.1:

https://www.pjrc.com/store/teensy40.html

My interpretation is that he has a multiple channel adc (like the LTC2324) that does simultaneous sampling of few currents and/or voltages at the same times, and the 50ish MHz frequency is the output data clock rate on the SPI/LVDS line. in power electronics the sampling is usually tied to the switching frequency (by far the most common thing to do is to sample stuff just once per cycle), even if he wants to do some oversampling and digital filtering to increase resolution/filter noise, there is no point in going faster than few hundred ksps sampling rate as the rest of the circuit (parasitic included) won't go that fast.

Now that is my assumption, so I stand to be corrected.

[OwO]

For an Artix/Spartan 7 speed grade 1, the ballpark numbers are:
100MHz - trivial to achieve, any trash design will meet timing
200MHz - easy to achieve, any reasonable design will meet timing
300MHz - doable without too much effort, requires some pipelining
380MHz - this is the block ram Fmax, a well optimized design will hit this limit.
460MHz - this is the multiplier Fmax. Specific components within the design may run at this speed.
500MHz - this is the shift register Fmax. Specific components within the design may run at this speed.

[NiHaoMike]

P.S. regarding the networking/raspberry side, have you taken into consideration the xilinx Zynq line (or intel cyclone SoC if you are in that camp) they have both processor cores capable of running linux and the fpga section, bonus points, the processing cores are tied with a much higher performance bus (AXI) so you can actually implement PWM, safety, heavy DSP signal processing in fabric and run the control laws on the processor (which is much easier than doing everything discretely in an FPGA). Being the processors multicore you can use a core with linux for the networking (so you dont have to either implement the network stack yourself or use a crappy embedded one) and use the other core to run hard real time bare metal code for the control.

Overkill considering I don't need much bandwidth between the FPGA and Raspberry Pi and nothing done over the network needs really low latency.

- The big question is the PLC communication, more info are needed on that, however I would argue that using a ready made module would save loads of time, and would be worth the hassle in the end, as you don't need to worry about all the EMI/safety testing needed, moreover if price is your concerns you have bigger fishes to fry, namely instead of an FPGA, you would have gone with a midrange MCU targeted at power converters (TI delfinos and concetos C2000, NXP kinetis k5x, NXP mcp5600/5700 etc) that will result in a much cheaper and simpler system (just 1/2 digital supplies with modest requirements, no external flash, cheaper to begin with, etc). And anyway if you really want to roll your own you should be able to just use a regular 1-3 MSPS SPI ADC and still be within the nyquist limit, in order even do the demodulation in digital (I'm assuming your bandwidth is somewhere in the tens/hundreds of kHz, if it is higher than a Mhz than that might indeed be an issue)

The low latency requirement makes ready made PLC solutions like Homeplug unusable for this application. The inverter has to respond to the current sensor in time to more or less cancel out the load current. It's also why I went with a FPGA - the inherent parallelism will allow less latency from the PLC input to the power electronics.

The PLC signal (from the current sensor to the FPGA) is on the order of a few hundred kHz to a few MHz with on the order of 100kHz bandwidth at most. I'm planning on implementing receive diversity to increase robustness to noise and might even implement a slow reverse PLC channel (FPGA to current sensor) by varying the PWM at some submultiple of the switching frequency. The latter could allow tricks like changing the forward PLC frequency in order to avoid narrowband noise.

It looks to me like the ADC drives the design.  Is there a timing advantage to using multiple ADCs in parallel?

The ADC channels are not used in parallel, they're monitoring different voltages and currents in the system. The inverter module itself has 6 outputs plus a buck/boost stage, so there would be 6 voltages and 6 currents that need to be sampled very fast along with 2 voltages and 1 current that need to be sampled at a somewhat slower rate. Most likely I'll have a dsPIC dedicated to controlling the buck/boost stage since that doesn't need the kind of flexibility the FPGA offers.

I have never used the Spartan 6 and, given that the current flavor is the  7 family (Spartan-7, Artix-7, Kinex-7, Virtex-7), I'm not sure I would without talking to Xilinx Marketing.  Some devices in the Spartan 6 family are hitting End Of Life.

Worse yet, the Vivado toolchain doesn't support the obsolete Spartan 6 family and you would be stuck using ISE 14.7 which is no longer supported.  Moving forward, well, there is no way forward...  The device is obsolete, the toolchain is obsolete and your design is unsupportable as soon as it is complete.

I already have several of the Spartan 6 boards I got for cheap (about $10 each) and it's a one off for personal use and as a fun learning experience.

[filssavi]

Ok given the clarification I am fully convinced you can set your system clock rate at whatever you need for the ADC converter and be done with it, it should be fast enough for what you need, you have no CDC to worry about, timing closure is basically unnecessary if the high level design is even minimally thought out.

[Bassman59]

Ok given the clarification I am fully convinced you can set your system clock rate at whatever you need for the ADC converter and be done with it, it should be fast enough for what you need, you have no CDC to worry about, timing closure is basically unnecessary if the high level design is even minimally thought out.

Timing closure is always necessary.

[filssavi]

Ok given the clarification I am fully convinced you can set your system clock rate at whatever you need for the ADC converter and be done with it, it should be fast enough for what you need, you have no CDC to worry about, timing closure is basically unnecessary if the high level design is even minimally thought out.

Timing closure is always necessary.

I might have expressed what I wanted to say wrong, timing analysis and proper I/O constraints are always needed, However at such low frequencies on Modern FPGAs there are extremely good chances that even a poor design will meet timing at the first try, given that few very basic design principles are followed (do not try to do everything in a single clock cycle, keep asynchronous logic to the required minimum, etc).

[Bassman59]

Ok given the clarification I am fully convinced you can set your system clock rate at whatever you need for the ADC converter and be done with it, it should be fast enough for what you need, you have no CDC to worry about, timing closure is basically unnecessary if the high level design is even minimally thought out.

Timing closure is always necessary.

I might have expressed what I wanted to say wrong, timing analysis and proper I/O constraints are always needed, However at such low frequencies on Modern FPGAs there are extremely good chances that even a poor design will meet timing at the first try, given that few very basic design principles are followed (do not try to do everything in a single clock cycle, keep asynchronous logic to the required minimum, etc).

Yes, I agree.

Always have that clock period constraint. FPGA tools can do some really dumb things!

[filssavi]

That would be a GIGO problem (garbage in garbage out)

Most of the FPGA P&R tools are timing driven , thus they need to know clock period and I/O setup/hold times. Either you provide them with constraints, otherwise they will either try and guess or just treat everything as a standard signal, with the subsequent shitty job

[NorthGuy]

For an Artix/Spartan 7 speed grade 1, the ballpark numbers are:
100MHz - trivial to achieve, any trash design will meet timing
200MHz - easy to achieve, any reasonable design will meet timing
300MHz - doable without too much effort, requires some pipelining
380MHz - this is the block ram Fmax, a well optimized design will hit this limit.
460MHz - this is the multiplier Fmax. Specific components within the design may run at this speed.
500MHz - this is the shift register Fmax. Specific components within the design may run at this speed.

464 MHz - max speed for BUFG/BUFH - that's the max clock speed you can achieve for your logic
600 MHz - max speed for BUFIO - that's the max speed you can achieve for IO operations
800 MHz - max clock speed you can get from  MMCM or PLL. Nothing you can do with it.

-2 grade is somewhat better.