Low-Maintenance Configuration ROMs for FPGA Design

[exmadscientist]

We have an existing design based around an off-the-shelf FPGA daughterboard (carrying a Spartan-6). We'd like to replace the expensive off-the-shelf module with our own in-house design, which is straightforward enough to do. The sticking point for me is how to get the FPGA configuration ROM programmed in a simple and straightforward manner. The module we've been using has its own very slick USB-based firmware loader, which works wonderfully but that we have neither the resources nor the inclination to duplicate.

This particular design already has multiple firmwares that need to be loaded and the absolute last thing I want to do is add yet another microcontroller that has to be programmed before we can even program the FPGA. To say nothing of maintaining another piece of firmware. So the "obvious" solutions of just programming a small MCU are unattractive.

This has to be a problem that's been tackled before and solved, but I'm coming up blank... hence the question. Some ideas:

• Use Xilinx iMPACT for programming. I admit to not knowing much about iMPACT and need to remedy that. The biggest question is whether it requires an ISE (or Vivado) install to run; I would prefer something more lightweight. That said, this is currently the leading option.
• Use an FTDI or similar USB interface chip to load the ROM. Do PC-hosted programming utilities exist to do this? I couldn't find any, but it seems like something that's out there somewhere.

The "ideal" would be a chip I could drop in to provide a workflow like the mBed or certain NXP LPC MCUs, where you boot the thing with the right configuration jumper settings and the configuration memory shows up on a PC as a USB mass storage device, ready to have the firmware image drag-and-dropped to it. (The best part of making programming this simple is that I could pass the job off to someone else!)

We are planning for a low-volume (around quantity 50) production run: small enough that we can do the programming and inevitable reprogramming ourselves, but large enough that we can allocate some of the BOM cost to fancy ICs that simplify life. Right now this design is at back-of-napkin level of detail, so there's plenty of flexibility.

Thoughts?

[radar_macgyver]

Xilinx provides source for an XSVF player (XAPP058), there are also third-party versions like libxsvf. These let you write hook functions that the library calls when it needs to set TDO, TCK or TMS to a specific value, and another to read the state of TDI. If you use an FTDI chip, you can write hook functions that put the chip into MPSSI mode and bit-bangs the JTAG pins.

EDIT: looks like one of the examples for libxsvf targets FTDI chips.

[Kilrah]

You can cetainly replicate the JTAG/config memory designs from open boards like those from Digilent.

[kc8apf]

The mBed-style USB mass storage is done by having another micro that would need to be programmed first. 

[dmills]

If you are doing JTAG for boundary scan anyway then you can usually program the config device over the same JTAG link and do it as part of running the JTAG test sequence.

We have tended to find however that an external micro is sometimes actually the beast approach as it allows you to upload a single file containing several FPGA images and have the correct one be selected at system startup (Put some pin strapping jumpers around the micro so the thing can figure out what to load).

Being able to plug in a USB key to do a field update is very, very cool.

Regards, Dan. 

[Ice-Tea]

Not sure I completely understand your needs down to the fine details... Do you need to have a a number of configurations stored and depending on the jumpers load a specific one? Or do you need to load just one image in it, depending on the configuration?

Anyways, there is a standalone iMPACT install. You can select the image you want to load, you click "program" and that's about it. If I remember correct, you can configure the tool and store the configuration for later use.

If you need to store multiple images, I think Xilinx has a mechanism for that too (don't know the details). You could also use the jumpers to select the CS to multiple EEPROMs....

[rstofer]

The classic method of using the JTAG chain is still about the only way to program the flash without having to add a microcontroller (which will program the flash with SPI).  Make sure to use the appropriate flash for either JTAG or SPI.  Perhaps the flash selection is also dependent on the actual FPGA.  I haven't followed along because all of the modern Digilent boards come with USB programming hardware on the board.

Digilent sells little modules that implement USB -> JTAG or SPI plus a couple of more advanced protocols:
http://store.digilentinc.com/all-products/jtag-programmers/
You just solder them on top of your PCB and call it a day.

The suggestion to copy an existing board is entirely valid.  Digilent, among others, posts the schematics for all of their products.

If I had 50 boards to make, I would implement the JTAG chain and just use Digilent's Adept software and a USB->JTAG cable.  It's fast and it only takes a 6 pin header.  In fact, if you look at some of Digilent's boards you will see they omitted the physical header and, instead, put the 6 pads/vias in a staggered row.  The stagger allows them to shove a 6 pin male header in place and make good connections on all of the pins.  Pretty clever!

[Rasz]

I have an idea, how about you migrate to ... off-the-shelf FPGA daughterboard with very slick USB-based firmware loader? ))

[jbb]

Hi exmadscientist

You can definitely do it!  There's no need to fork out for the expensive Xilinx flash chips.

IMPACT is an OK tool - can be called from command line for scripting.  Can also be used to develop multiboot image files.  I believe you can download a LabTools bundle which is big (200MB I think) but not as crazy as the full Xilinx tools (which can be >= 7GB when installed!!!!!!)

Unfortunately, when you move from an FPGA dev board to making your own you have to read a LOT of documentation.  In this case, I suggest you look at the Spartan 6 configuration guide (http://www.xilinx.com/support/documentation/user_guides/ug380.pdf)

Here's what we did in a previous (successful) board:

• We used an SPI flash memory (in our case AT45DB321E, cheaper ones available) for binary storage.
• We connected it to a Spartan 6 FPGA as per Figure 2-12 in UG380
• We "program the FPGA" using JTAG as per Figure 2-12
• You can program the FPGA directly with JTAG.  This is quick and good for dev work. But the config is lost when you cycle the power or hit PROG_B (will load code from SPI flash if possible).
• You can "program the flash" via the FPGA with JTAG.  What happens here is the FPGA is configured by IMPACT with a loader application, and then the SPI flash is loaded through it.
• Don't screw about with daisy-chained JTAG devices until you've got something working standalone!

There are two extra factors to consider:

• You can turn the SPI pins back into GPIO and access the SPI memory from you application.  This allows tricks like in-field firmware update (see section Reconfiguration and Multiboot of UG380) and using spare flash for application data (parameters, logs, software binaries, you name it!) but prevents you from doing the automatic continuous verification of the FPGA config (think aerospace/cosmic ray stuff, you'll know if you need it.)
• The SPI loader application is NOT documented (Xilinx doesn't really make it clear what's going on) and puts all the GPIO pins into an undocumented (note: not undefined) state.  I think weak pullup. Make sure whatever you've got connected to the FPGA can survive that.

Another really important thing to consider is what will happen to the IO pins during configuration.  One of the FPGA pins has something to do with their unconfigured state.  You may need to add external pullups/pulldowns to make sure nothing goes wonky.  Especially if you're connected to something that could be dangerous!

My final piece of advice: just pay for a proper JTAG programmer. It will save you a lot of pain. The Xilinx Platform Cable II is quite reasonable when you think about how much time you might save. The Digilent XUP programming cable (6003-410-011) may also do the trick.

(edit: fixed broken list format)

[suicidaleggroll]

IMPACT can write to a SPI-attached flash chip using the standard JTAG header.

IMPACT + Digilent JTAG-HS2 + Spartan 6 + compatible SPI-attached flash chip is all you need.  As for a compatible flash chip, IMPACT has a big list, and in the past I've used the Macronix MX25L6445, just tell IMPACT it's the "S25FL064P" when it asks you for the PROM type.  This is the same flash chip used on the Papilio Pro, you can reference their documentation and schematics to verify you're wiring things up correctly.

[uChip]

What jbb and suicidaleggroll said.

It took wading through several docs and schematics for me to figure out how to set this up.  The problem is not lack of documentation, but rather too many options.  The reference boards sort of helped but they didn't in that they were also TOO flexible.  They were wired for three or more potential solutions.  I just wanted the minimal set of pins to make it work.

The attached (partial) schematic shows just the pins involved with configuration.  This (the chip on the left) is for a Spartan-3A but I make the dangerous assumption that the Spartan-6 is very similar.  The no-connects on Mx and VSx inputs are pulled high internally.  If you are using those pins in your application then you need to make sure they are driven to the appropriate state during config time.  I use a digilent HS2 to program.  It's cheap and works fine.  The RC on PROG_B can be replaced with a system reset.  It's just there to hold off config until the SPI FLASH is ready.



  - Chip

[nctnico]

What I'm missing from the options is to load the FPGA from a microcontroller which is already onboard. In one of the products I worked on the FPGA images where part of the firmware and loaded by the microcontroller at startup.
Using IMPACT through the JTAG interface is also possible but extremely slow. Say 5 to 10 minutes for a Spartan 6 45LX part. Also the IMPACT user interface is very unstable (prone to crashing) so I'd avoid using IMPACT for production use.

In another project I'm involved in the FPGA is initially loaded using IMPACT but for field updates the FPGA can reprogram/update the SPI flash from the firmware it runs.

All in all the fastest (=shortest programming time) option would be to use a SPI capable programmer from a programming header which can keep the FPGA quiet for as long as the programming header is active.

[legacy]

In another project I'm involved in the FPGA is initially loaded using IMPACT but for field updates the FPGA can reprogram/update the SPI flash from the firmware it runs.

this trick is used by a company in Canada (their name doesn't come to my mind now), with both fpga's config flash and Blackfin's one
the fastest solution ever!

[radar_macgyver]

The RC on PROG_B can be replaced with a system reset.  It's just there to hold off config until the SPI FLASH is ready.

Is this normally an issue? I ask because in all my designs so far, I've used a Xilinx XCF flash chip. For my next design, I was thinking of using a SPI flash.

[jeroen74]

Using IMPACT through the JTAG interface is also possible but extremely slow. Say 5 to 10 minutes for a Spartan 6 45LX part.

I wonder why that is... Altera has a comparable system and that's super fast, mostly dominated by the full-chip erase time of ~15s. Actual programming takes only seconds (configuration for a EP1C6 and EP1C3).

JTAG UART thingy in Xilinx SDK is also painfully slow (and again) Altera's solution is speedy.

[nctnico]

The problem with JTAG is that it can be a very poor interface if a lot of data must be transferred through channels which have not been designed for that purpose. A very long time ago I worked on a MIPS platform and to program the external flash the CPU had to single step through several instructions for sending one word to the flash. Needless to say it took many (100 or even more) JTAG clock cycles to program one word. I guess something similar is going on when programming an external SPI flash from IMPACT. All in all I try to avoid using JTAG interfaces if I can because beyond the TAP statemachine there is nothing standard about it!

[jbb]

Very true, ncnitro

All of the 'program via JTAG' stuff is non-standard, which is why performance is so variable, and daisy-chaining (especially with different makes of chip) can be a disaster.
We found that loading the image into flash for an XC6SLX9 (2.7 Mbit) took about 5 seconds, which was fine for us.  By scaling up the time linearly, loading an LX150 could take up to a minute.  It's up to you to determine whether that's acceptable.

For those using the Avnet LX9 dev board (it's good!) http://products.avnet.com/shop/en/ema/development-kits/3074457345623514331, you may have noticed that programming the SPI flash is devastatingly slow.  This is due to the onboard emulator.  If you connect it to your PC via a USB2.0 hub it will magically get a lot faster.  Issue doesn't seem to impact loading the FPGA directly.

[radar_macgyver]

The way Xilinx implements SPI flash programming with Impact is to first configure the FPGA with a Flash writer tied to the BSCAN primitive, then send commands down JTAG to write to the Flash. Once done, the device configures itself from SPI. I can see why this can be time consuming.

On one design I've worked on (Virtex-6 LXT240, which has more configuration cells than the largest Spartan 6 device), Impact completes programming the FPGA in about 10-15 seconds, this is using a Platform USB II cable. If it takes minutes to do this, it might be using a lower clock speed to do the transfers. I leave mine at 12 MHz.

[exmadscientist]

The mBed-style USB mass storage is done by having another micro that would need to be programmed first.

Right, but I'd be willing to pay for such a thing in prepackaged form, if it existed.

Not sure I completely understand your needs down to the fine details... Do you need to have a a number of configurations stored and depending on the jumpers load a specific one? Or do you need to load just one image in it, depending on the configuration?

Just a single image, but it will of course need updating as bug fixes/feature enhancements/etc roll out.

I have an idea, how about you migrate to ... off-the-shelf FPGA daughterboard with very slick USB-based firmware loader? ))

Several reasons I want to replace this thing:

• Cost. This thing is really, really expensive compared to what we get out of it. We could also go down two FPGA size grades if we had any choice in the matter... which we don't with this dev board. (And before anyone goes there, this thing was chosen by my now-long-gone predecessor and we're locked in to it for now.)
• Form factor. Right now this daughterboard determines the final mechanical dimensions of the whole thing. We could shrink it significantly if we weren't so constrained.
• BOM simplification. I don't like having to maintain inventory of an expensive part not stocked at the usual distributors, and making sure we have one per assembly available, every time we build an assembly.
  
• Power. Its power circuitry is neither robust nor particuarly efficient. Also, there's a DDR2 memory onboard that we don't even use, which wastes about a watt.
• Fragility. The dev board is not particularly robust and we have already killed several due to both a complete lack of protection on the power input and a power connector that is trivially mated to the wrong thing, frying the entire board if your attention lapses for a moment.
  
• Miniaturization. Once we have an in-house FPGA design that's tested and known-good, next revision we can move it onto the mainboard and drop the daughterboard entirely. Then we can integrate and shrink things a lot, which matters for our (portable) device.

You can definitely do it!  There's no need to fork out for the expensive Xilinx flash chips.

I wasn't planning on going that route unless it simplified the software dramatically, but they'd be well within our budget if they are the right choice.

Unfortunately, when you move from an FPGA dev board to making your own you have to read a LOT of documentation.  In this case, I suggest you look at the Spartan 6 configuration guide (http://www.xilinx.com/support/documentation/user_guides/ug380.pdf)

I've already leafed through UG380 a couple of times, but haven't given it a serious readthrough yet. As I've said, I am most focused on keeping the software simple, and then the details of the hardware design will follow from that goal.

The SPI loader application is NOT documented (Xilinx doesn't really make it clear what's going on) and puts all the GPIO pins into an undocumented (note: not undefined) state.  I think weak pullup. Make sure whatever you've got connected to the FPGA can survive that.[/li][/list]

Another really important thing to consider is what will happen to the IO pins during configuration.  One of the FPGA pins has something to do with their unconfigured state.  You may need to add external pullups/pulldowns to make sure nothing goes wonky.  Especially if you're connected to something that could be dangerous!

I believe our COTS dev board does something similar during programming, so we should be OK there. (Also, there's nothing particularly fancy attached.) Still, always something to keep an eye on.

My final piece of advice: just pay for a proper JTAG programmer. It will save you a lot of pain. The Xilinx Platform Cable II is quite reasonable when you think about how much time you might save. The Digilent XUP programming cable (6003-410-011) may also do the trick.

Oh, we'll definitely be doing that once we move off of this bloody board! The Platform Cable is a fraction of the price of one of these things. I'll have no problem ordering one. Or two. Or three.

Anyway, thanks everyone for the pointers. I'm particularly grateful to hear the real-world experiences with iMPACT, as it sounds like it will do exactly what we really need. (Xilinx documentation, while good at describing details, isn't always good for choosing the right way to do something given a particular use case.)

[Ice-Tea]

Not sure I completely understand your needs down to the fine details... Do you need to have a a number of configurations stored and depending on the jumpers load a specific one? Or do you need to load just one image in it, depending on the configuration?

Just a single image, but it will of course need updating as bug fixes/feature enhancements/etc roll out.

That should be easy enough. Done that a million times (both Altera and Xilinx). iMPACT is the way to go there, especially if batch sizes are relatively small. If and when batch sizes get bigger you could pre-program the SPI chips (choose an SO-package for easy out-of-the board programming) or you could use a pin-bed with an ISP connector. For even bigger production batches (with a stable image) you could have all your memory pre-programmed at distributors or specialised folks that will deliver pre-programmed parts for cents.

Accidently, if you're looking to offload the design shrink/sink-in/whatever, feel free to drop me line.