Custom Spartan-7 FPGA board for beginners

[asmi]

Hi,

I've recently designed a simple board with Spartan-7 FPGA. The board was specifically designed to be very simple so that it could be used by others as a starting point for their own design, and also to prove that you can actually design a decent board using free tools. It was designed from the ground up in KiCAD 5, and full sources are publicly available in my Github repository. The board was designed for 4-layer JLCPCB.com process as the cheapest 4-layer boards with controlled impedance I was able to find, specifically for the JLC2313 spackup. The board is 72x89 mm in size, and 10 board is only $29 (+delivery), though I'd recommend to select ENIG finish which would raise the price to $43.5.
Initially I was going to make just a bare minimum board without any peripherals, but soon realized that it won't be very useful. So I've added 256 Mbytes of DDR2 memory (128Mx16). The reason I've chosen DDR2 over DDR3 is that it allowed me to save one DC-DC converter because DDR2 uses the same 1.8 V rail that is used by the FPGA itself, while DDR3 uses 1.5 or 1.35 V. There is also a coupe of user buttons (aside from CFG_RESET used to reset FPGA configuration, and active-low SYS_RESET used for system reset without reconfiguration), and a pair of RGB LEDs.
In order to make sure I'm providing a sources for the board that actually works, I've assembled the board and fully tested it. I've attached photos of that board. I've also uploaded pre-made Gerber packages ready to be uploaded to the JLCPCB.com (current revision is D), as well as schematics PDF, and assembly document which I used for manual assembly. I've installed XC7S50-2FTGB196C FPGA with MT47H128M16RT-25E:C DDR2 memory chip, which is pretty much the maximum configuration. I confirmed that the board is able to output 1080p@60Hz via HDMI, but the FPGA heated up enough that I need to place a small heatsink over it. I also confirmed that DDR2 memory works at 333 MHz without any bit errors, which indicates that my manual length matching in KiCAD was good enough.

Like I said, the idea for this board to be as simple as possible yet be useful, as well as being the basis for other designs, so I didn't pack the board as tight as possible to make manual assembly easier. The real challenge for assembly is 18 ultra-small 0201 0.47 uF capacitors. I tried to use 0402 size instead, but they don't fit between vias under FPGA without violating manufacturing tolerances.
It is possible to customize the hardware on board - you can install smaller capacity DDR2 chip as long as the data bus width is still 16 bit (make sure it's supported by MIG!) - or don't install any at all, or any Spartan-7 chip in 1mm pitch FTGB196 package (I wouldn't recommend using devices smaller than S25 if you keep DD2 memory as memory controller will consume some resources), you can opt to not fit any HDMI-related parts if you don't want. As a matter of fact, if you are not confident about BGA soldering, you can assemble the first board with the smallest S6 device to see just how easy it is to work with BGA chips.

KiCAD source for the board can be found here: https://github.com/asmi84/kicad-projects
It uses a library which is available here: https://github.com/asmi84/kicad-library

If you have any questions or suggestions, feel free to post them here.

[SiliconWizard]

Nice work.

The layout job for the FPGA looks like it must have been no picnic.

[asmi]

Nice work.
The layout job for the FPGA looks like it must have been no picnic.

Thanks! Actually it wasn't all that hard, I think it took me like a couple of evenings to get the DDR2 layout down, once that was completed everything else was quite simple as this package was designed for 4 layer boards, plus very generous 3.5 mil (~0.09 mm) tolerances provided by JLCPCB process made escapes (which often are a pain for BGAs) pretty much a non-issue.
As for DDR2 layout, the biggest hurdle was differential traces (CK, DQS) as built-in trace matching tools just don't work quite right for diff pairs. I did length matching manually, only used length-matching tools to see what is current trace length so that I will know which way I needed to adjust it. Once you get a hand on it, it's actually not nearly as bad as I was afraid it will be when I was just coming into it. One thing I did not do though is account for via length when matching traces laid on a top layer with ones on the bottom layer, but for 333 (or even 400) MHz this is not a problem, which was confirmed by running a memory test for about an hour without encountering any bit errors.

[mark03]

Thanks for this.  I plan to design my own Spartan-7 board soon with the same FTGB196 package, so it's nice to have an example.

For the PCB houses I've used (OSHPark and DirtyPCBs), the limiting factor for BGAs is usually the overall via diameter (drill plus annular ring).  Sometimes this bumps up against the limit and folks "cheat" a bit to make it work.  What size did you use for the dogbone vias?

Do you print paste on the BGA pads or flux only?  If paste, leaded or lead free?  I heard cautions about mixing leaded paste and lead-free balls; not sure if there's anything to that.

[asmi]

Thanks for this.  I plan to design my own Spartan-7 board soon with the same FTGB196 package, so it's nice to have an example.

That's great! Let me know if you have any questions or need any help - I'd be more than happy to help out. Or if you use KiCAD yourself, you can just take my design, get rid of parts you don't need and add your own components. The primary reason I've published the full sources is that I would like to dispel the myth that FPGAs are some kind of black voodoo magic, and demonstrate that it's not very hard to get basic board designs up and running without too much effort and money. Now - don't get me wrong, some FPGA designs are insanely complex, but that doesn't mean all of them are.

For the PCB houses I've used (OSHPark and DirtyPCBs), the limiting factor for BGAs is usually the overall via diameter (drill plus annular ring).  Sometimes this bumps up against the limit and folks "cheat" a bit to make it work.  What size did you use for the dogbone vias?

This board was designed around JLCPCB's 4 layer process, so I used 0.2/0.45 mm vias (which is the minimum they can produce). The advantage of this fab is that it is the only one in it's price class that offers impedance control at no additional charge, which is important for high-speed interfaces like DDR2/3 or HDMI. This via size makes for an easy escape of 0.8 mm BGA (which is what DDR2 is), not to mention larger 1 mm pitch of the FPGA, with no cheating required. On top of that, they also allow traces down to 3.5 mil (~0.09 mm), which allows to fit two traces between 1 mm pitch vias and so you will need less layers to escape. Even though on this board I used 0.1 mm minimum traces, which was enough for this design. JLC2313 stackup they are offering means that 50 Ohm impedance traces are only 5.78 mils wide (I used 0.15 mm wide traces), which is great for "dense" buses like DDRX as there are a lot of traces.

Do you print paste on the BGA pads or flux only?  If paste, leaded or lead free?  I heard cautions about mixing leaded paste and lead-free balls; not sure if there's anything to that.

This board was assembled using room temperature stable lead free paste (this one), and I used stencils for both sides (since the board is quite small, I could fit stencils for both sides on a single sheet of metal) - I ordered it also at jlcpcb together with boards. That is just because I have the equipment (manual stencil printer, and reflow oven), but I tried to design the board such that it could be assembled with just a hot air gun. You will need some kind of magnification as 0201 caps are just too small (0.6x0.3 mm IIRC). I'm using a stereo microscope from AmScope (SE410), and it is just great because it has a large working distance so that you can comfortably use soldering iron under it, while still being quite affordable (I think I got it for around $200 for a full set of eyepieces - x5/x10/x15/x20).

Though in the past I used leaded solder paste with lead-free BGAs and they turned out just fine too. One of the boards i used it on is now about 3 years old, and it still works OK today. So if you prefer using leaded paste, it should be fine as well.

[Dmeads]

Seriously cool! Thanks for putting this together!

Was power to the FPGA a hassel? Were the DC-DC converters switching?

Overall really amazing design.

[SiliconWizard]

Nice work.
The layout job for the FPGA looks like it must have been no picnic.

Thanks! Actually it wasn't all that hard, I think it took me like a couple of evenings to get the DDR2 layout down, once that was completed everything else was quite simple as this package was designed for 4 layer boards,

Yeah, not that it's very hard, but all this BGA fan-out is very tedious and I just hate doing that (usually have it done by someone else/subcontract layouts like this )

Some EDA packages have some kind of automatic BGA fan-out, I don't know whether KiCad does yet?

[langwadt]

Nice work.
The layout job for the FPGA looks like it must have been no picnic.

Thanks! Actually it wasn't all that hard, I think it took me like a couple of evenings to get the DDR2 layout down, once that was completed everything else was quite simple as this package was designed for 4 layer boards,

Yeah, not that it's very hard, but all this BGA fan-out is very tedious and I just hate doing that (usually have it done by someone else/subcontract layouts like this )

Some EDA packages have some kind of automatic BGA fan-out, I don't know whether KiCad does yet?

I believe there are some python scripts around to do automatic fanout of BGAs

[SiliconWizard]

I believe there are some python scripts around to do automatic fanout of BGAs

I'll have a look!

[asmi]

Yeah, not that it's very hard, but all this BGA fan-out is very tedious and I just hate doing that (usually have it done by someone else/subcontract layouts like this )

Actually I like that part the most - it's like playing chess when you've got to think and plan ahead. Especially in cases where you know you can't do a full breakout, when you need to be very strategic.

Some EDA packages have some kind of automatic BGA fan-out, I don't know whether KiCad does yet?

They all suck, as in most cases you need not just "a fanout", but a very specific one because things like DDRX or LVDS require certain way of doing it. On this board I didn't pay too much attention because I know it will run at only 333 MHz (as PHY in that package is only rated at that speed), and I know from my past experience with DDR3 memory that at these low frequencies I can get away with this. But I also know that's not how one would go about fanout when he would be laying out DDR3-1866 working at 933 MHz (for example for Kintex-7 board), as small things start to matter at that frequency. That is the hard part.
If you don't have any hi-speed or interface requirements (which I kinda doubt), than doing just a simple fanout is so fast in just about any EDA I worked in (and in the past year alone I worked in KiCAD, Orcad Pro and Altium), that any specific tool is just not needed.

[SiliconWizard]

Yeah, not that it's very hard, but all this BGA fan-out is very tedious and I just hate doing that (usually have it done by someone else/subcontract layouts like this )

Actually I like that part the most - it's like playing chess when you've got to think and plan ahead. Especially in cases where you know you can't do a full breakout, when you need to be very strategic.

Ok, to each their own!

Do you take contract work?

[asmi]

Do you take contract work?

Yes I do, when I'm not busy with different contract

[OwO]

I've done DDR3-1066 on 4 layers too and got it working reliably. The tricks are:

• All DDR traces must form a microstrip or stripline against the closest ground plane (usually 0.1mm away) which must be solid and have no breaks under the traces
• All ground and power pins on both the SoC and DDR chip should be accounted for; either they need to go to a plane, or have a decoupling cap right under the balls that goes to a plane. Every ground ball should have its own via, but a cluster of e.g. 4 ground balls are allowed to be connected together and grounded using 3 vias spread apart.
• If using a power plane, it must be stitched to the ground plane using a capacitor every 1-2cm, with higher density near signal layer transitions. For example, if a few DDR signals go from top to inner layer, you need a stitching capacitor or two around the via transition.

The stackup at the DDR area was: top (DDR traces); in1 (ground plane); in2 (DDR traces); bottom (VCCDDR plane). The rest of the board has ground planes on both in1 and bottom, with in1 being solid and bottom being broken by components. Therefore high speed signals are either routed on the top layer, or in2 while the bottom ground plane is cleared of components where striplines are routed. It goes without saying that stitching is critical.

All 3 boards tested so far passes memtests running under Linux for a few days. Btw the memtest that comes with Xilinx embedded SDK example projects is not as demanding as just booting Linux. I've had one inadequate BGA reflow that still passes the Xilinx memtest but Linux will fail to boot every other time and running any user application typically locks it up. The issue was fixed by adding more flux and redoing the reflow.

The design is on GitHub under gabriel-tenma-white/sdr5.
In the future I may design a Kintex-7 board with faster DDR3.

[Mechatrommer]

If you have any questions or suggestions, feel free to post them here.

for beginner its not easy to assemble that BGA package FPGA and RAM. it will need advanced tools such as stencil or rework station. so trying to send 4 layer PCB to fab house (more expensive for small qty) is another matter, assembling them is another hard matter. for someone who only just learning to play/program with FPGA, i will be interested at reasonably priced ready made dev board + step by step on how to program it and what toolchains etc... do you think about selling it?

[asmi]

for beginner its not easy to assemble that BGA package FPGA and RAM. it will need advanced tools such as stencil or rework station. so trying to send 4 layer PCB to fab house (more expensive for small qty) is another matter, assembling them is another hard matter.

That is the exact myth I'm trying to dispel with this project. No, you don't need any advanced or expensive tools, the only tool you need is a hot air gun, which can be had for about $100 on Aliexpress, ebay or Amazon. That, and some tacky flux + some solder for the rest of components.
Seriously. The cheapest XC7S6-1FTGB196C FPGA is only $14.7, get it and buy the rest of component on a board except memory, and give it a go. You will be pleasantly surprised how easy BGAs are to work with, compared with other packages (there is a TSSOP package with exposed pad on this board which is IMHO harder to solder as it sits on a ground plane for thermal reasons).

do you think about selling it?

I will need to get a reasonable volume (at least one tray, which is 90 chips IIRC) before I can approach Xilinx sales rep and get a good price. As it is, the BOM for a single board - as I assembled it - is around $80 (including PCB, but excluding passives as I have literally reels of them), and so the retail board price is going to be $100+ once I factor in all costs. And for that price, this is a very specialized board with a rather limited functionality (which, of course, is by design as I wanted to keep it simple), basically, you can connect a bunch of sensors and/or perhaps some kind of wireless transceiver, and display information gathered to a some kind of HDMI display or TV. Or implement some kind of display adapter in FPGA and connect it to a some external controller board.

That said, since I have few boards laying around (they are of REV B, which have some issues, but nothing that can't be fixed by a sharp knife, as I haven't ordered REV C yet due to this whole virus thing going on), if someone is interested in getting a board (assembled or not), leave a post here, and we can work something out. I can either send a bare PCB, or assemble it with whatever parts you want if you cover the cost of parts and postage. FPGAs are export-controlled, so unless someone tells me how to deal with these issues, I can only send assembled boards to Canada and perhaps US (as I imagine there shouldn't be too much trouble for re-exporting FPGAs back in there).

[SiliconWizard]

for beginner its not easy to assemble that BGA package FPGA and RAM. it will need advanced tools such as stencil or rework station. so trying to send 4 layer PCB to fab house (more expensive for small qty) is another matter, assembling them is another hard matter.

That is the exact myth I'm trying to dispel with this project. No, you don't need any advanced or expensive tools, the only tool you need is a hot air gun, which can be had for about $100 on Aliexpress, ebay or Amazon. That, and some tacky flux + some solder for the rest of components.
(...)
You will be pleasantly surprised how easy BGAs are to work with, compared with other packages (there is a TSSOP package with exposed pad on this board which is IMHO harder to solder as it sits on a ground plane for thermal reasons).

Agreed. And you don't need stencils to solder BGAs if you're going to do it with a hot air gun, just flux?

Yes packages with thermal pads are much more annoying to hand-solder. TSSOPs are not the worst IMO. The absolute worst are the QFNs with thermal pads, as it's very easy to have bad connections to some outter pads due to the package not laying quite flat, and it's much harder to inspect (and fix). 

[OwO]

BGAs are easy to *solder*, you are right about that one. However they are difficult to troubleshoot, impossible to probe, and on any project with lots of BGA components I have to budget around twice total development time and cost.

First iteration of sdr5 had an issue with a missing signal, and as I can not physically bodge a connection I can not verify that this was indeed the issue. That means extra board spins which may or may not fix the problem. I also can not verify with good confidence whether the BGA soldering was fault free, so I have to build multiple copies of every prototype iteration to rule out soldering issues.

Lead free balls makes the whole process far more painful because there is a narrow window of temperature between unreliable reflow and potential component damage. Sometimes I will specifically buy reballed chips because they usually come with leaded balls and are easier to deal with.

Given this I would absolutely recommend to cut BGA use to a minimum and only when it's justified (some parts are only available in BGA, size constraints, etc).

[asmi]

Agreed. And you don't need stencils to solder BGAs if you're going to do it with a hot air gun, just flux?

Yep. I only apply solder paste if I do a full board reflow.

Yes packages with thermal pads are much more annoying to hand-solder. TSSOPs are not the worst IMO. The absolute worst are the QFNs with thermal pads, as it's very easy to have bad connections to some outter pads due to the package not laying quite flat, and it's much harder to inspect (and fix). 

On this board I use TPS65581 triple DC-DC buck converter IC, which is TSSOP-20EP with exposed thermal pad, which sits on a solid chunk of ground plane with a bunch of thermal vias to ensure good thermal conductivity, so soldering it using lead-free solder would be somewhat tricky with hot air gun, but since these are prototypes, you can use leaded solder and that will make things much easier. These ground planes are obviously not an issue for an oven since entire board is heated up to soldering temperature. This chip is great for many reasons - very easy to use due to no need for external compensation, reasonably good efficiency (it's mostly 70+% for main 1.0 V rail and 80+% for other rails), but my experience with it shows it can heat up pretty severely under full load on all rails - hence the need for robust thermal path.

[asmi]

First iteration of sdr5 had an issue with a missing signal, and as I can not physically bodge a connection I can not verify that this was indeed the issue. That means extra board spins which may or may not fix the problem. I also can not verify with good confidence whether the BGA soldering was fault free, so I have to build multiple copies of every prototype iteration to rule out soldering issues.

It's easy - stop buying garbage chips that came from god knows where. Only buy from the vendor or reputable resellers. I never had any soldering failures with BGA that were sourced from reputable places, but few parts which I bought from aliexpress et al. all had to be baked before soldering as I had zero confidence in their solder balls, and was half-expecting them to not work at all - but I got lucky so far with no failures after 2 hrs baking at 125°C just before reflow.

Given this I would absolutely recommend to cut BGA use to a minimum and only when it's justified (some parts are only available in BGA, size constraints, etc).

I do the exact opposite - for my designs I use BGAs whenever I can as their advantages far outweigh their limitations. I will take BGA over QFN any day, for example.

[OwO]

No, all the soldering "failures" I've had were because of too low temperature/inadequate reflow and were fixed by a second reflow. This has nothing to do with the chips. You just can NOT know if your BGA soldering was actually sufficient and when you do encounter bugs you have to do many copies to rule out soldering issues. This also doesn't address the lack of debugging access to pins, which makes it impossible to test fixes and requires a board respin for every change. It's impractical to have test points on every possible connection, which wouldn't be necessary on a QFP because you can attach bodge wires to the pins.

[asmi]

No, all the soldering "failures" I've had were because of too low temperature/inadequate reflow and were fixed by a second reflow. This has nothing to do with the chips.

Of course it does. Garbage chips tend to come with garbage solder balls, so you either bake them, then cross your fingers and hope for the best, or you reballs them to known-good balls.

You just can NOT know if your BGA soldering was actually sufficient and when you do encounter bugs you have to do many copies to rule out soldering issues.

If course you can. You just need to be a tad smarter about this. Like test the board before soldering, including measuring trace impedance if that is going to be a problem (like for 10G links). If you know the board is good, and so are chips (because you bought them from reputable source, and not from some shady guy who will be nowhere to be found a day after the sale) and logic (say, you use MIG which pretty much guarantees the logic is correct), so by exclusion you can determine that the reason for the bug is indeed soldering. In a lot of cases there are easier ways to rule out soldering issues like using JTAG boundary scan.

This also doesn't address the lack of debugging access to pins, which makes it impossible to test fixes and requires a board respin for every change. It's impractical to have test points on every possible connection, which wouldn't be necessary on a QFP because you can attach bodge wires to the pins.

This is a moot point as I don't have equipment to "debug" anything in high-speed interfaces anyways, and for issues which are slow enough that equipment I have is sufficient, I can always scrape off a solder mark from via or track and get access that way.
But if I would have the right tools, there are ways to have test points on high-speed traces and not degrade SI too much.
Oh, and last point - care to show me QFP chip with DDR3 interface? Once you NOT find one, you might want to read up exactly why, and see if that has anything to do with BGAs.

[OwO]

for beginner its not easy to assemble that BGA package FPGA and RAM. it will need advanced tools such as stencil or rework station. so trying to send 4 layer PCB to fab house (more expensive for small qty) is another matter, assembling them is another hard matter. for someone who only just learning to play/program with FPGA, i will be interested at reasonably priced ready made dev board + step by step on how to program it and what toolchains etc... do you think about selling it?

There is a XC7A100T dev board with DDR3 from QMtech for about $80:
https://www.aliexpress.com/item/4000170003461.html

It's very hard to beat that if you buy parts at list price. There are also Zynq boards for even cheaper. There is a lot of documentation available if you look it up.

[OwO]

Of course it does. Garbage chips tend to come with garbage solder balls, so you either bake them, then cross your fingers and hope for the best, or you reballs them to known-good balls.

What does baking have to do with solder balls? The chips I got had date codes only a few months prior, and this is verifiable with device DNA over JTAG. They are not reballed chips, and actually I've had better luck with reballed chips because they have leaded balls. I can show you ball uniformity by pressing to a piece of glass with a bit of liquid but you still won't be convinced.

The signal that I needed bodged wasn't a high speed signal. I didn't say use a QFP FPGA. I said use BGA only when justified, such as the FPGA and DDR3 chips. However I frown upon gratuitous use of BGA such as power management ICs etc which isn't aimed at your design but rather in general. I still assert my points about BGA being much more difficult to troubleshoot so I think it's reserved for the more experienced designer. When troubleshooting I'd rather rule out the soldering first, rather than having to rule out everything else (which is usually much harder to do).

[asmi]

What does baking have to do with solder balls? The chips I got had date codes only a few months prior, and this is verifiable with device DNA over JTAG. They are not reballed chips, and actually I've had better luck with reballed chips because they have leaded balls.

What does crappy solder balls have to do with soldering defects? Is that a serious question?
Moisture and solder balls oxidation are two sure ways to get solder failures with BGAs. Baking alleviates the first one, but does nothing for the second one. Good flux (either stand-alone, or as part of solder paste) will help with removing oxidation, but only to the point.
Read up on MSL to get more information on this if you're interested. And if you continue using crap chips from shady sources, at least bake chips before reflow to reduce your chances of failures - this gets worse as ball size decreases, so you can get away with a lot on 1 mm pitch chips, but as you go down in pitch, your risk of getting a failure increases.

I can show you ball uniformity by pressing to a piece of glass with a bit of liquid but you still won't be convinced.

You don't need to show anything to me - as I said above, I've got some BGA chips off aliexpress so I know what I'm talking about. Just look at those balls under x20 microscope, and compare them to brand new chip that just came off a sealed package, and you will clearly see the difference.

The signal that I needed bodged wasn't a high speed signal. I didn't say use a QFP FPGA. I said use BGA only when justified, such as the FPGA and DDR3 chips. However I frown upon gratuitous use of BGA such as power management ICs etc which isn't aimed at your design but rather in general. I still assert my points about BGA being much more difficult to troubleshoot so I think it's reserved for the more experienced designer. When troubleshooting I'd rather rule out the soldering first, rather than having to rule out everything else (which is usually much harder to do).

Well using BGAs for PMIC is a stupid idea in my opinion, or in any package without thermal pad for that matter, unless it's a really low power one when thermals are not a factor. But for example for for QSPI flash chips I prefer 1 mm pitch BGA-24 package, not the least because there is a greater range of pin-compatible capacities available. I can't use 0.5 mm and below pitch packages because PCB tech required is way out of my price range, but 0.65 mm and up is a fair game.

[Mechatrommer]

and so the retail board price is going to be $100+ once I factor in all costs.
..if someone is interested in getting a board (assembled or not), leave a post here, and we can work something out. I can either send a bare PCB, or assemble it with whatever parts you want if you cover the cost of parts and postage.

FPGAs are export-controlled, so unless someone tells me how to deal with these issues, I can only send assembled boards to Canada and perhaps US (as I imagine there shouldn't be too much trouble for re-exporting FPGAs back in there).

bummer. this line destroyed what i'm about to reply on the above quote... long story short... for $100+ with step by step programming guide, that should be reasonable. i have $5 china altera fpga breakout laying for years and altera blaster bought separately, manually downloaded the fpga SW in many GB few years later (i dont even remember the name ) but dont have time to figure out how to start (namely the SW/VHDL/Verilog part), maybe due to lack of need, the toolchain maybe obsolete or invalidated by now. so they are unsolved puzzles. i need a puzzle that shows me how to solve it in quick time, like... this is the HW, this is the SW/IDE (download from <here>), you'll need this programmer, and this is the sample program, you compile it its surefire to do led blink. that kind of thing. i dont need another unsolved puzzle. good luck to you anyway. cheers.