Lowest cost way to capture 110 channels of ADC data

[ali_asadzadeh]

Hi,
One of my customers neededd me to design an FPGA board for him to be able to capture 110 ADC channels simultaneously, the ADC is LTC2378-16 and send it over to a PC trough a 1GB Ethernet port in realtime. Also the ADC clokc should have a low jitter clock in the order of 10ps, I have seen fs jitter clock generators, But they are for high frequency sampling like 250MSPS parts, do we have something for low sampling parts? and also the synchronization time between each channel should not be more than 1us.
The sample rate is around 600KSPS for each ADC, so I have calculated that I needed an FPGA with more than 1200 IO lines, so they are very pricey! Do you have any idea so it can use lower cost Spartan6 or ARTIX series in the 484 pin package, since I have very good prices for them In china, the problem would be to sync the Separate FPGA's and pack all the data in one of them and send it to the PC,

Any Hints or Ideas are highly appreciated!

[Marco]

That's the full throughput of 1 Gbps ethernet ... no room for overhead even. That's a bit optimistic.

In theory you only need 1 FPGA input pin per ADC, if you have room for some extra logic ICs to fanout SCK and CNV then those only need 1 pin each on the FPGA.

[Miyuki]

I agree, 1 pin per ADC and have some clock "amplifier" to feed all of them
Also if signals are not just noise, data can be reasonable compressed to conserve bandwidth

Most FPGAs have nice clock generating capability from any stable source
It look like a relative simple task even for small one

[nctnico]

You can get off-the-shelve data acquisition systems for this purpose.

[MagicSmoker]

You can get off-the-shelve data acquisition systems for this purpose.

You can, but the price will make a grown man cry... then again, so will designing this sort of thing yourself.

[Kalvin]

Use multilpe cheaper FPGAs for sampling the ADCs, and one more expensive as a concentrator and Gigabit Ethernet controller? Probably you could use only simple UDP data with large frames for the data transmission, which would make the FPGA implementation somewhat simpler compared to using the TCP protocol.

[SiliconWizard]

Yes. You absolutely DON'T need that many IOs. Just share MOSI and SCK among all ADCs. Only route MISO of each ADC to a different input. Manageable with even a modest FPGA. You shouldn't even need any buffering for the output IOs, just select higher output current (many FPGAs allow up to 12mA or over, that should do it IMO), but a few buffers would not cost you much either.

As to data throughput, that's just not realistic. That's 125 MiB/s (132MB/s). You can't get that with 1Gbit ethernet.
So yeah - either compress the data drastically - you'll have to find a good compression algorithm that can do that on the fly with this kind of throughput, not trivial - or find another solution. You could consider suggesting USB 3.0 SS. You should be able to get this throughput with a FTDI FT601 for instance. Of course if ethernet is a hard requirement from the customer, just try the compression approach. If you can't manage to pull this off, try considering 10Gbit ethernet. This would be pretty tough to implement though IMO.

[DaJMasta]

You could also just team the ethernet with a second port for the additional bandwidth, but it is a large requirement.

Have you looked into multiplexed ADCs?  If your sample rate is only 600ks/s you should be able to find some options that can give you 2-4 channels in a single package, though you may run into issues with timing offset between the multiplexed channels depending on your requirements.  Could save a bit on bom/board space and reduces your interface pin count.  That said, since your sample rate is fairly low, you should be able to daisy chain multiple ADCs (the LTC2378-16 supports it), so you can get by with fewer interface pins with a higher speed that way too.

It does sound like clock distribution will be a concern, but there are low jitter fanout buffers for LVCMOS and similar that can manage in the fs range of RMS jitter, so I can't imagine it would be impossible to find - you don't really have to worry about skew unless your trace length mismatch is real long because of your 1us synchronization requirement.  I think it's more the case that lesser fanout buffers don't advertise jitter specifications, rather than them actually having that bad performance.  The edge rate on the output if you go with a high frequency part could be a concern for EMI, but since it would be much higher frequency content it should be easy to filter out, at least.

[dmendesf]

Shift clock of this ADC can be up to 100MHz so you can daisy-chain between 2 to 4 units. PLL of FPGAs have a lot of jitter so use an external one. Divide ADCs between suitable numbers of small FPGAs, use FIFOs to sync data to a concentrator FPGA. You need more than one channel of gigabit Ethernet (or 2.5gigabit... look for it). Look for audio PLLs.

[NiHaoMike]

Just split it up to 2 or more units and use multiport Gigabit cards - they're really cheap.

[ali_asadzadeh]

Guys thanks for the TIPS, I think the Idea that use 1 Pin per ADC sound very good, the main problem would be the Clock and MOSI buffers, and also low jitter clock generation, do you suggest any parts for them? also I think I can convince them to use 10G Ethernet, I have done 1G Ethernet before, But not a 10G one.

Any ideas on that would be nice too? is it very hard compared to 1G Ethernet? also do you suggest an ARTIX or a ZYNQ part? the price is my concern, since my profit would come for using lower cost parts

[mariush]

Some super fast compression should be possible. 
Even something as simple as RLE on high byte and lo-bytes of samples read.

Or maybe you could use a FPGA with pci-e capability and simply dump the data onto a pci-e bus?
Then you could use a small computer to buffer the data in RAM or a SSD and compress it in real time and push it on a 1gbps ethernet port.

Instead of paying 50-100$ for a 10g network card, you could get a motherboard with embedded cpu like this one for example : https://www.newegg.com/asrock-j3355m-micro-atx/p/N82E16813157730
Lots of options under 75$. Add a stick or two of memory and a standard atx power supply (you can get boards with 12v DC in for <100$) and a small boot drive (a usb stick would work, an emmc, a 16-32gb ssd etc)

You could sell it as giving them them ability to add a SATA SSD to dump the data to it instead of streaming it to some other place, making everything more convenient, portable...

[Brutte]

One of my customers neededd me to design an FPGA board for him to be able to capture 110 ADC channels simultaneously, the ADC is LTC2378-16

So your customer specified 110 x LTC2378-16 ADCs and you are asking for the lowest cost solution? You have very interesting customers 
The 110 ADCs cost €2000 - what is the difference if the FPGA costs 1€ or €100?

[MagicSmoker]

So your customer specified 110 x LTC2378-16 ADCs and you are asking for the lowest cost solution? You have very interesting customers 
The 110 ADCs cost €2000 - what is the difference if the FPGA costs 1€ or €100?

The OP has a habit of asking for the cheapest solution to lots of complicated problems and without having done much/any of the initial legwork himself.

[Brutte]

I'd also strongly advice buying one of those ADC chips and check if this is according to the parameters specified in a datasheet, before you spend €2000 for the batch of 110.

I have learned Linear Technology marketing tricks the hard way. It is Analog Devices now.

[teksturi]

Guys thanks for the TIPS, I think the Idea that use 1 Pin per ADC sound very good, the main problem would be the Clock and MOSI buffers, and also low jitter clock generation, do you suggest any parts for them? also I think I can convince them to use 10G Ethernet, I have done 1G Ethernet before, But not a 10G one.

Just split 110 adc board to 4. Then you can design with 1G ethernet. but 30 for each. Then if some are not working you can just mark them broken. Also board cost would be lower.

the price is my concern, since my profit would come for using lower cost parts

If you spend too much time to select low cost parts you will lose lot more money. The main factors should be: easy to design, understandable manuals, easy to buy. You will save your time and that should be valuable. I do not think these board's will be manufacture's large count.

[SiliconWizard]

Yeah if the price is your "concern" (will you be manufacturing the devices for the customer?), the fact the cost of the FPGA will be negligible compared to your 110 ADCs still holds.

And if you're actually manufacturing them instead of just selling your design, maybe that contractually means that you'll be allowed to reuse your design for other customers? In that case, that would also be a good point justifiying designing a board with less inputs, and making them able to be coupled/networked for more inputs. A lot more flexible, a lot less constraints. (The only constraint, if you need all inputs to be sampled synchronously, will be to provide a sync signal that coupled boards can share. Thus a single board would have a master sync output and a slave sync input...)

And yeah, you could also look for ADCs with more inputs. Can't you find a similar spec'ed ADC with 4 inputs for instance? That will limit the number of parts, and the overall cost.

[Scrts]

Just split 110 adc board to 4. Then you can design with 1G ethernet. but 30 for each. Then if some are not working you can just mark them broken. Also board cost would be lower.

This.

I was doing the math... 16bit * 600000 samples/sec * 110 channels = 1.056Gbps?

The data alone would saturate Gbps link, without even talking IP/UDP headers and switching processing time. So even if there will be 4 acquisition boards, the switch has to be capable of >1Gbps link to the PC. Most likely 10Gbps switch?

By the way, the PC load getting such fast data input will be enormous. Is it going to process that much of data? CPU will be chocked as well.

[RCinFLA]

Sound like you are specifying what chips can do, not customer requirements.

Given 110 channels.

1) Do the 110 channels have to sample synchronously?  If so, what is time tolerance, jitter, on simultaneous samples?

2) Dynamic range, number of useable bits (ENOB)

3) Sample rate.

Depending on requirements you might want to look into ultrasound Rx chips that provide multiple channels of relatively high synchronous sample rate with high speed LVDS output interface mux'g multiple channels.

At other extreme, requiring low sample rating, non-synchronous,  look for chips with required resolution and greatest number of muxed input channels. If you need synchronous samples you need a common clock and simultaneous control of start of conversion. 

To reduce FPGA IO requirements I would look for a parallel output version ADC with similar 600 Ksps sample rate and use FPGA to read parallel outputs at a multiple subsample rate using each ADC parallel output buffer as a holding buffer.  With FPGA running a parallel bus input of about 150 MHz you could circularly read each of 110 ADC's within the 600 ksps sample period.  You may be limited by the ADC output enable timing and have to have more parallel FPGA buses for smaller group of ADC's to reduce parallel read rate.

[NiHaoMike]

I was doing the math... 16bit * 600000 samples/sec * 110 channels = 1.056Gbps?

The data alone would saturate Gbps link, without even talking IP/UDP headers and switching processing time. So even if there will be 4 acquisition boards, the switch has to be capable of >1Gbps link to the PC. Most likely 10Gbps switch?

By the way, the PC load getting such fast data input will be enormous. Is it going to process that much of data? CPU will be chocked as well.

Don't need a switch, just get a NIC with 4 Gigabit ports, I managed to get one for under $15.

As for handling the data, any half decent SSD will save it with ease. Of which, maybe it would make more sense to make a standalone unit that just accepts a SSD for storage? (How difficult is it to implement SATA host on a FPGA?)

[coromonadalix]

Why does it need to be sampled at this high speed in real time ?? 

I have nidaq cards at my job and i sample at least 9 times a sec on each channels and its enough to see some glitchs and problems

I measure voltages and rf sources for ISO logging  / reports,  we needed an powerful pc to run this because of the bottleneck issues ...


You dont tell what are you needed to check / verify at theses  sampling speed

Lots of speculation ...


In the past i did a project with NI acquisitions cards in a pxi frame,  108 chanels to be precise, for milli seconds to almost nano seconds measurements, dumped thru network ... and yes its a ton of data passing thru, and you may need some buffering to catch what you want.

Have you tought of the wiring issues too ?  I know you want costs down,   but you seem to take a strange road      my 2 cents.

[Rerouter]

You also can't forget about the costs for any input protection or buffering, Even laying out that monster would be interesting, 110 Analog inputs, probably with each given its own ground return, not to mention, if everything is commonly clocked to be in sequence, you have AMPS of switching noise at the sample rate, so even power routing will be interesting

All i can say is, I hope that it succeeds, And I fear for your sanity during layout.

[ali_asadzadeh]

Thanks guys, for your hints, I know the cost of ADC's are higher than the FPGA alone,But we are engineers and we need to squeeze the jucie out of these chips Becasue after all it show how we are capable! OK, fair enough, I think a Zynq part would be much nicer, also maybe isolating each channel would be nice too, since it at least isolate each channel and the switching currents would not affect the ADC performance, maybe by using DC/DC parts like MAUxx series from meanwell and Digital isolators from Analog like ADUM3401

[Brutte]

I know the cost of ADC's are higher than the FPGA alone,But we are engineers and we need to squeeze the jucie out of these chips

Is your goal delivering a product or is it squeezing the juice out of the last flip-flop of FPGA? Maybe it is time you should inform your customer that you are burning his money?