Looking for high speed USB memory stick or SDHC reading solution

[mikeselectricstuff]

All I want to do is be able to read data from a file and throw it out of either multiple UARTS, fast SPIs or 8 bit parallel, into an FPGA at rates of 1-4Mbytes/sec continuous (i.e. need enough extra speed & RAM buffering to cover any filesystem latency).

I've used the PIC24F parts with USB host and SD, which work fine as far as they go but the fundamental speed of the interface is a limiting factor. (as well as, at least in the case of SD, inefficient code, but this is fairly easy to tweak for a 2-3x efficiency gain)

Does any one know of any MCU that satisfies the following :

USB 2.0 high speed or SDHC port, and working free example code to read from a memory device, which can be compiled with a free or low-cost toolchain. (Ideally IAR EWB kickstart edition)

Not BGA, and readily available in small qty (10s). Onboard Phy would be nice.

Not under Linux - just power up & go. It will not be doing anything else - just reading a file and streaming it out so don't want the baggage and learning curve of an OS

NXP LPC1800 looks a possibility with their nxpUSBlib library but not clear if their USB library supports hi-speed.  This post indicates that there are some major performance issues  at the moment
http://www.lpcware.com/content/forum/mass-storage-readwrite-performance-lpc18xx
At least non-BGA packages appear to be becoming more available now.

Bare metal code on a RasPi may be an option if I can get the data out somehow - maybe parallel via IO pins - not looked in detail yet - does anyone know if there is a file system avaialabe yet for  bare-metal users?

This looked promising but speed is limited by the slow interface to the host, and the 4MBytes/sec speeds claimed turn out to only be useful for card-card copying.
http://www.ghielectronics.com/catalog/product/340
I think this is based on a ST ARM Cortex M3 chip.
If they had a way to, for example read USB and spit it out of the 4-bit SPI interface, under control of the UART port that could be ideal.

Something that can be got up & running easily with minimal learning curve is highest priority - parts cost not a major issue.

Just wondering if anyone else has looked into this before I start digging any further..

[andersm]

Have you looked at Freescale's Kinetis series? At least some of the bigger parts claim to support all of SDHC, USB HS and external bus interfaces. Never used them myself, so can't comment on any of that.

[nctnico]

Maybe the the person is using the LPC1800 wrong. AFAIK the speed of the USB connection is set by a resistor to the data lines. If he got that wrong the USB connection is not hi-speed. And there are two ways of sending packets over a USB connection. One method is for messaging, the other for bulk data.

For your purpose you could probably use an USB to SPI chip like thise one http://www.ftdichip.com/Products/ICs/FT232H.htm to convert the data stream on the USB bus to an SPI interface.

[firewalker]

For your purpose you could probably use an USB to SPI chip like thise one http://www.ftdichip.com/Products/ICs/FT232H.htm to convert the data stream on the USB bus to an SPI interface.

Is this chip able to read file system on a usb device connected?

I believe Maxim has some ready to go SPI to USB mass storage chips. Don't know how fast it is.

Alexander.

[mikeselectricstuff]

Maybe the the person is using the LPC1800 wrong. AFAIK the speed of the USB connection is set by a resistor to the data lines. If he got that wrong the USB connection is not hi-speed. And there are two ways of sending packets over a USB connection. One method is for messaging, the other for bulk data.

For your purpose you could probably use an USB to SPI chip like thise one http://www.ftdichip.com/Products/ICs/FT232H.htm to convert the data stream on the USB bus to an SPI interface.

No - I need USB Host, not device.

[mikeselectricstuff]

The issue is not so much speed (once the basic interface is running High Speed) but something with a software solution that works & that I can tweak for my needs without having a huge learning curve for USB etc.

[firewalker]

Something like the device bellow, or I got all wrong?

http://tinyurl.com/aypqpaa

Alexander.

[enz]

How about the Vinculum chip from FTDI?
http://www.ftdichip.com/Products/ICs/VNC2.htm

It's a programmable USB-Host and available as DIP-Modules:
http://www.ftdichip.com/Products/Modules/DevelopmentModules.htm#V2DIP1

I have no experience with this product, but maybe it can do what you are looking for?

Martin

[free_electron]

I would go with the vinculum as well. It can even handle the filesystem. Look at vinculum 2 in special.

[andersm]

Neither Vinculum support high-speed though. The FT313H does, but looks like you have to run the USB stack externally.

[mikeselectricstuff]

I would go with the vinculum as well. It can even handle the filesystem. Look at vinculum 2 in special.

The VNC2 would be exactly what I need IF it were high speed, but it isn't.

[mikeselectricstuff]

Something like the device bellow, or I got all wrong?

http://tinyurl.com/aypqpaa

Alexander.

Nope. Not high speed and needs a SW stack on a seperate MCU.

[djsb]

What about the Chipkit Max32?

http://ww1.microchip.com/downloads/en/DeviceDoc/chipKIT%20Max32_rm.pdf

Just a stab in the dark.

David.

[andersm]

What about the Chipkit Max32?

Doesn't support high-speed USB or SD cards.

[Marco]

Have you looked at Freescale's Kinetis series? At least some of the bigger parts claim to support all of SDHC, USB HS and external bus interfaces. Never used them myself, so can't comment on any of that.

Unless you want to port the stack to GCC you will need Codewarrior for this BTW.

[marshallh]

Take a look at
http://www.digikey.com/product-detail/en/LPC4320FBD144,551/568-9449-ND/2840461



I solved this problem by having an fpga with custom SD controller do block DMA transfers to sdram. An external CPU woudl read the filesystem and find the longest span of contigous file sectors, then load the sector range in the FPGA where hardware dma was performed.
10mb/sec capped.

[mikeselectricstuff]

What about the Chipkit Max32?

http://ww1.microchip.com/downloads/en/DeviceDoc/chipKIT%20Max32_rm.pdf

Just a stab in the dark.

David.

Not high speed. "USB 2.0 Full Speed" just means plain old USB.

[mikeselectricstuff]

Take a look at
http://www.digikey.com/product-detail/en/LPC4320FBD144,551/568-9449-ND/2840461



I solved this problem by having an fpga with custom SD controller do block DMA transfers to sdram. An external CPU woudl read the filesystem and find the longest span of contigous file sectors, then load the sector range in the FPGA where hardware dma was performed.
10mb/sec capped.

Was this SD in SPI mode or full 4-bit? Documentation on the latter appears scarce as you need to join the SD card club to get it.
This may be a viable approach, even just  doing it a sector at a time.
But it's frustrating that there are several MCUs out there that should be able to do it all, it just needs the software stack to be there. 

[firewalker]

The SPI bus on an SD card what speed can offer?

Alexander.

[marshallh]

You can get ~2-3 mbyte/sec through SPI. But, there is sufficient info in the SD simplified spec to implement 4bit mode. I sat down with a printout of the spec and had it working in 3 days, only sticking point was the separate rolling crc16 on each data line, but here you go

Code: [Select]

crc16_3 <= {crc16_3[14:0], 1'b0} ^ ((pin_dat_out[3] ^ crc16_3[15]) ? 16'h1021 : 16'h0);
crc16_2 <= {crc16_2[14:0], 1'b0} ^ ((pin_dat_out[2] ^ crc16_2[15]) ? 16'h1021 : 16'h0);
crc16_1 <= {crc16_1[14:0], 1'b0} ^ ((pin_dat_out[1] ^ crc16_1[15]) ? 16'h1021 : 16'h0);
crc16_0 <= {crc16_0[14:0], 1'b0} ^ ((pin_dat_out[0] ^ crc16_0[15]) ? 16'h1021 : 16'h0);


[mikeselectricstuff]

The SPI bus on an SD card what speed can offer?

Alexander.

Not sure what the max speed is in single SPI mode , which is essentially MMC card emulation. The MMC spec says 20MHz - it could be that the more recent SD cards may inherit a bit more speed from the SD hardware.
The above post on FPGA gives me an idea of how I could get a lot more speed using my existing SD card code in simple SPI mode. I know when the filesystem library is reading data from my file (because I've already hacked the code a lot it to speed it up!) ,  so I could externally mux the SPI bus with a CPLD or FPGA so when I know the file data read is happening, the external logic clocks the card as fast as possible and catches the data, the PIC host being oblivious to the data transfer.

[firewalker]

The non GNU/Linux requirement is because you just don't want to fiddle with it?

Alexander.

[nctnico]

The SPI bus on an SD card what speed can offer?

Alexander.

Not sure what the max speed is in single SPI mode , which is essentially MMC card emulation. The MMC spec says 20MHz - it could be that the more recent SD cards may inherit a bit more speed from the SD hardware.
The above post on FPGA gives me an idea of how I could get a lot more speed using my existing SD card code in simple SPI mode. I know when the filesystem library is reading data from my file (because I've already hacked the code a lot it to speed it up!) ,  so I could externally mux the SPI bus with a CPLD or FPGA so when I know the file data read is happening, the external logic clocks the card as fast as possible and catches the data, the PIC host being oblivious to the data transfer.

Just to get things clear: you want to use a USB stick OR an SD-ish card to store a file and dump it into an FPGA?
If yes: Some microcontrollers in the LPC2000 series from NXP have a dedicated SDHC interface. It can do 4 bit transfers. With this FAT filesystem http://elm-chan.org/fsw/ff/00index_e.html you can have the SDHC part up & running real quick (it has drivers for the LPC2300). To transfer the data into the FPGA you can use SPI and DMA. IMHO SDHC is a much easier route than USB host.

[mikeselectricstuff]

The non GNU/Linux requirement is because you just don't want to fiddle with it?

Alexander.

A few reasons - Mainly I don't want to learn Linux for what should be a simple job, I want it to start quickly on powerup, I need a guaranteed data flow rate, so can't have any unpredictable latency issues. I also don't want to have to burn unnecessary power/cycles on things that aren't needed.

[firewalker]

The non GNU/Linux requirement is because you just don't want to fiddle with it?

Alexander.

A few reasons - Mainly I don't want to learn Linux for what should be a simple job, I want it to start quickly on powerup, I need a guaranteed data flow rate, so can't have any unpredictable latency issues. I also don't want to have to burn unnecessary power/cycles on things that aren't needed.

Ohhh, that's easy! Just download the source code of the real time Linux kernel (Wind River Systems) play with it for couple of years to customize it and you are done!

 

Alexander.

[free_electron]

I would go with the vinculum as well. It can even handle the filesystem. Look at vinculum 2 in special.

The VNC2 would be exactly what I need IF it were high speed, but it isn't.

They also have a new host that is high speed.

hang on ...

FT120. take a look at that. and it has a parallel data/address bus. so you should be able to plunk that on an fpga and crank away.

[mikeselectricstuff]

Just to get things clear: you want to use a USB stick OR an SD-ish card to store a file and dump it into an FPGA?

Yes - the card will be written on a PC, I just need to read it and throw out the data in the right format at a fixed rate. What I ultimately need to do is spit it out of multiple UARTS as parallel streams (Yes this is for driving lots of LEDs!)
My current solution is a PIC24FJ256GA006, outputting on its own 4 UARTs and also feeding a fast SPI stream  to a second PIC24 which then outputs the other 4 streams on its UARTs. This does the job fine for the last project (4600 LEDs) but the next one will be over twice the size, probably needing about 20 UART streams, so a <insert fast filesystem here> feeding an FPGA to do the UARTs using internal RAM to do the de-interleaving is  probably going to be a better solution.

[firewalker]

What if you split the file and use 2xcurent set up. Will it be difficult to sync or something?

Any infos with the led cube?

Alexander.

[mikeselectricstuff]

Any infos with the led cube?

Alexander.

Awaiting documentation video to be done - if you're in London you can go see it at http://www.thecrystal.org/_html/visit-the-crystal.html

[free_electron]

what if you do the heavy lifting on the PC?

Write the file in such a format that 1st byte is 1 stream ,second byte is second stream , 3 byte is third stream and so on.
that way you only need to open 1 file and do a cyclic step through it. hit the end and begin again.  that eliminates tons of overhead.

4600 led's each requireing what ? 1 byte for intensity control. let's say you do 50 refreshes a second.
thats 4600x50 bytes per second or 1.84 megabit per second you need to pump. the vinculum2 can do 12 megabit per second.

use a simple FPGA to do the byte demultiplexing and throw it in custom shift registers that spit out the streams. you can create all the serial ports you want in the fpga. since you do not use handshaking nor need to read back you can simply build an 8 bit shifter under control of a master clock generator. when a parallel load is done into the shift register it begins spitting out data at the tick of the master clokc. if you really need rs232 its a matter of making a 11 bit shift register holding the 2 start bit , 8 databits and the stopbit.

minimalistic code ont he vinculum opens a fixed filename and begins streaming it through the parallel fifo bus ( no need to resreialize . the vinculum spits out bytes in parallel on a clock base. you can have the fpga deliver this clock to the vinculum.
basically the vinculum opens the file and waites for a 'command; form the fpga'. the fpga says 'next byte' and the vinculum puts it on its outgoing databus. when last byte is reached you handshake through another signal adn the whole thing begins again.

[marshallh]

If you require precise timing, complete dependence on data being available, you will want to buffer the data in some sdram. How much data are we talking about?
32mb or 64mb sdram is available on the Terasic DE0 devboard (cheap) you can buffer locally in some blockram FIFOs to get around the sdram refreshes.

[mikeselectricstuff]

If you require precise timing, complete dependence on data being available, you will want to buffer the data in some sdram. How much data are we talking about?
32mb or 64mb sdram is available on the Terasic DE0 devboard (cheap) you can buffer locally in some blockram FIFOs to get around the sdram refreshes.

Probably only a few tens of K - I need a consistent framerate with frames of the order of 10Kbytes, at maybe up to 50fps. I typically need to put a whole frame in RAM as it will be sliced out over multiple UARTs, and double-buffer it. Internal RAM on an FPGA should be plenty

[mikeselectricstuff]

what if you do the heavy lifting on the PC?

Write the file in such a format that 1st byte is 1 stream ,second byte is second stream , 3 byte is third stream and so on.
that way you only need to open 1 file and do a cyclic step through it. hit the end and begin again.  that eliminates tons of overhead.

4600 led's each requireing what ? 1 byte for intensity control. let's say you do 50 refreshes a second.
thats 4600x50 bytes per second or 1.84 megabit per second you need to pump. the vinculum2 can do 12 megabit per second.

use a simple FPGA to do the byte demultiplexing and throw it in custom shift registers that spit out the streams. you can create all the serial ports you want in the fpga. since you do not use handshaking nor need to read back you can simply build an 8 bit shifter under control of a master clock generator. when a parallel load is done into the shift register it begins spitting out data at the tick of the master clokc. if you really need rs232 its a matter of making a 11 bit shift register holding the 2 start bit , 8 databits and the stopbit.

minimalistic code ont he vinculum opens a fixed filename and begins streaming it through the parallel fifo bus ( no need to resreialize . the vinculum spits out bytes in parallel on a clock base. you can have the fpga deliver this clock to the vinculum.
basically the vinculum opens the file and waites for a 'command; form the fpga'. the fpga says 'next byte' and the vinculum puts it on its outgoing databus. when last byte is reached you handshake through another signal adn the whole thing begins again.

I need to be able to tell a customer to take the video, shove it through Virtualdub with the right settings to produce an uncompressed AVI and put it on the memory device, so funky formats are not an option.
I doubt the Vinculum will get anywhere near 12mbits sustained transfer rate - that's just the raw USB rate. However a High speed host should have enough spare bandwidth to be able to waste a lot and still be fast enough.

[Psi]

As nice as Virtualdub is, it's quite old now and cant read newer video files, like mp4 flv mkv etc..

[nctnico]

If you require precise timing, complete dependence on data being available, you will want to buffer the data in some sdram. How much data are we talking about?
32mb or 64mb sdram is available on the Terasic DE0 devboard (cheap) you can buffer locally in some blockram FIFOs to get around the sdram refreshes.

Probably only a few tens of K - I need a consistent framerate with frames of the order of 10Kbytes, at maybe up to 50fps. I typically need to put a whole frame in RAM as it will be sliced out over multiple UARTs, and double-buffer it. Internal RAM on an FPGA should be plenty

If you use a fast (ARM) controller which has a data/address bus and seperate UARTs (with FIFO) you might be able to pull this off without an FPGA. You might underestimate the complexity of the FPGA design because you may want / have to construct your own UART. The UARTs available from Opencores are not the best to put it mildly.

[marshallh]

The UARTs available from Opencores are not the best to put it mildly.

Open sores, as I call it. And yes they are total crap.

[mikeselectricstuff]

As nice as Virtualdub is, it's quite old now and cant read newer video files, like mp4 flv mkv etc..

I like it because it does a simple job very well with no unnecessary bells and whistles.
It adoesn't complain when I want to output bizarre uncompressed video formats like 800x1 pixel monochrome, 16 bit 5:6:5 colour or (as I'm doing right now) 4x1 pixels RGB with a mono 48K audio.
I can also hack together a filter for things like error diffusion dithering for smoother 5:6:5 displays by modding an example filter without needing to know too much about the plugin architecture.
I can just tell a customer to go download it (no installation needed) , set a few settings & it just works.
Maybe there are other options out there but if it ain't broke....
 

[mikeselectricstuff]

The UARTs available from Opencores are not the best to put it mildly.

Open sores, as I call it. And yes they are total crap.

A transmit UART is so trivial to implement I wouldn't even think to look at using a ready-made one. 

[mikeselectricstuff]

If you require precise timing, complete dependence on data being available, you will want to buffer the data in some sdram. How much data are we talking about?
32mb or 64mb sdram is available on the Terasic DE0 devboard (cheap) you can buffer locally in some blockram FIFOs to get around the sdram refreshes.

Probably only a few tens of K - I need a consistent framerate with frames of the order of 10Kbytes, at maybe up to 50fps. I typically need to put a whole frame in RAM as it will be sliced out over multiple UARTs, and double-buffer it. Internal RAM on an FPGA should be plenty

If you use a fast (ARM) controller which has a data/address bus and seperate UARTs (with FIFO) you might be able to pull this off without an FPGA. You might underestimate the complexity of the FPGA design because you may want / have to construct your own UART. The UARTs available from Opencores are not the best to put it mildly.

Yes I can probably do it all in software, except possibly for the "20 UARTS" part. Although it can be bit-bashed even at quite high rates (I've done 8x250kbaud on an AVR in the past)  the amount of bit-shuffling you need to do can get excessive and easier to do in an FPGA, especially if your filesystem is tight on bandwidth - you don't want to slow it down with a high interrupt load.
The timing stuff is pretty easy as long as the worst-case latency plus loading time is less than one frame period. Background task throwing out a frame on a timer int with a two buffer swap, and a foreground task filling the buffer. Gets messier if you have unpredictable latency (e.g. cluster boundaries), but even then it's usually just a case of adding more buffers to make a FIFO.

[sca]

Was this SD in SPI mode or full 4-bit? Documentation on the latter appears scarce as you need to join the SD card club to get it.
This may be a viable approach, even just  doing it a sector at a time.

Not sure if it meets any of the other requirements, but, IIRC, the LPC2300 range has an MCI port with 4 bit SD, and versions of the elm-ChaN Fat-FS stack support it.

sca

[andersm]

They also have a new host that is high speed.

hang on ...

FT120. take a look at that. and it has a parallel data/address bus.

Nope, not that one either. The FT313H is the one you want.

[mikeselectricstuff]

They also have a new host that is high speed.

hang on ...

FT120. take a look at that. and it has a parallel data/address bus.

Nope, not that one either. The FT313H is the one you want.

That's just a host peripheral - need to add an MCU plus USB stack plus filesystem

[nctnico]

If you require precise timing, complete dependence on data being available, you will want to buffer the data in some sdram. How much data are we talking about?
32mb or 64mb sdram is available on the Terasic DE0 devboard (cheap) you can buffer locally in some blockram FIFOs to get around the sdram refreshes.

Probably only a few tens of K - I need a consistent framerate with frames of the order of 10Kbytes, at maybe up to 50fps. I typically need to put a whole frame in RAM as it will be sliced out over multiple UARTs, and double-buffer it. Internal RAM on an FPGA should be plenty

If you use a fast (ARM) controller which has a data/address bus and seperate UARTs (with FIFO) you might be able to pull this off without an FPGA. You might underestimate the complexity of the FPGA design because you may want / have to construct your own UART. The UARTs available from Opencores are not the best to put it mildly.

Yes I can probably do it all in software, except possibly for the "20 UARTS" part. Although it can be bit-bashed even at quite high rates (I've done 8x250kbaud on an AVR in the past)  the amount of bit-shuffling you need to do can get excessive and easier to do in an FPGA, especially if your filesystem is tight on bandwidth - you don't want to slow it down with a high interrupt load.
The timing stuff is pretty easy as long as the worst-case latency plus loading time is less than one frame period. Background task throwing out a frame on a timer int with a two buffer swap, and a foreground task filling the buffer. Gets messier if you have unpredictable latency (e.g. cluster boundaries), but even then it's usually just a case of adding more buffers to make a FIFO.

Serving 20 UARTs isn't a problem if they have to transfer the same amount of data. You only need to use one interrupt from the last UART you serve. If the last UART's FIFO is half emtpy (*), you can be sure you can dump half a FIFO size worth of data into the UART. If the data bus is wide enough you might be able to write 2 or 4 bytes to the UARTs in one cycle.

(*) Choosing the right low water mark takes some tuning.

[mikeselectricstuff]

Serving 20 UARTs isn't a problem if they have to transfer the same amount of data. You only need to use one interrupt from the last UART you serve. If the last UART's FIFO is half emtpy (*), you can be sure you can dump half a FIFO size worth of data into the UART. If the data bus is wide enough you might be able to write 2 or 4 bytes to the UARTs in one cycle.

(*) Choosing the right low water mark takes some tuning.

The problem is not so much serving 20 UARTs, but finding an MCU that has 20 UARTS to service..!
I also sometimes need to add a little inter-byte padding e.g. 3.5 stopbits due to constraints at the receive end (typically a couple of hundred PIC10F322s doing a soft rx UART at 333kbaud) ,  this is doable using a timer int instead of UART ints but  precludes the use of UART FIFOs to reduce interrupt loading. Easier to just throw it from the disk as it comes, out of a reasonably wide port and let an FPGA sort out all the interleaving and critical timings .

[nctnico]

You can still buy UARTs as a seperate chip. You just need 5 of these:
http://uk.farnell.com/exar/xr16v554ij-f/ic-uart-quad-16b-fifo-68plcc/dp/1716899

You can always opt for sending the data from a timer interrupt. If you know the baudrate then you know when its time to send the next bytes. A 33.3kHz interrupt is very doable on an ARM. I recently made a design which uses a 10kHz interrupt to send bytes over an SPI bus while the rest of the software may be busy on writing to an SD card. By the time the next timer interrupt comes the previous SPI transfer is done so its safe to dump the data into the TX buffer.

edit: I don't think you'll need an address/data bus. If you choose the databus as consequtive bits on a port you could 'bit-bang' it and still be fast enough. Maybe its even too fast for the UART to keep up...

[Marco]

Erich Styger has done quite a bit of open source work to make the Kinetis chips work smoothly with SDHC. Integrating FatFS (read only open source implementation of FAT) with Freescale's SDHC library into an eclipse component. Dunno what the throughput is though. One of many entries on the topic :

http://mcuoneclipse.com/2012/11/18/arduino-data-logger-shield-with-the-frdm-kl25z-board/

[amyk]

Is CF an option? The protocol is much simpler (basically ATA)  and the interface is 16 bits wide. You can have the microcontroller do directory traversal/file finding, and send the list of blocks in the file to the FPGA, to be stored in its RAM. It can then read the blocks itself and push the data out its UARTs, no additional filesystem overhead required so the latency is fixed; you can even clock the card synchronous to the data being output.

10Kbyte/frame * 50 frame/sec / 2bytes/clock = 250KHz CF clock.


The same idea could be done for SD although at higher clock frequency. Not possible with USB.

[mrflibble]

Maybe something like the CY7C67300 would do the trick?

Currently in stock at digikey/mouser.

[mikeselectricstuff]

Is CF an option? The protocol is much simpler (basically ATA)  and the interface is 16 bits wide. You can have the microcontroller do directory traversal/file finding, and send the list of blocks in the file to the FPGA, to be stored in its RAM. It can then read the blocks itself and push the data out its UARTs, no additional filesystem overhead required so the latency is fixed; you can even clock the card synchronous to the data being output.

Yes, quite possibly - not quite as small or ubiquitous as SD or USB, and more work on PCB layout but would be acceptable.
Small size is sometimes important  - I did one SD player using a QFN PIC24, which had the same outline as the MicroSD card. However higher channel counts are less likely to be space-constrained. 
The Microchip library does appear to support CF over the parallel master port peripheral so may be worth  a look. It's not supported on the Explorer devboard, so may not be as well tested or supported but may be worth a go.
May even be able to just passively skim off the parallel data stream into the FPGA.

Maybe something like the CY7C67300 would do the trick?

Not high speed.

[firewalker]

Just curious. You are using monochrome LEDs or RGB ones?

Alexander.

[mikeselectricstuff]

Just curious. You are using monochrome LEDs or RGB ones?

Alexander.

Usually monochrome. RGB often looks nasty due to colour balance issues. Mono (usualy white) is a lot more power efficient and usually looks nicer.
If you're doing interesting shapes and forms with LEDs, then colour is often only a small incremental improvement over the impact of the basic form, but adds a lot of complexity, cost and power draw. 
For example This and This (colour tint via filters), This (yellow LED) and This

[bingo600]

How about Xmos (the old INMOS xputer design .. renewed)
http://www.xmos.com/discover/products/xkits/reference

/Bingo

[mikeselectricstuff]

How about Xmos (the old INMOS xputer design .. renewed)
http://www.xmos.com/discover/products/xkits/reference

/Bingo

XMOS is something I've been meaning to look at - I even have an eval board somewhere. However it doesn't like they have any host stuff (yet)

[JVR]

Another idea.

Take a Cortex M3/M4, add a few MB of SDRAM on its EBI port, let it run a basic USB host.

You then plug the flash drive into the unit, cortex copies files from USB to SDRAM, and voila, deterministic clocking at any speed you want.

Hell of a lot less dev time / expense than an FPGA.  A decent 160Mhz+ M4 will probably do 20 SW UARTS without bitching to much.

[firewalker]

There will be about 10k pixels (leds) with 1 byte per pixel (?) @ 50 fps. So 10000*1*50 = 500.000 bytes per second. Does that sound correct?

Alexander.

[ptricks]

There is USB full speed (12Mbps) and very common on chipsets, then there is USB(480Mbps) high speed much harder to find on chips that do not contain processors and lots of pins .
I would copy the data to flash memeory from the sd card or usb drive and then use that as the source for the main processor. The cost would be lower and you don't need to use anything expensive to perform the task. You can use any 12Mbps capable micro ,  once it is in flash you can easily read it at high speed.  You wouldn't even have to change chips, you could still use the Pic24 chip to read the data , shift the bits out to flash and the pic24 could power off.

[bingo600]

I think i read that the $16 STM43F4-Discovery board will do HS-USB if you add an external USB-Phy
http://www.st.com/stm32f4-discovery
It can run up to 160Mhz

https://my.st.com/public/STe2ecommunities/mcu/Lists/STM32Discovery/Flat.aspx?RootFolder=%2Fpublic%2FSTe2ecommunities%2Fmcu%2FLists%2FSTM32Discovery%2FSTM32F4discovery%20hight%20speed%20USB%203216&FolderCTID=0x01200200770978C69A1141439FE559EB459D75800084C20D8867EAD444A5987D47BE638E0F&currentviews=250

ULPI IF
http://www.ebay.com/itm/USB3300-USB-HS-Board-Host-OTG-PHY-Low-Pin-ULPI-Evaluation-Development-Module-Kit-/251069458682?pt=LH_DefaultDomain_0&hash=item3a74e7e8fa#ht_1299wt_1168

STM Doc (Search ULPI)
http://armvn-repo.googlecode.com/files/STM32F4-Technical-Training.pdf

STM ULPI Bug (Check the ULPI DS before using a specific one)
https://my.st.com/public/STe2ecommunities/mcu/Lists/cortex_mx_stm32/Flat.aspx?RootFolder=%2Fpublic%2FSTe2ecommunities%2Fmcu%2FLists%2Fcortex_mx_stm32%2FSTM32F2F4%20problems%20with%20various%20ULPI%20USB%20PHYs&FolderCTID=0x01200200770978C69A1141439FE559EB459D7580009C4E14902C3CDE46A77F0FFD06506F5B&currentviews=524


/Bingo

[bingo600]

Seems like the STM is a but buggy unless you use their ULPI chip

This is a NXP with 32M Ram & USB-HS with onboard Phy (a bit pricey)
http://www.ebay.com/itm/Olimex-LPC-H3131-nxp-LPC3131-arm9-high-speed-USB-interface-SDRAM-FLASH-AUDIO-/280948717610?pt=UK_BOI_Electrical_Components_Supplies_ET&hash=item4169d9382a

Another 99€ + vat + ship
http://www.embeddedartists.com/products/oem/lpc3152_oem.php

With board 179€ (outch)
http://www.embeddedartists.com/products/kits/lpc3152_kit.php


The big lpc1830/50 sems capable also (Note the quad spi flash lanes)
http://www.digikey.com/us/en/ph/nxp/lpc1830-cortex-m3-mcu.html

But as with every ARM , check the erata before buy ...
The feature yo need might be buggy

Olimex claims this one has a USB HS port (nice price) , but sw ??
https://www.olimex.com/Products/OLinuXino/iMX233/iMX233-OLinuXino-MICRO/

More olimex (check if PHY is onboard/onchip)
https://www.olimex.com/Products/OLinuXino/iMX233/


Cypess new USB3 chip - EZ-USB® FX3
http://www.cypress.com/?id=3526&tabID=1
http://www.cypress.com/?docID=39055
http://www.cypress.com/?docID=41315
http://www.silica.com/product/cyusb3014-bzxi-ez-usb-fx3-usb-30-peripheral-con.html
http://uk.farnell.com/cypress-semiconductor/cyusb3kit-001/ez-usb-fx3-usb-3-0-jtag-dev-kit/dp/2082746

Cheaper board (But check if host)
http://agile-sdr-solutions.com/USB-3-0-Dongle-DVK
http://agile-sdr-solutions.com/USB-3-0-DVK

[andersm]

This is a NXP with 32M Ram & USB-HS with onboard Phy (a bit pricey)

Both the LPC3xxx and the Freescale i.MX2xx are ARM9-based applications processors, usually used with Linux and the like, I think Mike was looking for something more like a microcontroller.

[tesla500]

If you have an FPGA, why not use the Opencores SD card controller along with the open source FATFS stack, and read an SD/SDHC card directly? I have this working in one of my applications, getting about 1.4MB/s read with no optimization (only single block read, bursts of multiple blocks would be much faster).

[Psi]

You can get some pretty fast CF cards. (They're used in the high end DSLRs where speed is needed for high burst speed with RAW images)
They also make "rugged" CF versions with large operating ranges for temp and humidity (if that's useful)

The disadvantage is SD is slowly taking over CF, but it will sill be around for a while to come.

[mikeselectricstuff]

I noticed a cheap LPC1830 board at Farnell when I was looking for something to make up a minimum order,
http://uk.farnell.com/nxp/om13028-598/eval-cortex-m3-lpc1830-xplorer/dp/2217599?in_merch=New%20Products
This has SDHC and USB HS. The other thing I like about this range is that it has a huge 200K of RAM, which is often the first thing I run out of.

[bingo600]

This is a NXP with 32M Ram & USB-HS with onboard Phy (a bit pricey)

Both the LPC3xxx and the Freescale i.MX2xx are ARM9-based applications processors, usually used with Linux and the like, I think Mike was looking for something more like a microcontroller.

I do agree , but USB-HS with an installed (here onboard phy) isn't easy to get.
Maybe the 1830 is in the mcu class.

I just saw this lpc3131 board at a reasonable price
http://shop.embedded-projects.net/index.php?module=artikel&action=artikel&id=1296
http://gnublin.embedded-projects.net/

While i agree it's mostly used for linux , it's just an arm9 , and this ona also have 32M Ram

/Bingo

[mikeselectricstuff]

Onboard Phy & lowest cost is way down the list compared to something I can get working easily.
Thinking about it some more, I think SD(HC) is the way to go, as at least I'd stand a chance of debugging a "compiled it all but it's not working" situation - with USB there are so many things that could be wrong it would be a total nightmare to track down.
Also with SD it will be much easier to scope the pins to work out exactly where any performance bottlenecks are.

[mikeselectricstuff]

Just had a quick look at FatFS & looks very nice  & well documented. Also doesn't suffer from being split into gazillions of different source and header files, - something I hate about some C examples you find as it makes it very hard to follow.

The cheap NGX LPC1830 board I bought comes with some sample code for SD sector reads/writes in IAR so I'll have a go at welding the two together...

An interesting possibility with the LPC1830 is that as it has a nice fast clock and plenty of RAM to run code from (or put lookup tables in) , I may be able to implement a fairly large number of bit-bashed fast TX UARTs in software. Twenty 250kbaud ports at 150MHz is 30 instructions per bit per port, which should be doable.
I've done some hairy FIQ assembler on ARMs in the past so could be fun - pity they took out FIQ mode from the M3 as its dedicated register bank was very handy when doing very high rate interrupts.
 

[talsit]

This looked promising but speed is limited by the slow interface to the host, and the 4MBytes/sec speeds claimed turn out to only be useful for card-card copying.
http://www.ghielectronics.com/catalog/product/340
I think this is based on a ST ARM Cortex M3 chip.
If they had a way to, for example read USB and spit it out of the 4-bit SPI interface, under control of the UART port that could be ideal.

I'm using this for a project of mine. I'm doing basically fast writes to an SD, I never really read data off it, so I never looked at the read part, but after looking through it again, it's kinda lame. They support direct DMA writes, so in theory you can get quite fast throughput, but reading is done through their standard interface, which needs 4us delay between bytes. That's incredibly silly, especially since they have everything else.

[talsit]

I'm using this for a project of mine. I'm doing basically fast writes to an SD, I never really read data off it, so I never looked at the read part, but after looking through it again, it's kinda lame. They support direct DMA writes, so in theory you can get quite fast throughput, but reading is done through their standard interface, which needs 4us delay between bytes. That's incredibly silly, especially since they have everything else.

Not anymore!! I've abandoned the ALFAT, since it's an incredible load of utter crap. I was doing tests with it, trying to get the "Fast SPI Write" command running, and I noticed that between issuing the command and ALFAT telling it was ready for me to send the first 8KB block, it would take around 100ms. Then, at the end of that command, it would take a further 50ms. You could do multiple 8KB transfers per "Fast SPI Write", but if you're streaming data to disk, this 150ms total wait just kills you. The worse thing is that nowhere in the datasheet does it say that this is normal. I asked the manufacturer and they very calmly responded that this is normal, and sorry that it wasn't in the datasheet. So, effective throughput for streaming on a device with less than 128KB free is around 50KB/s, not the advertised 1.4MB/s.

So, I recall my recommendation....