Very fast memory to accompany an MCU

[Entropia]

Hi. I've been pondering about this idea of capturing single frames from a CMOS imaging sensor. The sensor outputs 5 megapixel images (2592x1944) at 12 bits per pixel. This data is coming in parallel format, clocking in at 48 MHz so I can definitely capture it with an STM32F4, but the problem is storing it. The STM32F4 has no room for this amount of data so the only solution is external DRAM. I'm thinking maybe a CPLD to act as a DDR controller. But what about DRAM refreshing... I've skimmed through a Xilinx appnote regarding CPLD DDR controller but it doesn't say how long it takes to refresh the memory. Maybe a BUSY-output on the CPLD to indicate when not to write, combined with a FIFO buffer? Any thoughts? The external memory doesn't have to be CPLD/DDR but that is definitely a bonus since DDR and CPLDs are cheap.

After the frame has been stored, I can read it back, crunch it into a JPEG and store it on an SD card at my own leisure.

[Psi]

Wouldnt some SRAM be easier to work with?

[Entropia]

Sure. But it takes a heapload of I/O pins away from the MCU (I could implement some sort of SPI using the CPLD if I went that route) and a sufficiently large SRAM in hand-solderable package costs well over 100 EUR (VAT 0, checked at Digi-Key). So, scratch that.

[T4P]

Or some notebook DDR3 1GB Modules that are underclocked ? (19SGD$ for the kingston DDR3 8500 / 1066 CL7 )
Sure , you can use DDR but they are astronomically priced now .
But yeah you would need many astronomically priced SRAM's to make a 700mbit module , and it looks like you would need a high pin count FPGA to save costs or you can have a DDR module 184 pins at 50$ instead of DDR3 modules at 240 pins and some advanced clock timing but you can copy the details anyway
But there are sockets for them .

[jpelczar]

Do you have to use STM32 ? Maybe it would be better to choose a part with integrated SDRAM controller ? I'm using LPC3250 in my next project, so you could use e.g. some LPCXXXX part or AT91 ARM ?

The other solution could be FPGA attached to the STM32 FSMC which would work as a write-only external RAM with fast SRAM FIFOs and SDRAM controller, but for me it seems like overkill.

[Psi]

But yeah you would need many astronomically priced SRAM's to make a 700mbit module

The 700mbit requirement was a different thread/person.

As I understand it, he only needs enough ram to store one full frame. Then it can be jpeged and saved to SDcard.
So 2592x1944 at 12 bits equals 7,558,272 bytes

[jpelczar]

So he could use 8 megabyte PSRAM (13 bucks on Digikey).

[T4P]

Er ... Wrong thread sorry .
A simple 10MB buffer would do .

[free_electron]

PSRAM is the way to go , the FSMC in the STM32F4 knows how to deal with that.

[Entropia]

I don't have to use STM32F4, but I'd like to get to know it better. Also I think PSRAM is ruled out since 64Mb sizes are in BGA only and they are non-stock (0 available) at Digi-Key. Farnell doesn't have them at all.

LPC32x0 seems nice but again, those are only available in BGA which I have no way of soldering. AT91 seems promising as well, but a 180 MHz one with EMI (AT91RM9200-QU-002) costs around 24 euros at Digi-Key. Comparing that to STM32F407ZGT6 the price difference is 13 euros for which I can get a DDR chip and a CPLD. If I wanted to go ARM, I'd probably go for Freescale i.MX233 (MCIMX233DAG4B) which runs at 450 MHz, has DDR controller, comes in LQFP and only costs around 9 euros each. I'm not ungrateful for the suggestions, don't get me wrong.

Maybe I could just clock the CMOS imaging sensor at really low speeds, get a fast enough SD card and eliminate the buffer memory altogether.

[jpelczar]

How do you access the camera from microcontroller ? Maybe you could use CPLD/FPGA to completely bypass the STM32 and grab data directly from the camera. When frame is ready notify the STM via interrupt and provide SPI (or parallel) interface to read the data back from SDRAM ?

[T4P]

Hmm ... there are couple of differences with the STM32F4 of course , it's a speed demon .
And a rather power hungry demon .
And , the MCIMX233DAG4B is a MPU not MCU .

[Entropia]

How do you access the camera from microcontroller ? Maybe you could use CPLD/FPGA to completely bypass the STM32 and grab data directly from the camera. When frame is ready notify the STM via interrupt and provide SPI (or parallel) interface to read the data back from SDRAM ?

The camera (MT9P031) has a parallel databus with some control signals (much like HSYNC/VSYNC on VGA). Commands go in via I²C. Your idea is actually pretty good and I thought of it. The reason why I dismissed it before was that FPGAs need external bitstream storage whereas with CPLD you don't need any other chips. But... Why not?

the MCIMX233DAG4B is a MPU not MCU

You are correct, but in this case I would need the external memory anyway and I could share the SD card for MCIMX233 program memory as well. So parts count isn't affected in this case.

[riktw]

LPC4330 (available in LQFP144) and some SDRAM should do the trick.
The serial GPIO function can clock in data a several dozen of MHz, sounds fast enough to me

[nctnico]

Hi. I've been pondering about this idea of capturing single frames from a CMOS imaging sensor. The sensor outputs 5 megapixel images (2592x1944) at 12 bits per pixel. This data is coming in parallel format, clocking in at 48 MHz so I can definitely capture it with an STM32F4, but the problem is storing it. The STM32F4 has no room for this amount of data so the only solution is external DRAM. I'm thinking maybe a CPLD to act as a DDR controller. But what about DRAM refreshing... I've skimmed through a Xilinx appnote regarding CPLD DDR controller but it doesn't say how long it takes to refresh the memory. Maybe a BUSY-output on the CPLD to indicate when not to write, combined with a FIFO buffer? Any thoughts? The external memory doesn't have to be CPLD/DDR but that is definitely a bonus since DDR and CPLDs are cheap.

After the frame has been stored, I can read it back, crunch it into a JPEG and store it on an SD card at my own leisure.

Why not use a controller which can handle SD or DDR DRAM? This would solve a whole lot of problems. Such a controller may even have a camera interface. There is more to reading a camera than just capturing the data! You'll need to take care of the Hsync and VSync as well.

[A-sic Enginerd]

Have fun with it. Slap down an FPGA and build your own DDR controller. But then I'm a digital design puke so maybe you'd rather skip it and not invest that much energy into this aspect of your project....I get that.

DDR controllers can be non-trivial things to design. Every project I've ever worked on had a guy that was dedicated to nothing but designing the memory controller. Typically you'll find separate modules / blocks that handle things like refresh, and an arbiter that coordinates with it to mux in "client" requests to access the memory. Yes, there needs to be some buffering someplace. How much and where to put it is application and implementation specific. Will need to do some calculations on access latency, time to do a refresh, etc. compared with how much data will be streaming in during that time.

Not a trivial problem to solve, but can be a fun one if you want to go there.

Oh, and be sure to look to the datasheet for the specific memory part you're using before making any assumptions about what's required of the controller.

[Entropia]

This whole camera thing has been marinating in my head for a few weeks now and I think I'll bring a shotgun to a knife fight by implementing FPGA. I already have a couple of Xilinx FPGA dev kits collecting dust. The Spartan 3E SK has the required DDR on board so I don't have to worry about HW issues just yet. I think I'll connect the imager directly to the FPGA and the MCU talks to the imager via FPGA. When a frame has been snapped the FPGA tells the MCU a frame is ready to be processed and things progress from there...

This might change though, since I already have a few projects under development. If you guys have more good ideas, let's hear them!

(Changed the topic heading to reflect the discussion better.)