Which uC to tackle Camera project?

[soFPG]

I really love to play with my camera in my spare-time and I also like electronics and microcontrollers (and software development).

So I thought - why not combine everything and build a simple camera?
I know that Omnivision CMOS sensors are quite common but they are old sensors as far as I am aware of and the also come with a pretty bad lens (which is also not interchangeable and removing it will have a high chance to break the sensor).

There are some new 2MP ones (OV2718, OV2775) which are produced as a backside illuminated sensor (BSI). As far as I know this feature should increase low-light capabilities drastically.
These are 1/2.9 inch sensors (x7 crop factor) and there are a lot of CCTV-lenses on Aliexpress etc. which should be a good fit for them. But there are some problems:


• They come in a chip-scale package with 77 pins with a 0.61mm ball pitch -> I don't think JLCPCB is able to manufacture this?
• One can decide between MIPI CSI-2/LVDS and 12-bit parallel DVP output (I am not familiar with these interfaces so don't really know what to choose but with MIPI you can achieve 120fps compared to only 42fps with DVP)
• Which microcontroller would fit this project? They say 9 - 30MHz clock input so 1920 x 1080 pixels x 12-Bit ADC ~ 3Mbyte / s. But there is not only this large amount of data but also some post-processing which must happen as the pixels are arranged in a Bayer-pattern (this means each pixel either has the color green, red or blue)

And then I also have to output the pixels on a high-resolution color display (at least 320x240). I would assume the uC needs quite a high amount of processing power for this?

What are your suggestions?

[nctnico]

This is something a Raspberry Pi or similar is a good platform for.

[tmadness]

If you need a microcontroller, sony makes these https://developer.sony.com/develop/spresense/ . They are absolute beasts, probably need to dive deep into the architecture to program them.

[Scrts]

Which microcontroller would fit this project? They say 9 - 30MHz clock input so 1920 x 1080 pixels x 12-Bit ADC ~ 3Mbyte / s.

You assume 1FPS? Because otherwise, you should estimate like this:
1920 x 1080 (x 1.15 for blanking if you prefer) x 30FPS [OV2775 limitation if full frame] x 12bit (x2 times capture if you want HDR). But these sensors output RAW Bayer data. You'd need to couple it with, lets say OV491 or OV495 to get YUV422 output.

[JLCPCB Official]

Hello ;

You can make the schematic of a customized design for a camera and then produce it.. make sure that you are using the appropriate uC to handle the high volume of frames generated by the camera sensor.. a high frequency uC is good and nothing less than 32 architecture and it would be better if the uC has a I²S and separated DSP for a better audio and signal processing. I recommend the use of ST uC

[chriva]

bitluni managed to connect a camera to an esp32 and make it work ( I swear something is wrong with that guy and I love him for that ).

Those things are surprisingly powerful as long as you don't push too many fpu calculations but I had no clue they could do that.

[Kasper]

ESP32-cam for cheap, low power, wireless camera.  Downside is hardware quality.  I bought 2 and one worked. Lots of amazon reviews had similar stories.

Ardu-cam for wide variety of options. That can be plugged into a variety of MCUs.

Raspberry pi cam for easiest programming.  Downside is boot time. With the usual programming it takes much longer but you can boot to webserver in a few seconds if you want to do it the hard way.

[fchk]

nVidia Jetson Nano.

Has a bunch of CSI-Lanes, hardware image accelerators and a GPU for image processing with CUDA, so more then enough power.

Or the bigger brother Xavier NX with even double the computing power.

https://developer.nvidia.com/embedded/jetson-nano-developer-kit
https://developer.nvidia.com/embedded/jetson-xavier-nx-devkit

The system is open, so you can use just the compute module itself and design your own baseboard.