Complete computer vision module in 0.78 cubic centimeter

[blueskull]

Greetings.

As a side project emerged from my desktop pick and place machine, I separated its computer vision module into a brand new OSHW project, the ExpeVision.

This project aims to provide simple computer vision functions in a small physical package and with an easy-to-use interface and development tools.

At its core is a combination of a CMOS image sensor, an AI coprocessor and a WiFi SoC.

The CMOS image sensor, OV5640, captures the image, and offers cropping, scaling and corrections before feeding the area of interest to the AI coprocessor.

The AI coprocessor, K210, receives up to 640*480 image frames from the sensor, pre-processes them, then runs them through a convolutional neural network processor. It also compresses them into JPEG format for streaming.

The WiFi SoC, ESP32, handles wireless SDR stack, network server and other user apps, as well as bootoading the K210.

To make it really easy to use, all tricky parts of such a system, including USB support, WiFi RF circuitry and the lens system, are all packaged in with no user modification needed.

Similarly, framework for K210 and ESP32 firmware are planned to be offered so application development is made simple.

------------------------------

The first release is an all-in-one design which places all components on one PCB. This is for prototyping only, and will not be the final form factor.

Once the concept is validated, there will be an LGA version based on stacked PCBs, where all components are placed on multiple stacked PCBs to save planar area. This version will be 18mm (L) * 9mm (W) * 4.8mm (H).

------------------------------

The GitHub link contains the schematic of the AIO prototype version 2.0 drawn in KiCAD. PCB layout is on its way.

BTW, I went a bit more aggressive on ver 1.0 by directly jumping into the stacked version, and it ended up being a huge mess that is not even debuggable. I also changed a few chips to greatly reduce its size and cost.

------------------------------

https://github.com/blueskull/ExpeVision/tree/AIO

As usual, critics and comments are greatly appreciated. I've quaduply checked it, and for the first three times I could all find bugs here and there, and I'm pretty sure there might be hidden bugs even in this version.

[jduncan]

This is awesome, I've been playing with Omnivision cameras and the K210 for other projects and the thought crossed my mind just yesterday that they might be useful for OpenPNP. I'm interested in speeding up and augmenting the vision pipeline as I'm planning to use mainly loose part and cut tape feeders.

Of course, the mechanical components I purchased for my project have been sitting untouched since pre-COVID, so I'm glad someone else is actually making it happen.

[SilverSolder]

This sounds very awesome.  So will the module be able to recognize misaligned components etc. on its own, or is that the job of the upstream system?  (Struggling to understand what the neural network will be doing...   will it be configured by the upstream system for the particular components in use, or something like that?)