Lessening Load on RbPi for Object Detection by using FPGA (Want Advice)

[Mattjd]

Hi, I am looking into methods for object detection on a drone of mine. I was reviewing the use of OpenCV and Raspberry Pi, as described in this article. I've got that tutorial running on my desktop using a camera. I have a raspberry pi 3 and purchased a camera and plan on getting that working.

Ideally I'd like to get the object detection running in real time. The article however states that the Raspberry Pi would be a poor choice for this as it is only capable of 0.9 FPS.

I'm rather new to raspberry pi and computer vision. I do have a good deal of experience in machine learning but I realize that openCV has pretrained neural nets,so the machine learning is for the most part abstracted away, and I'll have no training of a neural net to do, only testing (ie classification).

This is a bit of a shot in the dark, but while thinking of ways to try to make the Raspberry Pi more feasible (I'd rather not ditch it, because openCV makes the computer vision so easy), I was thinking of capturing the images with a camera connected to a FPGA. I have a DE0 development board, and found this camera on Altera's website. Now the DE0 has a SoC, and considering that openCV is just a wrapper of C code. I was wondering if there is someway to lessen the burden on the Raspberry Pi by capturing the images with camera connected to DE0, using the fpga (EP3C16F484C6, Cyclone III) and SoC, to do all the processing necessary, and sending the data over SPI (or maybe DMA, but idk if thats possible) to the Pi for actual classification.

Again its a shot in the dark, and I'd like to investigate it, but before wasting the time, I was hoping if someone with more expertise could give their opinion if what I want would even be capable and/or worth it. Or I would even take an alternate solution.

[xani]

NVIDIA Jetson boards are kinda targeted for that (It's ARM CPU + NVIDIA GPU), altho quite a bit more expensive than rPi.

[Mattjd]

NVIDIA Jetson boards are kinda targeted for that (It's ARM CPU + NVIDIA GPU), altho quite a bit more expensive than rPi.

that does look it like it would do that trick, I could train everything on my desktop or amazon/google services and then use that for classification, but boy I dont have $600 laying around.

[Mattjd]

NVIDIA Jetson boards are kinda targeted for that (It's ARM CPU + NVIDIA GPU), altho quite a bit more expensive than rPi.

In your opinion though, what is the feasibility of my idea?

[Mattjd]

So Ive been thinking about this more. I guess what I am ultimately asking is if I decide to capture and preprocess the images using the FPGA, and then send it over SPI to the Pi (which will only have to perform the classification). Will this new process be fast enough that the pi will be able to handle more than .9 FPS?

[Mattjd]

Is this too advanced for the beginners section? Would I be best served moving it to the embedded/fpga section?

[iMo]

Lattice claims they are the best in that area..
https://www.latticesemi.com/

[tyrel]

You could use a movidius stick (https://developer.movidius.com, https://github.com/movidius/ncappzoo).
It is compatible with the raspberry pi, and with a lot of existing network architectures (not all, only caffe and tensorflow models, and there is some limitation on the network size)

[Mattjd]

You could use a movidius stick (https://developer.movidius.com, https://github.com/movidius/ncappzoo).
It is compatible with the raspberry pi, and with a lot of existing network architectures (not all, only caffe and tensorflow models, and there is some limitation on the network size)

that actually looks like a very promising start, and its so cheap.

https://www.pyimagesearch.com/2018/02/19/real-time-object-detection-on-the-raspberry-pi-with-the-movidius-ncs/