FPGA for video processing...?

[Jackster]

I got a product idea and I am looking at coming up with a prototype to demo it to potential users of the device.

It processes video and displays information about that video feed though overlaying data such as how bright a pixel is, edge detection and changing the colour of a pixel based off a 3D LUT file. Very similar to how SmallHD monitors work. https://www.smallhd.com/

I am looking at farming out the prototype programming work so just after knowing if something like the Digilent Arty Z7 would be a good dev board for something like this?
Not that knowledgeable on FPGA. I know how they sort of work but would not know where to start if you gave me one.

The spec I need is HDMI in and out with enough processing speed and RAM to work with 1920x1080 video frames.


Thanks
Jack

[Berni]

Yep this is the exact application that FPGAs are good at.

How big of an FPGA you need depends a lot on the exact details of what you are doing. Memory is expensive inside FPGAs (but incredibly fast) so you usually try to avoid buffering entire frames of video if possible, a lot of the time buffering just a few lines is enough. But if you do need to buffer entire frames then external DDR memory might be needed. Due to the parallel nature and precise timing of FPGAs its often possible to process things on the fly as data passes trough without even buffering it at all. Your application sounds like it doesn't need to hold entire frames in memory, so you could just pipe video trough it while introducing near zero time lag (Probably about 1/200th of a frame)

Also if you want to do HDMI directly into the FPGA you need a chip that supports such high speed IO lines, Alternative is to use external chips to convert HDMI to parallel RGB and back again.

The dev board you have suggested is indeed a good choice. It gives you the ports you want and has plenty of resources. The included ARM processor is also very useful for doing any UI stuff.

[rstofer]

If you run out of horsepower on the Arty board, Digilent makes a board more oriented toward A/V:
https://store.digilentinc.com/nexys-video-artix-7-fpga-trainer-board-for-multimedia-applications/

It use a MUCH larger chip which brings along more logic, more memory, more IO lines and so on.

And, yes, it costs a LOT more money.

[hamster_nz]

Development of the prototype's firmware to 'feature demo' status will be the bulk of the product development work. The PCB design will be mostly moving the DevBoard's electronics to a new PCB, and adding missing features (e.g. power mgmt, user interface).

It is close to impossible to develop a feature-rich 'demo' video pipeline that isn't nearly a production ready video pipeline, unless you make it using resource-hungry (and maybe expensive) IP blocks.

Odds are that an IP-based pipeline won't fit in the target device that meets your price point, so you will need to throw it all away and then perform a complete lightweight re-implementation - starting from nothing again.