Image synthesizing DSP receiver for Analog FPV video

[AlessandroAU]

Hi everyone,

So, I race quads/drones as a hobby, if you are familiar with the hobby you will know that we are stuck using CVBS video tech from halfway through last century, and it kinda sucks.

I want to develop a receiver technology that intelligently combines multiple video streams, not just RSSI switching but actually using all the information in multiple data-streams to generate a combined image. This is what Iftron's ClearView is advertised to do, but how it really works I will never really know. It seems to excel in a heavy multi-path environment compared to traditional receivers.

ClearView compared to a standard RX: https://youtu.be/WLcRIEPffdQ?t=154

Anyway, I want to invent my own version. So I figure that to even begin experimenting with processing methods I will need some sample data, that is 3x uncompressed PAL/NTSC data-streams, all in sync, all receiving the same VTX signal from 3 separate receivers, under a variety of reception conditions. I am not aware of anything commercial that will do this as the data rate of 3 uncompressed SD streams is already quite significant and I absolutely do not want compression on top of my raw data.

I had some previous experience with the TVP5151 CVBS decoder IC and FPGAs from this video scaler project https://www.rcgroups.com/forums/showthread.php?2993411-Open-source-analog-video-to-HDMI-720p-scaler-based-on-Artix-7-FPGA/page6

So I think it should be possible to combine 3 of those, an FPGA, and a FX3 USB3.0 interface chip. The FX3 is able to handle hundreds of MB/s of sustained transfer to it should not be an issue to get the data out. Reading the documentation it appears it will run in slave fifo mode which is exactly what I want to keep the FPGA logic as minimal as possible.

So, the first goal is to get some data-streams onto a PC so we can start experimenting with different processing methods. I have purchased a FX3 + Artix 7 development board combo and am currently designing a PCB to hold the 3x TVP5151's and 3x RX5808 receiver modules.

Link to FPGA + FX3 board:
https://www.aliexpress.com/item/QMTECH-Cypress-CYUSB3014-USB-3-0-Development-Board-Xilinx-Artix7-FPGA-XC7A35T/1000006618157.html

I expect the image processing will be the most challenging aspect of the project, my first idea at the moment would be to do some sort of spectral entropy analysis on a per line or per block basis, comparing the measurement of each RX and selecting perhaps the best 2 or 3 to average into a final image.

This approach is based on the assumption that an increase in noise due to multi-path fading or interference will increase the instantaneous spectral entropy in the signal regardless of the measured RSSI. With this information you could take the best (or reject the worst) slices from each RX to form into a final image.

Anyway, the purpose of this thread is to get some discussion going!, so if anyone has any ideas or input lets hear them!  I am not a DSP guy, so if you are I'd love to know your ideas.

edit:Images

[AlessandroAU]

I've developed a simulation tool to test different reconstruction algorithms.
Using a static image I simulated some receiver noise/interference in 3 different streams.

A first order attempt, which just measures the noise in each line and slices together the best lines appears to work very well already.

[AlessandroAU]

Got my PCBs and have soldered them up

Still working on the VHDL code for the moment

[JVR]

This looks very interesting. That pic of the soldered board is class, if it was 16:9 I'd have a new background image.