Digital FPV video for drone racing

[Marco]

I would start by giving the design of your receivers a serious look, proper filters and high IMD3 amplifiers and mixers would seem to be a good starting point.

A massive undertaking. If I really wanted an alternative to the RTC6715 based modules I'd just pay TI the eye watering price for the LMX8410L and design around that ... the problem is hard enough as is and it's not like they sell other necessary parts for a more discrete design for reasonable prices either, a synthesizer alone will run you the price of a cheap RTC6715 module.

Having the receivers close to the course is NOT always the smart play, because the inverse square law then puts a drone close to your rx aerials as having a VASTLY stronger signal then one on the other side of the course, where if your aerials are further away the two signals are much closer in (lower) level, level you can make up with antenna gain, blocking DR not so much.

Nice thing about the cheap receiver modules is that you can do a diversity receiver by simply having lots of modules and switching the IF output.

You might be able to do interesting things with multiple receivers and sneaking training sequences into the VBI to allow your system to calculate the impulse response of the channel on a frame by frame basis.

I doubt frame by frame is fast enough if you try to do channel equalization.

[ogden]

I will try my best to explain the process the best I can, but if you are still confused, check out https://en.wikipedia.org/wiki/JPEG , It's a great article.

https://en.wikipedia.org/wiki/Motion_JPEG  Is even greater article.

I am afraid that you underestimate complexity of digital video compression & transmission. You may want to reconsider and check what you can make out of existing stuff like digital MJPEG cameras capable of 640X480 MJPEG@120fps and 5.8GHz WiFi transceivers.

Camera I am talking about is something like this: https://www.google.com/search?q=USBFHD01M

Those usually are using popular MJPEG USB chip: https://www.realtek.com/en/products/computer-peripheral-ics/item/rts5822

[StillTrying]

After looking at some of the small fast quad FVP videos I don't think this is solvable by making the video digital. Even when using 2 receiver diversity BOTH receivers very often drop out at the same time, often for many frames.
I think some improvements would be to get the TX antenna further away from the body of the quad, which is not possible on a racing quad, or to have quite a few meters between 2 diversity receiver antennas, which isn't very practicable either.
Even with digital video I think diversity that works is going to be essential for any improvement.

[TheDane]

I am wondering why you are trying to re-invent something that already exists.
Try google UDP FPV Video, and you'll see a lot of projects already working with low latency and digital video

https://www.google.com/search?q=udp+fpv+video

[Kilrah]

Do you have the space/power for some low power SBC (i.MX6ULL based?). The amount of effort needed to experiment with video coding algorithms will be a lot less than with a FPGA.

There is nothing inherent in digital which causes latency, not even with motion compensation.

Nothing "inherent", but it's still percieved as such (and the case in practice with anything one tries stuff with) becasue 99.9% of implementations don't care about minimizing it or systems have "something" that increases it needlessly. That includes hardware encoders in most SoCs. And they all use standards that are not appropriate for that use as is.

For FPV what's crucial is:
- low latency
- as much as possible no breakups at all
- ultra fast resync if there's one anyway
- immunity to noise, not in the sense of keeping a clean output but in the sense that the garbled output is still somewhat recognisable, if a couple of frames get garbled so bad that your image essentially nothing more than 9 "color blobs" aka huge pixels that's okay and much preferable to a breakup
- interference should corrupt as little as possible of the image. Many systems send a frame as one packet or 2 and have no sync within the frame, so if one packet can't get recovered by error correction you lose a significant portion of the frame or possibly the entirety or all the rest of it after the corruption because once good data starts coming again the receiver has no idea where it should go in the image. Better to send a frame as 1000 small packets, and include coordinates in them so that the receiver can place good packets at the right place.

Problem is, most compression methods achieve their results by working on a large dataset, and we precisely want them as small as possible.

I was thinking of streaming stuff in a way similar to progressive JPEG, aka you stream the same frame multiple times with increasing "resolutions".

[Kilrah]

After looking at some of the small fast quad FVP videos I don't think this is solvable by making the video digital. Even when using 2 receiver diversity BOTH receivers very often drop out at the same time, often for many frames.

Sensitivity of an analog receiver is really low. Digital with proper modulation/error correction should be capable of doing quite a bit better, but whether that's possible with existing analog hardware is questionable.

I am wondering why you are trying to re-invent something that already exists.

There's a dozen or so attempts at it around, but none of them is quite to the point of really being usable, mostly because they all try to stay within the bounds of existing stuff with minor modifications instead of starting something from scratch that is really developed for the specific purpose. Hence gotta keep trying to approach the problem differently, and yes it will likely involve an FPGA on each side... even the display solution needs to be somewhat custom.

[ogden]

Sensitivity of an analog receiver is really low. Digital with proper modulation/error correction should be capable of doing quite a bit better, but whether that's possible with existing analog hardware is questionable.

Even if miracle happens and one can find analog video transceiver with phase performance good enough to run better than BPSK or 2FSK/4FSK modulation, it anyway would be insanity to design modem for analog video 5.8GHz transceiver, because excellent MIMO 5.8GHz digital transceivers exist and they are widely used:

https://wikidevi.com/wiki/Atheros#AR9200.2FAR9500_series

[Marco]

whether that's possible with existing analog hardware is questionable.

Direct FM modulation isn't that bad. The only real problem apart from the potential lack quality of the mixers/LNAs is the way the cheap receiver syncs to the carrier (the VCO gets phase aligned with the carrier). With an IQ demodulator you can handle phase misalignment digitally and fast, now if it loses sync with the carrier you're simply screwed for a bit. Could be solved with a better receiver, but as I said ... kind of a big project.

[Marco]

because excellent MIMO 5.8GHz digital transceivers exist

They are very polite compared to the analogue video transmitters. Being impolite has advantages for robustness, ignoring tragedy of the commons that is.

[ogden]

because excellent MIMO 5.8GHz digital transceivers exist

They are very polite compared to the analogue video transmitters. Being impolite has advantages for robustness, ignoring tragedy of the commons that is.

   Could you explain what you mean?

[Marco]

They don't care if something else is transmitting on the band, they'll just try to shout over it ... since your transmitter is close to your receiver that's a shouting match you usually win.

[TheDane]

There are 480 scan lines (not including overscan) in a 480p video signal.
(1/480)*16 = 0.0333...
As it will take a little less than 16 scan lines to transfer all the data (for those 16 scan lines), and as all of the data is needed to re-construct those 16 scan lines, it will have a minimum of a little less than 32 scan lines of latency.
(1/480)*32 = 0.0666...
If we take into account that there are 60 frames per second:
((1/480)*32)/60 = 0.001111... seconds of latency
that is about 1.111... Milliseconds of latency!!!

Running some numbers:
50 [km/hr] = 50.000/3600 [m/s] = 13.89 [m/s] - breakneck drone speed?
30 [Hz] = 33.33 [mS] - progressive picture frame update rate
60 [Hz] = 16.67 [mS]
100 [Hz] = 10.00 [mS]
240 [Hz] = 4.17 [mS]

Distance the drone is moved pr. picture frame update at 50km/hr, if it can fly that fast:
13.89 [m/s]/30 [Hz] = 46.3 cm
13.89 [m/s]/60 [Hz] = 23.15 cm
13.89 [m/s]/100 [Hz] = 13.89 cm
13.89 [m/s]/240 [Hz] = 5.79 cm

Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
 Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
  Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
   Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
    Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Which is why motion-pictures@24 [Hz] are/were such a big thing - it is moving a frame, pausing and illuminating, moving a frame - so no motion sickness is induced
Most people will not notice the flickering on a big screen@24[Hz] - but it is not gaming. Scenes tend to be relatively slow moving and the director is directing your attention.

You are talking about 60 [Hz] image update rate = 16.67 [mS] image update rate, and it seems you worry about 1.xxx [mS] latency and partial scan updates.
An open WiFi network should have ping rates in the 1.xxx [mS] range, and I presume you don't need encryption - your competitors won't be looking on your screen trying to cheat like it was a game of CS:GO 
Another benefit is having the option of using a range extender on your network - ensuring packets are always received. No more drop-outs, though a bit of network setup is required to sort out duplicate frames. This is all about drone flying, not video jamming each other - right?

Another big requirement is your screen/viewing equipment. Does it support 60 [Hz] framerates, and even higher - without blurring the image itself?

A homemade system sounds great, expensive and quite heavy!
WiFi dongles and SBC's are easy to obtain and replace (and have spares of) in case of crashes, and it won't burn a hole in your wallet. Most are even CE/FCC certified 

Looking forward to hearing more about your interesting project.

[radar_macgyver]

Nice thing about the cheap receiver modules is that you can do a diversity receiver by simply having lots of modules and switching the IF output.

Or even by switching the video. There are some commercial products out there which do this already, and they work decently well to avoid dropouts due to multipath. They don't do anything to avoid frequency overlap, though, which was one of the problems the OP was attempting to solve.

... I'd just pay TI the eye watering price for the LMX8410L and design around that ...

OT, but have you done any tests on that part yet? I was eyeing it for a new design and some of the specs (like phase noise) almost look too good to be true, while others (19 dB noise figure) left me scratching my head. The mixers' spurious responses (other than 2x2 and 3x3) are not characterized in the data sheet either.

[Marco]

Even if miracle happens and one can find analog video transceiver with phase performance good enough to run better than BPSK or 2FSK/4FSK modulation

It's good enough to see intensity grading of a video image, which is more than 4 levels, so you can do a hell of a lot better than 4FSK ... you'd probably drive it with a 8 bit DAC and since it's a FM modulator that is equivalent to 256-DPSK, you'd probably not use all those bits but 4FSK is needlessly pessimistic.

I was wrong about the cheap receiver syncing to the signal, it doesn't bring it to baseband DUH.

I was thinking of streaming stuff in a way similar to progressive JPEG, aka you stream the same frame multiple times with increasing "resolutions".

In and of itself that doesn't really help any, unless you encode the lower resolutions with more FEC.

[dmills]

Thing is a 24 FPS cinema projector has a 48Hz flicker rate due to the two bladed shutter (Actually, some had a three bladed shutter for a 72Hz flicker rate!)...

TECO might be worth a look, it is a wavelet transform codec aimed at UHD baseband (Because 10Gb/s is not enough for 4k60 4:2:2, annoyingly you need 12Gb, which is boring given ethernet standards), it is low latency (as little as a single video line) and capable of up to about 10:1 data reduction.

That would get your ~100Mb/s 8 bit 4:2:0 down to something that could be coded in QAM8 or 16 with a reasonable amount of FEC.

I still think your receivers are the low hanging fruit, and I would absolutely start with some serious spatial and polarisation diversity probably with a pilot tone on the audio channel from the transmitter that my processing can use to assess quality. A few 20M or so cat 6 cables to carry the baseband back from the receivers and to carry power to the receivers, some processing magic (Fast ADCs, fpga with some sdram, video DAC or HDMI output) to dynamically equalise the path delays, and an alpha blend to combine the video signals based on signal quality indications, job done.

Incidentally, if combining multiple noisy signals, pick the darkest pixel, not the brightest, dark pixels due to noise are MUCH less of a problem then bright ones, which is why PAL transmitters had sync be full amplitude and white be minimum amplitude.

Regards, Dan.

[Marco]

OT, but have you done any tests on that part yet?

Nah, just did a quick search for integrated receivers with synthesizer and mixer from the usual suspect and that's what was out there.

[Marco]

I still think your receivers are the low hanging fruit, and I would absolutely start with some serious spatial and polarisation diversity probably with a pilot tone on the audio channel from the transmitter that my processing can use to assess quality. A few 20M or so cat 6 cables to carry the baseband back from the receivers and to carry power to the receivers, some processing magic (Fast ADCs, fpga with some sdram, video DAC or HDMI output) to dynamically equalise the path delays, and an alpha blend to combine the video signals based on signal quality indications, job done.

I don't think the race organizers will take kindly to the mess.

You don't need multiple receivers to equalize a channel, you just need the impulse response.

[ogden]

They don't care if something else is transmitting on the band, they'll just try to shout over it ... since your transmitter is close to your receiver that's a shouting match you usually win.

Why would anyone shall [digitally] transmit on frequencies allocated for analog video! There's always option to transmit on legal WiFi channel and it is even advised to do so.

[dmills]

I don't think the race organizers will take kindly to the mess.
You don't need multiple receivers to equalize a channel, you just need the impulse response.

But the channel you are trying to equalise is NOT LTI, because it is the FM channel you need to equalise and NOT the baseband, and there is a mess of bessel functions involved in mapping one to the other.
The impulse response only helps to the extent the transmission channel is linear and noise free which yours is not.

That TI part is in some ways overkill, if you make the first IF reasonably high (500Mhz - 1GHz??) then you don't need an I/Q mixer, because you can use front end filtering to get your image rejection, particularly if you use a high side LO.
Make the first downconverter LO fixed so you can use a multiplier from a good quality low phase noise rock, rather then a VCO and do all the selectivity work at the first IF. Minicircuits probably have a diode ring mixer that would work for this (Remember to terminate the IF power with a diplexer if you want good IMD3), and I bet one of the usual suspects have mmics for providing the considerable LO power needed by a high level mixer.

As to the organisers, you might be surprised, especially if you could offer them say everyone's baseband to put up on a big screen for the audience?  That TI chip has the IF bandwidth to be able to bring a dozen 30Mhz wide FM signals (FM occupied bandwidth is always much wider then baseband for obvious reasons) down to where you can decode them SDR style and having all the FPV video to stuff into a switcher and then onto a big screen might be a very attractive offering, more so if you can also offer all the teams better realtime video then they have from their own kit.

Half a dozen small boxes on short photographic tripods scattered around the course with cat 6 back to a little card cage with the baseband processing in it, then a BNC to each team and a bundle to the organisers?

Just a thought.

Regards, Dan.

[Marco]

That TI part is in some ways overkill

Of course it's overkill, but everything at that frequency is boutique or completely application specific. Functionality and price have only a tenuous connection, you don't pay much more for the integration and it makes your life a little easier. There's not a lot of integrated solutions AFAICS.

[dmills]

The nice thing about microwave is that a lot of the RF is just shapes on a bit of Duroid or Isola.

No need for much integration in the front end, do something like a printed interdigital filter followed by some gain (MMIC with 1/4 wave line for bias) and a diode mixer, then some more printed filter... LO multiplier chain can be not so very different with some MMIC to provide the power to the mixer, start with something low phase noise like a driscoll oscillator or such. Print the aerials on the back of the board. 

I figure a 5Ghz to 1Ghz transverter is only a handful of parts on a suitable board, and that none of them need to be highly integrated. Once you are down to a GHz the field of suitable mixers and demodulators opens right up.

TI, Analogue and Linear make good money by taking the design out of RF projects, but that is not the only way to roll.

Regards, Dan.

[Marco]

You'll still need a ~5 GHz synthesizer, already a boutique part to start with.

[hexahedron]

Ok, update time! I'm seeing a LOT of discussion about WiFi, and I would like to talk about that.
For a minute, let's imagine a world where wifi is legal for drone racing, and that most pilots use it. There is another competition I would like to talk about as an example. FTC. FTC (first tech challenge) is a competitive robotics program that uses 2.4Ghz wifi for all communications with the robot. Now, what does an FTC competition look like, purely from the perspective of the communication protocol? Well, firstly, there are always at least 2 people with specialized wifi scanners finding anyone that has wifi enabled on a device not used for the robot control system. When these inspectors find said people, they are asked to turn off their wifi or leave. This is already not a good sign. Before every match, each team (4 in total) are assigned a wifi band, and they will only have their wifi turned on, on that band for the duration of the match. During the match, the 2 devices (controller and robot) are never more than 15 feet away from the each other. Even with this extremely well organized and controlled system, nearly 25% of the matches have some sort of disconnection issue related to wifi. Here's why.
As you can see in this image, out of the 14 wifi bands available, only 4 do not overlap, and even when an organized system is in place to make sure the 4 contestants are on these 4 non-overlapping bands, there are STILL connection issues (yes, all the systems use WiFi direct).
Now take a moment to imagine trying to have 8 direct wifi connections going on at the same time, where the connection distance is often very large (depending on the track). Need I say more?

[radar_macgyver]

I meant 802.11a where the chances of overlapping bands is much less of a problem.

[hexahedron]

So, now that we know that we can't use 2.4Ghz wifi for this project, let's look at what we can use. Firstly, I would like you to recall the target audience for this product, MultiGP drone racers. Here is a link to the official rules: https://docs.google.com/document/d/16UYizESQ2ydsoO81ipasLb6IVkmiJKcattWiLboVm9M/edit#
I would like you to turn your attention to pages 8 and 9. Page 8 tells us that

"There are NO spec requirements on the following components:"
"Camera (Example: Runcam, Foxeer)"

page 9 tells us that:

"When constructing your MultiGP Spec Class Racer, you may pick one (1) component from each of the following categories:"

"3.2.3 VTX" (video transmitter)
"TBS UNIFY HV RACE"
"IRC TRAMP HV (with or without NFC board)"

Now, from that information, I hope we can all conclude that any video transmitter other than the 2 listed is out of the question.