Electronics > Projects, Designs, and Technical Stuff
Digital FPV video for drone racing
<< < (15/28) > >>
ogden:

--- Quote from: IDEngineer on January 21, 2019, 05:54:17 pm ---The thing I want to share with everyone - from the direct hands-on perspective of a top-ranked pilot and the literal embodiment of the target market for such products - is that latency is your single most important criteria.  Many times at competitions my son has sacrificed video quality to reduce latency.

--- End quote ---

Kudos to your son and thanx for valuable input. If latency is paramount then as you say, only solution is out of the box thinking. OP did very good job already. If every millisecond counts, then existing solutions of low cost digital video cameras and existing video compressors do not qualify, unless running at insane framerates like 240 FPS.

If we want to beat analog TV transceiver latencyy using digital tech - we can't unless we increase frame rate. So, it shall be 120Hz or higher. Also whole frame buffering to compress/decompress is way too expensive waste of precious time. Analog TV transmits every scan line w/o buffering, digitally it could be "slices" of the frame (16, 32 or 64px vertically), each individually compressed as separate (jpeg?/h.264?) frame/video and transmitted in separate radio frame to receiver. BTW pilots notice rolling shutter effect running 60FPS *interleaved* analog NTSC?

I believe that approach to the problem shall be iterative. Video compressor/decompressor shall be designed first. Uncongested 20MHz 5.8GHz (802.11ac) WiFi can provide sub-1ms one-way latency, such "radio" is good enough for first prototypes: https://mentor.ieee.org/802.11/dcn/18/11-18-1160-00-0wng-controlling-latency-in-802-11.pptx
Kilrah:

--- Quote from: IDEngineer on January 21, 2019, 05:54:17 pm ---Video problems are the NUMBER ONE ISSUE in this sport.

--- End quote ---

Frankly I'm surprised contests and races even exist already. I've been doing FPV for 16 years, and 10 years back or so we were discussing setting such stuff up and just concluded that video was just too poor for that to provide a decent experience and amount of enjoyment both for pilots and public and ensure fairness between competitors.

It's still just as poor and now demonstrably so, but it seems some are surprisingly somewhat OK with the compromise and ended up doing it anyway...
IDEngineer:

--- Quote from: ogden on January 21, 2019, 07:50:56 pm ---If every millisecond counts, then existing solutions of low cost digital video cameras and existing video compressors do not qualify, unless running at insane framerates like 240 FPS.
--- End quote ---
Frame rate isn't necessarily related to latency. 30FPS, or even 24FPS, is a sufficient frame rate as long as the end-to-end latency is low enough.

A few more thoughts:

* Analog video is fine, from the perspective of the pilots. There's no inherent advantage to going digital, or even to higher resolutions. They're happy with 480 lines. Sure more is better, but I promise they'll give up resolution to get low latency and reliability (no dropouts).

* Next to latency, dirtbag @%$##$%ing other pilots plugging in their quads in the pits on the same FREAKING channel as someone in an active heat is the next biggest problem. Despite rules and penalties, this happens ALL THE TIME even at international competitions. Happened to my son during his very first qualifying heat in Shenzhen, in fact... he was doing great and then suddenly he's looking at some guy's shoes in the pit area. His quad crashed into a gate and he was taken out of that qualifier, and due to the scheduling they didn't offer reflies at this event, so his total practice time on the track before the actual races was reduced due to some other jerk's stupidity. (We tracked down who it was using recorded video and got him disqualified, but that didn't help TeamUSA.)

This problem is almost always some pilot believing "It won't matter for just a quick second". They drop their transmitter power to "pit mode" (generally a couple of mW, whereas most racing is at 25mW these days), remove their transmitting antenna (never mind the SWR mismatch), etc. and "plug in for just a quick test". But at these speeds it only takes a moment of zero visibility to crash into something, and since they've dropped to 25mW to reduce inter-pilot interference the receivers are that much more sensitive and can pick up one of these "quick tests" in the nearby pits sufficient to corrupt or entirely override video coming from a quad that's hundreds of yards away on the other side of the track. Yes, we use directional antennas (some really good ones, in fact) but it's amazing how often the offending &!#$&%&!'er must be in one of the side lobes of the pattern and gets picked up just fine.

THIS is the reason we are considering going to digital video. Not because we need better resolution, but because we need to be able to reject interference from other people on the same analog frequency. The uplinks resolved this a few years ago by going digital and using a token-based system where you "pair" a given transmitter (remote control) to the aircraft's receiver. This system works quite well and they can have dozens of pilots flying "line of sight" (not FPV) in real time without incident. But those are relatively low bandwidth comms compared to (even low-res analog) video.

We've thought about doing some sort of cellphone-like TDMA system, but up in the 5.x GHz ISM band where FPV video lives, so that we can successfully accommodate "plug-ins" like I described. It will take some sort of system like that to fix this, because you literally don't know and can't prevent when some stupid, idiotic, self-serving jerk will plug in during a race and take out a pilot by generating an analog signal or a data stream right on top of what the pilot SHOULD be receiving.

* Another issue, though less important than solving the above, is dropouts. FPV pilots scan eBay for truly analog mini-monitors that display snow instead of a blue screen when no signal is being received. All FPV goggles are analog too. Why? Because signal strength varies all over the place during a race. The antennas are constantly changing orientation to each other, the distances (and thus propagation loss) change wildly, etc. Digital video has the incredibly annoying "feature" of simply turning off when the signal strength goes below some threshold. But analog does not suffer from that... your image quality may degrade, you may lose color (due to loss of color burst on each line) and fall back to B&W for a moment, but the key is graceful degradation rather than brick-wall failure. Once a digital video system "loses sync" it can take several seconds to resume operation. That's an eternity. Imagine driving on the freeway inside a curving tunnel at 80 MPH and someone covers your eyes... seconds feel like years, even a couple of seconds can spell disaster. Far better to gracefully degrade like analog video, from loss of clarity to loss of color to higher S/N ratio (snow) because the human brain and vision system are quite good at interpolating in a noisy environment. But they can't interpolate from a blue screen!

For this reason, my son and I have been toying with employing DSP correlation in a multichannel diversity receiver. There are serious volume, mass, and power limitations on the aircraft themselves, but the ground stations are basically unlimited so doing lots of processing on the ground is totally feasible. (Correlation will not address the issue of bat-freaking braindead idiots transmitting on the same channel so that problem would still have to be addressed.)

* Whatever the front end, the output signal would be best as baseband NTSC. The entire industry is tooled up to support that, and the race operators (MultiGP, FAI, etc.) will resist anything that isn't plug compatible. Why do they care? Because they have live judges dedicated to each pilot position, plus they record all video streams, in case someone files a protest. All that infrastructure is NTSC. If a pilot arrives and he cannot provide an NTSC signal to show his FPV, at best he will lose any protests (for lack of evidence) and at worst he will be prohibited from racing.
IDEngineer:

--- Quote from: Kilrah on January 21, 2019, 08:04:45 pm ---Frankly I'm surprised contests and races even exist already. I've been doing FPV for 16 years, and 10 years back or so we were discussing setting such stuff up and just concluded that video was just too poor for that to provide a decent experience and amount of enjoyment both for pilots and public and ensure fairness between competitors.
--- End quote ---

It's big, and getting bigger FAST. Drone racing is already covered on ESPN. It has been on 60 Minutes. It has enormous corporate sponsors (think Red Bull, Pepsi, etc.). My 16YO son has four corporate sponsors and was courted aggressively while in Shenzhen by local electronics manufacturers who treated him like a rock star. They were hunting him down at the venue, calling for him by name over the crowds. Two of them took us on whole-day tours of their R&D and manufacturing facilities, with the senior management (including CEO's) taking us to very fancy, very authentic Chinese lunches, just to spend time with the TeamUSA pilots and get their input on current and future drone racing products and offer them sponsorships.

If they smell opportunity like that, you know this industry is going places. It's one of the reasons we're talking about improved video systems (and that's just one of the product ideas we have). There's real money moving around and some really big players in the electronics industry are starting to take notice.
TheSteve:
I think you're in for a heck of challenge to successfully make a digital system that can replace the analog ones.
Low power, low latency, instant fade recovery, interference rejection etc. As you mentioned the brain does an amazing job filtering noise and working through the signal fades.

In terms of other people turning a transmitter on in the pits, this is a problem that has been faced by RC since it began. Before 2.4 GHz is was routine for big RC car races to have a transmitter impound. You got your transmitter back just before your heat. Same thing in the RC airplane/helicopter world - frequency pins allowing you and only you to use to use a specific channel and of course transmitter impounds. It won't prevent 100% of problems but it works quite well and is what can be done again.

Long term I see FPV racing ending up about the same as battle bots. It will continue but popularity on TV will suffer some. I love RC but the issue I have with FPV is that we could just simulate the entire thing like a video game.

btw, if you've raced at major drone racing events then odds are very good you've used hardware I've built!
Navigation
Message Index
Next page
Previous page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod