EEVblog Electronics Community Forum
Electronics => Projects, Designs, and Technical Stuff => Topic started by: hexahedron on January 18, 2019, 07:16:33 pm
-
Hey all, first time poster here, I have a pretty cool project I've been thinking about for a few years. I started working on this project a few months ago, and I'd like to share my progress so far. If I'm posting in the wrong area, or if you have any other problems, please let me know! I don't want a bad rep right off the bat!
So, what is this project? Well, currently, all competitive drone racers use an "fpv" (first person view) system to pilot their drone at breakneck speeds. This system involves an all-analog system, based on analog television broadcast standards. This system uses the 5.8Ghz band, and when as few as 4 pilots are flying at once (there can be up to 8 in a race!) the signals start to interfere with each other, resulting in poor visibility. In many competitions, the pilots are flying with a mostly black and white signal that is riddled with static. My project intends to solve this problem.
The plan is to create a system that will compress a 480p, 60fps video stream into a signal that will fit within a bandwidth of 6Mhz. Why 6Mhz? MultiGP (largest drone racing thing) only allows for a select few video transmitters to be used for a drone to be legal. As these transmitters are designed for an NTSC video signal input, they all have low-pass filters on the video input, which will kill any higher frequencies (correct me if I'm wrong). MultiGP does NOT restrict the camera that can be used however, which means that my system should be legal.
How can we achieve compressing the video stream into that bandwidth constriction? The answer is that it isn't easy. To start, instead of using a 1 bit data stream, we can use 3 bits (8 values) for our data stream. This triples our data rate from 6Mbits/sec to 18Mbits/sec. Let's get some math out of the way. 640*480*60 is ~ 18.5Mpixels/sec, meaning that for every pixel in the video stream, we can only transmit less than 1 bit of information without going over our limit. I'll get into exactly how this can be accomplished in my next post, but for now I want to see how this community responds to this post.
-
I'm a amateur radio operator. Could you please design your rig so emissions stay clear (60 dB down) of the 5.7 Ghz weak signal calling frequencies. Our emissions are narrow FM voice and SSB Voice at these frequencies, and some of our operators are doing good if they are within 1 Mhz of center.
ITU Regions:
5,668.2 MHz Region 1 Calling Frequency 1[3]
5,760.1 MHz Region 2 Calling Frequency[4]
5,760.2 MHz Region 1 Calling Frequency 2[3]
Thanks....
Steve
-
Will do. Most people using an fpv link don't go over 25mw
-
Well, I guess almost no response is better than a negative one, moving on!
This post will be about the concepts of how this will work going forward, and the next will be about my progress so far. Feel free to skip this post if you are inclined to do so.
So, how is it possible to compress the video stream in such as way that each pixel represents less than a single bit? The answer is found everywhere on the internet, in the form of jpeg compression! I will try my best to explain the process the best I can, but if you are still confused, check out https://en.wikipedia.org/wiki/JPEG , It's a great article.
For now, let's assume we just have an image we need to compress. The first step is to convert the color-space from RGB to YCbCr. This will separate the brightness from the color of the image. Why do this? Our eyes are not as good at sensing changes in brightness as they are in changes in color. This allows us to reduce the resolution of the 2 color channels in the image, with no perceptible difference. In our example, we will reduce the resolution by a factor of 2 in both the X and Y dimensions. Next, we will split each color channel into 8*8 pixel blocks. Then we will apply the Discrete Cosine Transform (DCT) algorithm to each 8*8 block. Explaining what DCT does is very difficult, I would recommend checking out https://en.wikipedia.org/wiki/Discrete_cosine_transform . In essence, we will separate the 8*8 block into another 8*8 block where the different frequencies of the image are separated, where the top left is the low frequency, and the bottom right is high frequency. Why do this step? Well, our eyes are not very good at perceiving high frequency patterns in images as the low frequency ones. This means we can reduce the amount of high frequency data contained in the image without there being a perceivable difference, to an extent. That step is called "quantization" where we divide each block (now DCTified) by a 8*8 quantization table. This table determines the level of compression in the image, and generally is a gradient of numbers where the values increase as they get closer to the lower right corner. This leaves us with 8*8 blocks that have largeish numbers in the upper left corner, and mostly numbers that are less than 1 in the rest of the image. We then round down each 8*8 block, removing all decimal points, resulting in a 8*8 block that mostly consists of zeros. At this point, depending on the quantization table, our image now mostly consists of zeros, and numbers on a range of -127-127 (single byte). The next step is called zig-zag encoding, where we simply convert our 8*8 image into a 64 value long string of bytes from a zig-zag pattern, See image. (https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/JPEG_ZigZag.svg/800px-JPEG_ZigZag.svg.png)
We now can apply a compression algorithm called "run length encoding". https://en.wikipedia.org/wiki/Run-length_encoding This wikipedia article explains it much better than I can, so go read that. And we're done! Now, how effective was this? Well, let's do some math.
The OV7725 camera module (the one I will be using) uses 16 bits to represent each pixel.
Vres hres fps bits
640 x 480 x 60 x 16 = ~ 295Mbits/sec
In order to fit 295Mbits/sec into a 18Mbits/sec datastream, we need to reduce the data by a factor of 16:1. If we look at the JPEG wikipedia article, We can see that the jpeg compression standard "High quality" has a compression ratio of 15:1. I have made some programs that implement this in python, and I can confirm that ratios of 16:1 are indeed possible without being able to tell the difference without a reference image! The next post will be about what I have done to implement this algorithm on an FPGA.
-
Well, I guess almost no response is better than a negative one, moving on!
Well I don't think it can be done, but I hope I'm wrong. :)
Any gap in a digital compressed stream will leave a much bigger hole than in an analogue stream.
-
That is a very good point! From my observations of how noise is introduced into a data signal, it tends to be fairly consistent as a function of how far away the transmitter is. One of my ideas I've had is to have a second data link which tells the camera to further compress the image and reduce the speed at which data is sent. IE: it waits longer in between each time it updates the data stream with new bits. This will allow the ADC on the other end to have more time to sample the incoming data for each bit. As some of my experiments have shown, this reduces the likelihood of errors when reading the data stream.
Additionally, I plan to utilize the audio channel to mark when each pixel block is being sent. This means that if an error is introduced, it should only effect the single pixel block.
-
I think on even a relatively clean signal you'd have to use 25% of the bit rate as error correction, which complicates it a bit. :)
-
compression, speed and lossy medium doesnt work very well, you need only to pick 2 out of that 3. one lost bit means losing the whole frame.
-
Well, Let's hope that signal degradation won't be too much worse than I was imagining.
Here's what I've done so far.
I've interfaced a a fpga with an OV7725 camera module: https://www.youtube.com/watch?v=PGxcpnUV03o (https://www.youtube.com/watch?v=PGxcpnUV03o) (I have fixed the issue shown in the vid)
And I've implemented 1D DCT on a FPGA: https://www.youtube.com/watch?v=8tAj1vDoTEQ (https://www.youtube.com/watch?v=8tAj1vDoTEQ)
Along with A lot of other experimentation before and inbetween those 2 videos.
Today I will be attempting to setup 2D DCT on the fpga while occasionally checking this forum.
EDIT: I have implemented 2D DCT, but it wasn't too interesting, So I wont make a post about it
-
What about run a connectionless Wifi at 2.4GHz in parallel for better reliability? A Raspberry Pi Zero W would be nearly ideal for that.
https://befinitiv.wordpress.com/wifibroadcast-analog-like-transmission-of-live-video-data/
-
To start, instead of using a 1 bit data stream, we can use 3 bits (8 values) for our data stream. This triples our data rate from 6Mbits/sec to 18Mbits/sec.
What do you plan on using for the RF system here out of curiosity? Single carrier? Multi-carrier? FEC? I assume you must have an off-the-shelf module in mind to interface with.
-
The talk about interference and 2.4 GHz is irrelevant, he wants to use transmitters compliant for drone racing which usually take NTSC signals and use them to carry digital video. They are noisy, shouty things ... but that's out of his hands.
6 MHz != 6 Mb/s by the way ... you can stuff a whole lot more data through. How much depends on SNR, intersymbol interference and how you implement error correction.
-
John Adams from Horizon Hobby and Spektrum discusses digital FPV video and the problems it creates in this interview.
https://youtu.be/BbwDRqF6Kb0?t=1294
-
but for now I want to see how this community responds to this post.
Do you have the space/power for some low power SBC (i.MX6ULL based?). The amount of effort needed to experiment with video coding algorithms will be a lot less than with a FPGA. Stuff like training sequence based equalization is also a hell of a lot easier in C than VHDL (I think it's a safe bet that signal strength is going to affect intersymbol interference).
-
I'm thinking that using both the 5GHz transmitter and an off the shelf 2.4GHz transmitter together would yield a more robust signal than either one alone.
-
You'll find the thing that kills you is latency (as mentioned in @Gary350z's video). Flying any quad FPV (let alone for racing) is hard, and even the smallest latency will be quickly noticed by the pilot. One of my buddies was into racing, and he asked if this was possible, I spent a while looking into it but the latency turned out to be the hardest problem to solve. My plan was to use a RPi zero and send the video stream over wifi. You might have better luck if the video compression is done in hardware. Maybe a Zynq, with the PL handling the video encoding and the PS streaming the data over wifi.
-
There is nothing inherent in digital which causes latency, not even with motion compensation.
Unless you consider the number of scanlines needed for a row of blocks to be significant latency.
-
I've been looking into the exact same kind of project. In my case the focus is on range and cost; all existing digital FPV solutions are about HD video, have shitty range, and cost >$500. I did some experiments with JPEG a long time ago and found that for 480p you can get the size down to about 20KB per frame and still get a usable image, which is 4.8Mb/s at 30fps. However video frames are highly redundant and with just a little bit of inter-frame prediction you should be able to do much better. H.264 for example can give you HD 1080p video at 5Mb/s.
For the RF data link I would recommend not going above QPSK because efficiency (bits you can get across per unit energy) rapidly falls as you stuff more bits into a given bandwidth. For a 6MHz channel this means 12Mb/s uncoded rate, and you will ideally want FEC at rate 1/2, which means 6Mb/s maximum actual bit rate. You also need to factor in some amount of protocol overhead (framing etc).
I would not consider wifi because of inherent protocol flaws and bad off-the-shelf implementations (bad listen-before-talk thresholds, etc) that will give you unexplained high latency.
Please do keep us updated because I'm sure your findings will be helpful for my project as well ;)
-
There was a VHDL H.264 encoder but I haven't gotten around to trying it yet: http://hardh264.sourceforge.net/H264-encoder-manual.html (http://hardh264.sourceforge.net/H264-encoder-manual.html)
I would imagine for a simpler implementation doing the JPEG stuff on a soft core might be fast enough, and you can accelerate major bottleneck components like motion estimation in hardware. You would send a full jpeg frame every e.g. 5 frames followed by partial frames (motion vectors & differences).
The other thing I would add is a 6MHz channel is a lot to expect in terms of phase coherence; 6MHz corresponds to a delay spread of <160ns, which means any reflections can corrupt your signal. You can probably predict whether this will be a problem or not by using an analog video transmitter/receiver and looking closely at the received image. If you see any ghosting or shadows then you might run into issues with inter-symbol interference.
-
For the RF data link I would recommend not going above QPSK because efficiency (bits you can get across per unit energy) rapidly falls as you stuff more bits into a given bandwidth.
Don't these 5.8 GHz analogue video transmitters simply take the input (nominally NTSC/PAL) and FM modulate the carrier with it? Trying to get cutesy with carrier coding for the input of an FM modulator doesn't seem much use to me ... just treat it as a 1D channel, nice and simple.
What I would personally try to do is adapt something like the old V.34 channel equalization (except 1D instead of 2D). Every kB or so send a sync+CAZAC pulse to get an accurate equalization filter with FFT, rest of the time use adaptive LMS equalization. Only use 0 values for the sync, use say 16-256 for the data.
-
For 5.8GHz at shorter distances or even indoors and with lots of movement, I don't think circular polarization and up to 4 receiver diversity could be beat by any digital fixing, 5.8GHz is almost radar and reflects off almost everything.
There is nothing inherent in digital which causes latency, not even with motion compensation.
In theory, but sometimes there's around 100ms of latency when a miniature camera is directly analogue connected to a 5in LCD monitor.
-
I don't think circular polarization and up to 4 receiver diversity could be beat by any digital fixing
Analogue can't use channel equalization and compression. The reflections change fast on a human timescale, but do they change fast enough to be a problem on the LMS algorithm's timescale?
Also you could do something in between, use half the bandwidth to just send a PCM image (but with the benefit of channel equalization, so still less ghosting than pure analogue) use the other half to send a compressed refinement layer.
-
WOW! Thanks for all the feedback on my project so far! Apologies for not responding, as I live in the USA I was asleep :P The standard fpv system does indeed just directly fm modulate the 5.8Ghz signal, and has a 9Mhz sub-carrier for audio. I'm seeing a lot of talk about H.264, and while that would indeed compress the video much further, it has a few problems. Firstly and foremost, latency! From what I know, implementing H.264 on the transmitter and receiver end will add at least 1 frame of latency, which is unacceptable! Please correct me If I'm wrong on that. My system at most should add about 0.1 frames of latency to the system, here's why. If you recall from my explanation of how JPEG works, we reduce the color channels resolution by a factor of 2 in each dimension, then split it into 8*8 blocks. This means that 16 scan lines (horizontal row of pixels) are required for compression. My fpga system should be able to compress all the blocks in those 16 scan lines within 1 extra scan line from my calculations. We can then send all of the data for those 16 scan lines in less than 16 more scan lines (the brightness channel is calculated after 8 scan lines, so we start transmitting there). There are 480 scan lines (not including overscan) in a 480p video signal.
(1/480)*16 = 0.0333...
As it will take a little less than 16 scan lines to transfer all the data (for those 16 scan lines), and as all of the data is needed to re-construct those 16 scan lines, it will have a minimum of a little less than 32 scan lines of latency.
(1/480)*32 = 0.0666...
If we take into account that there are 60 frames per second:
((1/480)*32)/60 = 0.001111... seconds of latency
that is about 1.111... Milliseconds of latency!!!
Obviously, computing time will add a little more to that number, but I don't think latency will be an issue.
Again, thanks for the positive feedback and useful discussion! This is the first forum I've found where people actually care about and understand what I'm working on, and I can't tell you how much that means to me!
-
It depends on the encoder. Say X264 can just encode blocks in sync with the scan pattern (they've done development work for game streaming).
-
I would start by giving the design of your receivers a serious look, proper filters and high IMD3 amplifiers and mixers would seem to be a good starting point. Having the receivers close to the course is NOT always the smart play, because the inverse square law then puts a drone close to your rx aerials as having a VASTLY stronger signal then one on the other side of the course, where if your aerials are further away the two signals are much closer in (lower) level, level you can make up with antenna gain, blocking DR not so much.
In terms of image coding, start by defining how much latency is acceptable (as little as possible is NOT a specification), which will tell you how many video lines you have to work with (Don't forget that some cameras and monitors have a nasty habit of introducing a frame or two for their own purposes, for low latency a vidicon (Or image Orthicon) tube and CRT is hard to beat, but possibly a little heavy on an aerial drone!
You might be able to do interesting things with multiple receivers and sneaking training sequences into the VBI to allow your system to calculate the impulse response of the channel on a frame by frame basis. Multipath can for example be detected by looking for AM on the audio sub carrier, which with multiple receivers can be used to lower the 'fitness' of any receiver that has significant multipath.
There are really two parts to the digital approach, data reduction and the data coding for the link, and you should probably provide some sort of back channel so that the coder can be told what the characteristics of the signal at the receivers are and can modify the amount of bandwidth it allocates to FEC to suit, it will then have to communicate with the video data reduction codec to indicate how much bandwidth is available...
One trick that can be useful is to shuffle the data going into the transmitter so that bursts of interference tend to be spread out as a few bit errors per line rather then being a burst that completely wipes out a whole block of data, this is especially useful if the FEC data is added before this is done as it allows reconstruction of even badly corrupted data providing the temporal spread is large compared to the length of the interference burst.
Just some thoughts.
Regards, Dan.
-
I would start by giving the design of your receivers a serious look, proper filters and high IMD3 amplifiers and mixers would seem to be a good starting point.
A massive undertaking. If I really wanted an alternative to the RTC6715 based modules I'd just pay TI the eye watering price for the LMX8410L and design around that ... the problem is hard enough as is and it's not like they sell other necessary parts for a more discrete design for reasonable prices either, a synthesizer alone will run you the price of a cheap RTC6715 module.
Having the receivers close to the course is NOT always the smart play, because the inverse square law then puts a drone close to your rx aerials as having a VASTLY stronger signal then one on the other side of the course, where if your aerials are further away the two signals are much closer in (lower) level, level you can make up with antenna gain, blocking DR not so much.
Nice thing about the cheap receiver modules is that you can do a diversity receiver by simply having lots of modules and switching the IF output.
You might be able to do interesting things with multiple receivers and sneaking training sequences into the VBI to allow your system to calculate the impulse response of the channel on a frame by frame basis.
I doubt frame by frame is fast enough if you try to do channel equalization.
-
I will try my best to explain the process the best I can, but if you are still confused, check out https://en.wikipedia.org/wiki/JPEG (https://en.wikipedia.org/wiki/JPEG) , It's a great article.
https://en.wikipedia.org/wiki/Motion_JPEG (https://en.wikipedia.org/wiki/Motion_JPEG) Is even greater article.
I am afraid that you underestimate complexity of digital video compression & transmission. You may want to reconsider and check what you can make out of existing stuff like digital MJPEG cameras capable of 640X480 MJPEG@120fps and 5.8GHz WiFi transceivers.
Camera I am talking about is something like this: https://www.google.com/search?q=USBFHD01M (https://www.google.com/search?q=USBFHD01M)
Those usually are using popular MJPEG USB chip: https://www.realtek.com/en/products/computer-peripheral-ics/item/rts5822 (https://www.realtek.com/en/products/computer-peripheral-ics/item/rts5822)
-
After looking at some of the small fast quad FVP videos I don't think this is solvable by making the video digital. Even when using 2 receiver diversity BOTH receivers very often drop out at the same time, often for many frames.
I think some improvements would be to get the TX antenna further away from the body of the quad, which is not possible on a racing quad, or to have quite a few meters between 2 diversity receiver antennas, which isn't very practicable either.
Even with digital video I think diversity that works is going to be essential for any improvement.
-
I am wondering why you are trying to re-invent something that already exists.
Try google UDP FPV Video, and you'll see a lot of projects already working with low latency and digital video
https://www.google.com/search?q=udp+fpv+video (https://www.google.com/search?q=udp+fpv+video)
-
Do you have the space/power for some low power SBC (i.MX6ULL based?). The amount of effort needed to experiment with video coding algorithms will be a lot less than with a FPGA.
There is nothing inherent in digital which causes latency, not even with motion compensation.
Nothing "inherent", but it's still percieved as such (and the case in practice with anything one tries stuff with) becasue 99.9% of implementations don't care about minimizing it or systems have "something" that increases it needlessly. That includes hardware encoders in most SoCs. And they all use standards that are not appropriate for that use as is.
For FPV what's crucial is:
- low latency
- as much as possible no breakups at all
- ultra fast resync if there's one anyway
- immunity to noise, not in the sense of keeping a clean output but in the sense that the garbled output is still somewhat recognisable, if a couple of frames get garbled so bad that your image essentially nothing more than 9 "color blobs" aka huge pixels that's okay and much preferable to a breakup
- interference should corrupt as little as possible of the image. Many systems send a frame as one packet or 2 and have no sync within the frame, so if one packet can't get recovered by error correction you lose a significant portion of the frame or possibly the entirety or all the rest of it after the corruption because once good data starts coming again the receiver has no idea where it should go in the image. Better to send a frame as 1000 small packets, and include coordinates in them so that the receiver can place good packets at the right place.
Problem is, most compression methods achieve their results by working on a large dataset, and we precisely want them as small as possible.
I was thinking of streaming stuff in a way similar to progressive JPEG, aka you stream the same frame multiple times with increasing "resolutions".
-
After looking at some of the small fast quad FVP videos I don't think this is solvable by making the video digital. Even when using 2 receiver diversity BOTH receivers very often drop out at the same time, often for many frames.
Sensitivity of an analog receiver is really low. Digital with proper modulation/error correction should be capable of doing quite a bit better, but whether that's possible with existing analog hardware is questionable.
I am wondering why you are trying to re-invent something that already exists.
There's a dozen or so attempts at it around, but none of them is quite to the point of really being usable, mostly because they all try to stay within the bounds of existing stuff with minor modifications instead of starting something from scratch that is really developed for the specific purpose. Hence gotta keep trying to approach the problem differently, and yes it will likely involve an FPGA on each side... even the display solution needs to be somewhat custom.
-
Sensitivity of an analog receiver is really low. Digital with proper modulation/error correction should be capable of doing quite a bit better, but whether that's possible with existing analog hardware is questionable.
Even if miracle happens and one can find analog video transceiver with phase performance good enough to run better than BPSK or 2FSK/4FSK modulation, it anyway would be insanity to design modem for analog video 5.8GHz transceiver, because excellent MIMO 5.8GHz digital transceivers exist and they are widely used:
https://wikidevi.com/wiki/Atheros#AR9200.2FAR9500_series (https://wikidevi.com/wiki/Atheros#AR9200.2FAR9500_series)
-
whether that's possible with existing analog hardware is questionable.
Direct FM modulation isn't that bad. The only real problem apart from the potential lack quality of the mixers/LNAs is the way the cheap receiver syncs to the carrier (the VCO gets phase aligned with the carrier). With an IQ demodulator you can handle phase misalignment digitally and fast, now if it loses sync with the carrier you're simply screwed for a bit. Could be solved with a better receiver, but as I said ... kind of a big project.
-
because excellent MIMO 5.8GHz digital transceivers exist
They are very polite compared to the analogue video transmitters. Being impolite has advantages for robustness, ignoring tragedy of the commons that is.
-
because excellent MIMO 5.8GHz digital transceivers exist
They are very polite compared to the analogue video transmitters. Being impolite has advantages for robustness, ignoring tragedy of the commons that is.
:-// Could you explain what you mean?
-
They don't care if something else is transmitting on the band, they'll just try to shout over it ... since your transmitter is close to your receiver that's a shouting match you usually win.
-
There are 480 scan lines (not including overscan) in a 480p video signal.
(1/480)*16 = 0.0333...
As it will take a little less than 16 scan lines to transfer all the data (for those 16 scan lines), and as all of the data is needed to re-construct those 16 scan lines, it will have a minimum of a little less than 32 scan lines of latency.
(1/480)*32 = 0.0666...
If we take into account that there are 60 frames per second:
((1/480)*32)/60 = 0.001111... seconds of latency
that is about 1.111... Milliseconds of latency!!!
Running some numbers:
50 [km/hr] = 50.000/3600 [m/s] = 13.89 [m/s] - breakneck drone speed?
30 [Hz] = 33.33 [mS] - progressive picture frame update rate
60 [Hz] = 16.67 [mS]
100 [Hz] = 10.00 [mS]
240 [Hz] = 4.17 [mS]
Distance the drone is moved pr. picture frame update at 50km/hr, if it can fly that fast:
13.89 [m/s]/30 [Hz] = 46.3 cm
13.89 [m/s]/60 [Hz] = 23.15 cm
13.89 [m/s]/100 [Hz] = 13.89 cm
13.89 [m/s]/240 [Hz] = 5.79 cm
Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Doing a rolling picture update could very well induce smeering/blurring - confusing your brain
Which is why motion-pictures@24 [Hz] are/were such a big thing - it is moving a frame, pausing and illuminating, moving a frame - so no motion sickness is induced
Most people will not notice the flickering on a big screen@24[Hz] - but it is not gaming. Scenes tend to be relatively slow moving and the director is directing your attention.
You are talking about 60 [Hz] image update rate = 16.67 [mS] image update rate, and it seems you worry about 1.xxx [mS] latency and partial scan updates.
An open WiFi network should have ping rates in the 1.xxx [mS] range, and I presume you don't need encryption - your competitors won't be looking on your screen trying to cheat like it was a game of CS:GO 8)
Another benefit is having the option of using a range extender on your network - ensuring packets are always received. No more drop-outs, though a bit of network setup is required to sort out duplicate frames. This is all about drone flying, not video jamming each other - right?
Another big requirement is your screen/viewing equipment. Does it support 60 [Hz] framerates, and even higher - without blurring the image itself?
A homemade system sounds great, expensive and quite heavy!
WiFi dongles and SBC's are easy to obtain and replace (and have spares of) in case of crashes, and it won't burn a hole in your wallet. Most are even CE/FCC certified ;)
Looking forward to hearing more about your interesting project.
-
Nice thing about the cheap receiver modules is that you can do a diversity receiver by simply having lots of modules and switching the IF output.
Or even by switching the video. There are some commercial products out there which do this already, and they work decently well to avoid dropouts due to multipath. They don't do anything to avoid frequency overlap, though, which was one of the problems the OP was attempting to solve.
... I'd just pay TI the eye watering price for the LMX8410L and design around that ...
OT, but have you done any tests on that part yet? I was eyeing it for a new design and some of the specs (like phase noise) almost look too good to be true, while others (19 dB noise figure) left me scratching my head. The mixers' spurious responses (other than 2x2 and 3x3) are not characterized in the data sheet either.
-
Even if miracle happens and one can find analog video transceiver with phase performance good enough to run better than BPSK or 2FSK/4FSK modulation
It's good enough to see intensity grading of a video image, which is more than 4 levels, so you can do a hell of a lot better than 4FSK ... you'd probably drive it with a 8 bit DAC and since it's a FM modulator that is equivalent to 256-DPSK, you'd probably not use all those bits but 4FSK is needlessly pessimistic.
I was wrong about the cheap receiver syncing to the signal, it doesn't bring it to baseband DUH.
I was thinking of streaming stuff in a way similar to progressive JPEG, aka you stream the same frame multiple times with increasing "resolutions".
In and of itself that doesn't really help any, unless you encode the lower resolutions with more FEC.
-
Thing is a 24 FPS cinema projector has a 48Hz flicker rate due to the two bladed shutter (Actually, some had a three bladed shutter for a 72Hz flicker rate!)...
TECO might be worth a look, it is a wavelet transform codec aimed at UHD baseband (Because 10Gb/s is not enough for 4k60 4:2:2, annoyingly you need 12Gb, which is boring given ethernet standards), it is low latency (as little as a single video line) and capable of up to about 10:1 data reduction.
That would get your ~100Mb/s 8 bit 4:2:0 down to something that could be coded in QAM8 or 16 with a reasonable amount of FEC.
I still think your receivers are the low hanging fruit, and I would absolutely start with some serious spatial and polarisation diversity probably with a pilot tone on the audio channel from the transmitter that my processing can use to assess quality. A few 20M or so cat 6 cables to carry the baseband back from the receivers and to carry power to the receivers, some processing magic (Fast ADCs, fpga with some sdram, video DAC or HDMI output) to dynamically equalise the path delays, and an alpha blend to combine the video signals based on signal quality indications, job done.
Incidentally, if combining multiple noisy signals, pick the darkest pixel, not the brightest, dark pixels due to noise are MUCH less of a problem then bright ones, which is why PAL transmitters had sync be full amplitude and white be minimum amplitude.
Regards, Dan.
-
OT, but have you done any tests on that part yet?
Nah, just did a quick search for integrated receivers with synthesizer and mixer from the usual suspect and that's what was out there.
-
I still think your receivers are the low hanging fruit, and I would absolutely start with some serious spatial and polarisation diversity probably with a pilot tone on the audio channel from the transmitter that my processing can use to assess quality. A few 20M or so cat 6 cables to carry the baseband back from the receivers and to carry power to the receivers, some processing magic (Fast ADCs, fpga with some sdram, video DAC or HDMI output) to dynamically equalise the path delays, and an alpha blend to combine the video signals based on signal quality indications, job done.
I don't think the race organizers will take kindly to the mess.
You don't need multiple receivers to equalize a channel, you just need the impulse response.
-
They don't care if something else is transmitting on the band, they'll just try to shout over it ... since your transmitter is close to your receiver that's a shouting match you usually win.
Why would anyone shall [digitally] transmit on frequencies allocated for analog video! There's always option to transmit on legal WiFi channel and it is even advised to do so.
-
I don't think the race organizers will take kindly to the mess.
You don't need multiple receivers to equalize a channel, you just need the impulse response.
But the channel you are trying to equalise is NOT LTI, because it is the FM channel you need to equalise and NOT the baseband, and there is a mess of bessel functions involved in mapping one to the other.
The impulse response only helps to the extent the transmission channel is linear and noise free which yours is not.
That TI part is in some ways overkill, if you make the first IF reasonably high (500Mhz - 1GHz??) then you don't need an I/Q mixer, because you can use front end filtering to get your image rejection, particularly if you use a high side LO.
Make the first downconverter LO fixed so you can use a multiplier from a good quality low phase noise rock, rather then a VCO and do all the selectivity work at the first IF. Minicircuits probably have a diode ring mixer that would work for this (Remember to terminate the IF power with a diplexer if you want good IMD3), and I bet one of the usual suspects have mmics for providing the considerable LO power needed by a high level mixer.
As to the organisers, you might be surprised, especially if you could offer them say everyone's baseband to put up on a big screen for the audience? That TI chip has the IF bandwidth to be able to bring a dozen 30Mhz wide FM signals (FM occupied bandwidth is always much wider then baseband for obvious reasons) down to where you can decode them SDR style and having all the FPV video to stuff into a switcher and then onto a big screen might be a very attractive offering, more so if you can also offer all the teams better realtime video then they have from their own kit.
Half a dozen small boxes on short photographic tripods scattered around the course with cat 6 back to a little card cage with the baseband processing in it, then a BNC to each team and a bundle to the organisers?
Just a thought.
Regards, Dan.
-
That TI part is in some ways overkill
Of course it's overkill, but everything at that frequency is boutique or completely application specific. Functionality and price have only a tenuous connection, you don't pay much more for the integration and it makes your life a little easier. There's not a lot of integrated solutions AFAICS.
-
The nice thing about microwave is that a lot of the RF is just shapes on a bit of Duroid or Isola.
No need for much integration in the front end, do something like a printed interdigital filter followed by some gain (MMIC with 1/4 wave line for bias) and a diode mixer, then some more printed filter... LO multiplier chain can be not so very different with some MMIC to provide the power to the mixer, start with something low phase noise like a driscoll oscillator or such. Print the aerials on the back of the board.
I figure a 5Ghz to 1Ghz transverter is only a handful of parts on a suitable board, and that none of them need to be highly integrated. Once you are down to a GHz the field of suitable mixers and demodulators opens right up.
TI, Analogue and Linear make good money by taking the design out of RF projects, but that is not the only way to roll.
Regards, Dan.
-
You'll still need a ~5 GHz synthesizer, already a boutique part to start with.
-
Ok, update time! I'm seeing a LOT of discussion about WiFi, and I would like to talk about that.
For a minute, let's imagine a world where wifi is legal for drone racing, and that most pilots use it. There is another competition I would like to talk about as an example. FTC. FTC (first tech challenge) is a competitive robotics program that uses 2.4Ghz wifi for all communications with the robot. Now, what does an FTC competition look like, purely from the perspective of the communication protocol? Well, firstly, there are always at least 2 people with specialized wifi scanners finding anyone that has wifi enabled on a device not used for the robot control system. When these inspectors find said people, they are asked to turn off their wifi or leave. This is already not a good sign. Before every match, each team (4 in total) are assigned a wifi band, and they will only have their wifi turned on, on that band for the duration of the match. During the match, the 2 devices (controller and robot) are never more than 15 feet away from the each other. Even with this extremely well organized and controlled system, nearly 25% of the matches have some sort of disconnection issue related to wifi. Here's why. (https://www1.aerohive.com/wp-content/uploads/wifibacktobasics1.png)
As you can see in this image, out of the 14 wifi bands available, only 4 do not overlap, and even when an organized system is in place to make sure the 4 contestants are on these 4 non-overlapping bands, there are STILL connection issues (yes, all the systems use WiFi direct).
Now take a moment to imagine trying to have 8 direct wifi connections going on at the same time, where the connection distance is often very large (depending on the track). Need I say more?
-
I meant 802.11a where the chances of overlapping bands is much less of a problem.
-
So, now that we know that we can't use 2.4Ghz wifi for this project, let's look at what we can use. Firstly, I would like you to recall the target audience for this product, MultiGP drone racers. Here is a link to the official rules: https://docs.google.com/document/d/16UYizESQ2ydsoO81ipasLb6IVkmiJKcattWiLboVm9M/edit#
I would like you to turn your attention to pages 8 and 9. Page 8 tells us that
"There are NO spec requirements on the following components:"
"Camera (Example: Runcam, Foxeer)"
page 9 tells us that:
"When constructing your MultiGP Spec Class Racer, you may pick one (1) component from each of the following categories:"
"3.2.3 VTX" (video transmitter)
"TBS UNIFY HV RACE"
"IRC TRAMP HV (with or without NFC board)"
Now, from that information, I hope we can all conclude that any video transmitter other than the 2 listed is out of the question.
-
Need I say more?
5.8 GHz
https://en.wikipedia.org/wiki/List_of_WLAN_channels#5_GHz_(802.11a/h/j/n/ac/ax)
Pick and choose your channel(s) - it is ISM.
Talk with the organizers, if they do not understand technology progress - and yes, 2.4GHz is crowded, and not suitable for such things
Edit - More detail; (Pick and choose - it is ISM.)
-
"3.2.3 VTX" (video transmitter)
"TBS UNIFY HV RACE"
"IRC TRAMP HV (with or without NFC board)"
Now, from that information, I hope we can all conclude that any video transmitter other than the 2 listed is out of the question.
3.1.4 Exempted Components:
There are NO spec requirements on the following components:
Flight Controller (Example: Motolabs Cyclone, Revolt)
Flight Controller Software (Example: RaceFlight,BetaFlight)
TX (Example: X4R, Crossfire)
Radio (Example: Taranis, DX8)
Camera (Example: Runcam, Foxeer)
Flight Controller - an additional SBC, raspberry pi zero? (it says COMPONENTS)
TX/Radio - 5.8GHz WiFi Dongle
So, do you need to be limited by what is NOT allowed more than 1 unit of?
-
As you can see in this image, out of the 14 wifi bands available, only 4 do not overlap
5.8Ghz WiFi have way more non-overlapping 20MHz channels. Obviously 2.4 GHz is too overcrowded, it's not even worth to waste time talking about it. Also using Atheros chipset does not necessarily mean that you shall use all it's features, implement "WiFi as it is".
-
50 [km/hr] = 50.000/3600 [m/s] = 13.89 [m/s] - breakneck drone speed?
You can multiply that by 3. But it's not straight line speed that matters, that's a piece of cake. It's feedback in fast maneuvers you do when dodging obstacles.
Why would anyone shall [digitally] transmit on frequencies allocated for analog video!
These analog video transmitters send in the ISM band in the middle of everything (where most stuff is digital, and where digital stuff is allowed 10x more power!), they don't have reserved bands.
"When constructing your MultiGP Spec Class Racer, you may pick one (1) component from each of the following categories:"
"3.2.3 VTX" (video transmitter)
"TBS UNIFY HV RACE"
"IRC TRAMP HV (with or without NFC board)"
Now, from that information, I hope we can all conclude that any video transmitter other than the 2 listed is out of the question.
For the OP's initial idea yes, but I can assure you that if you can make a better system that is compliant with the RF regulations of all countries where these events take place they'll be more than happy to add your product to the list.
-
50 [km/hr] = 50.000/3600 [m/s] = 13.89 [m/s] - breakneck drone speed?
You can multiply that by 3. But it's not straight line speed that matters, that's a piece of cake. It's feedback in fast maneuvers you do when dodging obstacles.
There seems to be 2 MultiGP racing variants: Indoor and outdoor.
On the outdoor track(s), it looks like that it is more about remembering the course and when/where to turn - and then turn up the throttle to max.
The indoor track(s?) goes a lot slower, and needs to be more precise. Am I wrong?
https://www.multigp.com/leaderboards/ (https://www.multigp.com/leaderboards/)
- lots of YouTube videos, and ample opportunity to get airsick in front of your screen :popcorn:
-
You'll still need a ~5 GHz synthesizer, already a boutique part to start with.
Not really, use a crystal and multiplier chain instead of a synth and then do the selectivity at a lower frequency.
A chain of clippers and third harmonic filters is old school possibly but it usually has better phase noise then a synth, and the required filters can be printed onto the board at these frequencies, hell a nice quick snap or impatt diode or such and multiplying directly from 1.2GHz to 6 is not out of the question, I bet skyworks or microsemi or such have something that would work.
Once you have shifted the whole lot down to say 1GHz, the field opens way up in terms of how to do the selectivity.
Regards, Dan.
-
But the channel you are trying to equalise is NOT LTI, because it is the FM channel you need to equalise and NOT the baseband, and there is a mess of bessel functions involved in mapping one to the other.
It's been aeons since I had the mental faculties to do the math on this, but are you quite sure a constant multipath response won't approximately result in a linear response in the baseband as long as the carrier frequency is much higher than the bandwidth? The math in these course notes (https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-450-principles-of-digital-communications-i-fall-2006/lecture-notes/book_9.pdf#page=20) seems to suggest so, but as I said ... it's been aeons.
Intuitively if the multipath response caused nonlinear effects in baseband, baseband OFDM wouldn't work either.
The mapping of the two responses isn't really relevant, you measure it in baseband and you correct it in baseband.
-
Ok, so I've been looking into inter-frame video compression standards, and I feel like I need some help. I've found http://www.openh264.org/ (http://www.openh264.org/) , an open source .h264 implementation, and after going to the github, I got a bit stuck. You see, I've taught myself everything I know about programming, but as tends to happen when you teach yourself, you end up with gaps in your knowledge. I don't know where the hell to start trying to learn how any of the code in the github repository works. How does one even start reverse engineering such an incredibly large project? Does anyone know if there is some paper or article that goes into the individual steps in order that h.264 goes through? I know I could just find some implementation for an fpga and copy/paste, but I actually want to know how it works.
EDIT: would it be worth it to just try an implement MPEG1/2?
-
You don't, H.264 is not designed for high loss channels ... it's essentially useless to you.
Trying to do all this with a FPGA is a sure recipe for never succeeding IMO.
The output of the cheap receiver modules is actually not at baseband, you still need to downconvert it from 480 MHz before you can even start decoding it. Why not try to build a system which can sync the transmitter and receiver and sends an image as uncompressed PCM YUV before even worrying about compression?
-
Trying to do all this with a FPGA is a sure recipe for never succeeding IMO.
The more I read into compression standards, the more I have to agree with this sadly.
I guess the next logical step will be to purchase a beefy microcontroller of some sort. Any recommendations?
-
Why not try to build a system which can sync the transmitter and receiver and sends an image as uncompressed PCM YUV before even worrying about compression?
Why not use video camera that provides compressed digital video output?
Any inter-frame compressor like h.264 or MPEG will barely help for FPV drone video. FPV video stream is not like hollywood movie /w fixed camera, every next frame differ too much.
-
Any inter-frame compressor like h.264 or MPEG will barely help for FPV drone video.
H.264 720p will go a bit fuzzy during motion at 2 Mbps, MJPEG will just look like utter trash non stop.
-
Any inter-frame compressor like h.264 or MPEG will barely help for FPV drone video.
H.264 720p will go a bit fuzzy during motion at 2 Mbps, MJPEG will just look like utter trash non stop.
MJPEG is jpeg still of each video frame. Better you explain how exactly and why MJPEG will look like utter trash. I am all ears.
[edit] Oh, you mean - at same video stream bitrate. Yes, I agree in such case.
-
I guess the next logical step will be to purchase a beefy microcontroller of some sort. Any recommendations?
I'd try to find a SBC, but you have somewhat restrictive requirements. Light weight, I assume you want parallel CSI (for the cheap 720P60 cameras like OV5647), you'll need sufficient GPIO and a DMA engine which can push data to it for your DAC (I don't think Allwinner chips can do this for instance and Raspberry Pi was very slow with DMA AFAIR). As I said before, maybe something based on iMX6UL or iMX6ULL?
On the receiver the weight won't matter, but there you have to be able to attach a relatively fast ADC ... also as I said, if you use the standard receiver modules like RX5808, you'll still need to downconvert it to baseband.
PS. you could also only use the SBC for interfacing with the camera and compression, and then SPI the bitstream to a microcontroller and let that handle the DAC.
-
Why not use video camera that provides compressed digital video output?
Because latency from the onboard, non latency-optimized encoder is almost guaranteed to be terribly inappropriate.
FPV video stream is not like hollywood movie /w fixed camera, every next frame differ too much.
Correct.
-
I would start by implementing something simple like a JPEG stream over BPSK. Cameras like the ov2640 have JPEG compression built in which means all you need to do is buffer up the data and modulate it (and add some FEC later on). You can output the BPSK signal on a single digital pin on the FPGA which means no DAC needed.
For the receiving side I'd experiment using a PlutoSDR or LimeSDR, decoding the signal on the PC initially. Once you have the modulation scheme worked out you can implement the receiver on the FPGA of the SDR.
-
Actually I think the best signal to generate first would be a NTSC signal (https://hackaday.com/2013/03/27/color-ntsc-video-directly-from-an-avr-chip/).
I misunderstood the RTC6715, I thought it only brought IF out, but there's also another FM demodulator in there ... so I assume the IF just loops back around the outside to allow for filtering ... Oops. The mobile phone fpv receivers (https://www.banggood.com/Eachine-ROTG02-UVC-OTG-5_8G-150CH-Dual-Antenna-Audio-FPV-Receiver-for-Android-Tablet-Smartphone-p-1242422.html) make a lot more sense to use than the modules I mentioned before I think. The NTSC test signal can be used for tuning the receiver, before you start a digital data stream which will probably make the automatic tuning barf.
So with that the hardware I would use ... Nanopi Neo Air or Banana Pi Zero with a OV5640 camera for the transmitter, some other SBC with a LCD connector for the receiver (for prototyping you could use a laptop running Linux with preempt_rt), ROTG02 receiver, TLC7528 DAC, AD9200 ADC and EZ-USB® FX2LP to create a buffered connection between the SBCs and the ADC/DAC. All in all a rather substantial project getting all that to the point of even being able to communicate.
-
So with that the hardware I would use ... Nanopi Neo Air or Banana Pi Zero with a OV5640 camera, ROTG02 receiver, TLC7528 DAC, AD9200 ADC and EZ-USB® FX2LP to create a buffered connection between the SBCs and the ADC/DAC. All in all a rather substantial project getting all that to the point of even being able to communicate.
So you are suggesting to make video compressor, >= 6Mbps TX/RX modem(!) with FEC(!) out of Nanopi Neo Air or Banana Pi Zero. Also radio of choice is receiver (https://www.banggood.com/Eachine-ROTG02-UVC-OTG-5_8G-150CH-Dual-Antenna-Audio-FPV-Receiver-for-Android-Tablet-Smartphone-p-1242422.html?ID=224&cur_warehouse=CN) with specs/description "It has low latency around 100ms" :D
-
Yes, a 6 MHz datastream isn't really a huge problem for a 1.2 GHz 4 core processor.
The receiver has a NTSC->USB digitizer, that's where the latency comes in. You simply tap a line on the PCB, that 100 ms latency doesn't affect you.
-
Been watching this thread with interest while on the road and unable to comment.
My son is an internationally ranked drone racing pilot (as just point of reference, he's 1st Place member of the five-person TeamUSA that represented the United States at the IOC-sanctioned World Championships in Shenzhen China a couple of months ago). Video problems are the NUMBER ONE ISSUE in this sport. We talk about ways to address that all the time. We've considered quite a few of the things discussed in this thread.
I'm thrilled that this topic has generated so much interest here. The thing I want to share with everyone - from the direct hands-on perspective of a top-ranked pilot and the literal embodiment of the target market for such products - is that latency is your single most important criteria. Many times at competitions my son has sacrificed video quality to reduce latency. In fact, he flew with B&W-only video for a while because with the cameras and electronics of that era it reduced latency by a pilot-noticeable amount. He also suffered the loss of a qualifying round at Nationals in Reno NV in 2017 because the venue-provided diversity system had so much latency that he and other pilots could not control their aircraft; most of the top pilots switched back to their own gear, enduring the multipath and quality problems to get responsiveness.
If you haven't done this yourself, it's hard to realize just how important low latency is. Most of the world-level racing drones are spinning ~5 inch props on ~200-300mm frames, and on straights they'll hit 80+ MPH heading toward split-S turns that peak 150+ amps (not a typo!) from their 6S LiPo packs. The gates they're shooting for are sometimes only 5x5 feet, and when you're approaching at 80 MPH tens of milliseconds really do count.
There is "tribal lore" in the FPV industry right now that compression cannot be done because of the inherent latency. My son and I have a different opinion and think it's possible by thinking a bit outside the box, but my point is that the industry is so intensely focused on low latency video that everything is analyzed with that in mind.
I'm excited about this thread and where it will hopefully go, but wanted to offer a little guidance about what's important as it moves forward.
Thanks!
-
The thing I want to share with everyone - from the direct hands-on perspective of a top-ranked pilot and the literal embodiment of the target market for such products - is that latency is your single most important criteria. Many times at competitions my son has sacrificed video quality to reduce latency.
Kudos to your son and thanx for valuable input. If latency is paramount then as you say, only solution is out of the box thinking. OP did very good job already. If every millisecond counts, then existing solutions of low cost digital video cameras and existing video compressors do not qualify, unless running at insane framerates like 240 FPS.
If we want to beat analog TV transceiver latencyy using digital tech - we can't unless we increase frame rate. So, it shall be 120Hz or higher. Also whole frame buffering to compress/decompress is way too expensive waste of precious time. Analog TV transmits every scan line w/o buffering, digitally it could be "slices" of the frame (16, 32 or 64px vertically), each individually compressed as separate (jpeg?/h.264?) frame/video and transmitted in separate radio frame to receiver. BTW pilots notice rolling shutter effect (https://en.wikipedia.org/wiki/Rolling_shutter) running 60FPS *interleaved* analog NTSC?
I believe that approach to the problem shall be iterative. Video compressor/decompressor shall be designed first. Uncongested 20MHz 5.8GHz (802.11ac) WiFi can provide sub-1ms one-way latency, such "radio" is good enough for first prototypes: https://mentor.ieee.org/802.11/dcn/18/11-18-1160-00-0wng-controlling-latency-in-802-11.pptx (https://mentor.ieee.org/802.11/dcn/18/11-18-1160-00-0wng-controlling-latency-in-802-11.pptx)
-
Video problems are the NUMBER ONE ISSUE in this sport.
Frankly I'm surprised contests and races even exist already. I've been doing FPV for 16 years, and 10 years back or so we were discussing setting such stuff up and just concluded that video was just too poor for that to provide a decent experience and amount of enjoyment both for pilots and public and ensure fairness between competitors.
It's still just as poor and now demonstrably so, but it seems some are surprisingly somewhat OK with the compromise and ended up doing it anyway...
-
If every millisecond counts, then existing solutions of low cost digital video cameras and existing video compressors do not qualify, unless running at insane framerates like 240 FPS.
Frame rate isn't necessarily related to latency. 30FPS, or even 24FPS, is a sufficient frame rate as long as the end-to-end latency is low enough.
A few more thoughts:
* Analog video is fine, from the perspective of the pilots. There's no inherent advantage to going digital, or even to higher resolutions. They're happy with 480 lines. Sure more is better, but I promise they'll give up resolution to get low latency and reliability (no dropouts).
* Next to latency, dirtbag @%$##$%ing other pilots plugging in their quads in the pits on the same FREAKING channel as someone in an active heat is the next biggest problem. Despite rules and penalties, this happens ALL THE TIME even at international competitions. Happened to my son during his very first qualifying heat in Shenzhen, in fact... he was doing great and then suddenly he's looking at some guy's shoes in the pit area. His quad crashed into a gate and he was taken out of that qualifier, and due to the scheduling they didn't offer reflies at this event, so his total practice time on the track before the actual races was reduced due to some other jerk's stupidity. (We tracked down who it was using recorded video and got him disqualified, but that didn't help TeamUSA.)
This problem is almost always some pilot believing "It won't matter for just a quick second". They drop their transmitter power to "pit mode" (generally a couple of mW, whereas most racing is at 25mW these days), remove their transmitting antenna (never mind the SWR mismatch), etc. and "plug in for just a quick test". But at these speeds it only takes a moment of zero visibility to crash into something, and since they've dropped to 25mW to reduce inter-pilot interference the receivers are that much more sensitive and can pick up one of these "quick tests" in the nearby pits sufficient to corrupt or entirely override video coming from a quad that's hundreds of yards away on the other side of the track. Yes, we use directional antennas (some really good ones, in fact) but it's amazing how often the offending &!#$&%&!'er must be in one of the side lobes of the pattern and gets picked up just fine.
THIS is the reason we are considering going to digital video. Not because we need better resolution, but because we need to be able to reject interference from other people on the same analog frequency. The uplinks resolved this a few years ago by going digital and using a token-based system where you "pair" a given transmitter (remote control) to the aircraft's receiver. This system works quite well and they can have dozens of pilots flying "line of sight" (not FPV) in real time without incident. But those are relatively low bandwidth comms compared to (even low-res analog) video.
We've thought about doing some sort of cellphone-like TDMA system, but up in the 5.x GHz ISM band where FPV video lives, so that we can successfully accommodate "plug-ins" like I described. It will take some sort of system like that to fix this, because you literally don't know and can't prevent when some stupid, idiotic, self-serving jerk will plug in during a race and take out a pilot by generating an analog signal or a data stream right on top of what the pilot SHOULD be receiving.
* Another issue, though less important than solving the above, is dropouts. FPV pilots scan eBay for truly analog mini-monitors that display snow instead of a blue screen when no signal is being received. All FPV goggles are analog too. Why? Because signal strength varies all over the place during a race. The antennas are constantly changing orientation to each other, the distances (and thus propagation loss) change wildly, etc. Digital video has the incredibly annoying "feature" of simply turning off when the signal strength goes below some threshold. But analog does not suffer from that... your image quality may degrade, you may lose color (due to loss of color burst on each line) and fall back to B&W for a moment, but the key is graceful degradation rather than brick-wall failure. Once a digital video system "loses sync" it can take several seconds to resume operation. That's an eternity. Imagine driving on the freeway inside a curving tunnel at 80 MPH and someone covers your eyes... seconds feel like years, even a couple of seconds can spell disaster. Far better to gracefully degrade like analog video, from loss of clarity to loss of color to higher S/N ratio (snow) because the human brain and vision system are quite good at interpolating in a noisy environment. But they can't interpolate from a blue screen!
For this reason, my son and I have been toying with employing DSP correlation in a multichannel diversity receiver. There are serious volume, mass, and power limitations on the aircraft themselves, but the ground stations are basically unlimited so doing lots of processing on the ground is totally feasible. (Correlation will not address the issue of bat-freaking braindead idiots transmitting on the same channel so that problem would still have to be addressed.)
* Whatever the front end, the output signal would be best as baseband NTSC. The entire industry is tooled up to support that, and the race operators (MultiGP, FAI, etc.) will resist anything that isn't plug compatible. Why do they care? Because they have live judges dedicated to each pilot position, plus they record all video streams, in case someone files a protest. All that infrastructure is NTSC. If a pilot arrives and he cannot provide an NTSC signal to show his FPV, at best he will lose any protests (for lack of evidence) and at worst he will be prohibited from racing.
-
Frankly I'm surprised contests and races even exist already. I've been doing FPV for 16 years, and 10 years back or so we were discussing setting such stuff up and just concluded that video was just too poor for that to provide a decent experience and amount of enjoyment both for pilots and public and ensure fairness between competitors.
It's big, and getting bigger FAST. Drone racing is already covered on ESPN. It has been on 60 Minutes. It has enormous corporate sponsors (think Red Bull, Pepsi, etc.). My 16YO son has four corporate sponsors and was courted aggressively while in Shenzhen by local electronics manufacturers who treated him like a rock star. They were hunting him down at the venue, calling for him by name over the crowds. Two of them took us on whole-day tours of their R&D and manufacturing facilities, with the senior management (including CEO's) taking us to very fancy, very authentic Chinese lunches, just to spend time with the TeamUSA pilots and get their input on current and future drone racing products and offer them sponsorships.
If they smell opportunity like that, you know this industry is going places. It's one of the reasons we're talking about improved video systems (and that's just one of the product ideas we have). There's real money moving around and some really big players in the electronics industry are starting to take notice.
-
I think you're in for a heck of challenge to successfully make a digital system that can replace the analog ones.
Low power, low latency, instant fade recovery, interference rejection etc. As you mentioned the brain does an amazing job filtering noise and working through the signal fades.
In terms of other people turning a transmitter on in the pits, this is a problem that has been faced by RC since it began. Before 2.4 GHz is was routine for big RC car races to have a transmitter impound. You got your transmitter back just before your heat. Same thing in the RC airplane/helicopter world - frequency pins allowing you and only you to use to use a specific channel and of course transmitter impounds. It won't prevent 100% of problems but it works quite well and is what can be done again.
Long term I see FPV racing ending up about the same as battle bots. It will continue but popularity on TV will suffer some. I love RC but the issue I have with FPV is that we could just simulate the entire thing like a video game.
btw, if you've raced at major drone racing events then odds are very good you've used hardware I've built!
-
Any reason not to be thinking spread spectrum?
Link could still be basically FM, with spreading by a frequency hop during horizontal retrace.
Advantage is that unless someone has set the same spreading code as you a jammer will take out at most a few randomly scattered video lines instead of the whole thing.
The issue with FM is that it captures to the strongest signal in the receivers pass band, which makes numb nuts in the pits a problem.
The RC flying community back in the day used to handle this by only handing the crystal pairs to the pilots when they were going to fly, and taking them back afterwards (And there was hell to pay if you were found to be in possession of extra rocks).
73 Dan.
-
In terms of other people turning a transmitter on in the pits, this is a problem that has been faced by RC since it began. Before 2.4 GHz is was routine for big RC car races to have a transmitter impound. You got your transmitter back just before your heat.
Big difference this time is that the VEHICLE is the transmitter (video coming back to the operator). "Impounding" vehicles is a non-starter... folks have to work on/repair their aircraft between races. Props break, wires get sliced, PCB's and/or their components get dinged... it's a lot like an airborne demolition derby. Impound aircraft and the sport will cease to exist overnight.
And no, we cannot "just impound the video transmitter PCB". To keep size and cross sectional area to a minimum, the PCB "stack" is a tight assembly fully customized by each pilot. You can't just add or remove PCB's as a matter of convenience... even repairs can involve almost complete diassembly and reassembly depending upon how the aircraft is laid out. This problem needs to be solved with technology, not social engineering.
EDIT: Here's an example. At the World Championships in Shenzhen, DJI (the "stable camera platform" drone company) provided HD cameras with integrated video transmitters so the organizers could livestream high quality video from each aircraft's perspective during each race. Those little cameras were expensive, and they only had so many, so DJI staff were standing at the flight line to install them on each aircraft as they were carried out to start a heat and then they recovered them off each aircraft when they were carried back after each heat. They had a variety of ways to attach them - zipties, double sided foam tape, you name it - but the chaos was ridiculous. The pilots got into serious arguments when the DJI people picked where they wanted to mount them... many pilots didn't appreciate the DJI folks "manhandling" their aircraft as they treated the cameras like precious gems... the DJI people got angry each time a camera was damaged from an impact (as if the pilots were doing it on purpose?!?)... etc. This just illustrates the craziness of trying to remove/impound/add some piece of electronics prior to each heat, and in this case it didn't even have to tie into the other electronics on the platform.
There are at least a dozen popular video transmitters in current use in the FPV world. Each optimizes for some different set of features. Sometimes size is important. Other times features win out even if the board gets larger. Some people insist on coax connectors on the board, while others demand to have the coax soldered straight to the PCB. Some Tx's are standalone, while others tie into the flight controller to get overlay/HUD info such as battery voltage, present/peak current draw, elapsed time, etc. so the pilot knows how much is left "in the tank" and whether he has to nurse things along to survive the full heat length. There's no one size fits all, and if DJI's experience is any indication the swapping of hardware in real time isn't an acceptable solution.
the issue I have with FPV is that we could just simulate the entire thing like a video game.
Simulators (aka trainers) do exist. However, I notice that real fixed and rotary wing RC modeling continues to be popular despite incredibly real simulators being available for many years now on a variety of platforms. There's something about "reality". Plus, a huge draw of this sport for a lot of the pilots is the merging of their love of flying with their love of electronics, assembly, craftsmanship, etc. And frankly from this Dad's perspective, I'm a lot more supportive of FPV than I would be if my son were living in the basement playing video games like so many other teens these days. The skills he's building in hardware, software, mechanical, 3D printing, social interaction with sponsors and other pilots, etc. are transferable to a wide range of career options and life lessons. He's considering majoring in Aeronautical Engineering when he starts college next year, which I doubt would have happened if he'd been playing video games.
btw, if you've raced at major drone racing events then odds are very good you've used hardware I've built!
OK, you can't just leave that hanging out there! {grin} Fess up, what are you talking about? (And noting the flag you have in your ID, my son took first place in the Western Regional Canada Championships in Kelowna BC last year and 4th place (top non-Canada pilot) in Canadian Nationals in Ottawa ON a few months later, so if anything you've built is local to Canada the odds are actually quite high!)
-
Any reason not to be thinking spread spectrum? Link could still be basically FM, with spreading by a frequency hop during horizontal retrace.
No reason at all, and that's actually something we've talked about. Again, losing a line here and there is something the brain can accommodate.
EDIT: However, this only works if all pilots in the area switch to the new system. Otherwise the new system would be susceptible to a "old school" pure analog transmitter being powered up. I suppose if we used the entire available ISM band for the spread the effect of a single analog channel powering up could be minimized, but realistically any SS system would have to be designed to accommodate multiple pilots simultaneously... and we're back to channelized allocation again, where the subsequently narrower channel width makes that channel's spectrum overlap to a greater extent if the right/wrong analog channel is turned on. Figure support for eight simultaneously active pilots, so now the band is channelized to 1/8th the ISM band, the odds of overlap with analog get pretty high.
This is why I was talking about something similar to the tokenized system used for modern RC uplinks. It changes the presumption from "avoid interference" to "plan on interference and accommodate it".
The RC flying community back in the day used to handle this by only handing the crystal pairs to the pilots when they were going to fly, and taking them back afterwards
That's a better solution than "impounding aircraft", but since virtually everything is synthesized these days there's no physical objects to control.
(And there was hell to pay if you were found to be in possession of extra rocks).
I've never heard crystals called "rocks" before - very cool and singularly appropriate!
-
This is why I was talking about something similar to the tokenized system used for modern RC uplinks. It changes the presumption from "avoid interference" to "plan on interference and accommodate it".
Right. In case of widely accepted digital FPV, every transmitter shall have unique "call sign" broadcast (like SSID) so you know who is to blame when *t hits the fan. There could be even some channel management system (like DHCP for IP) which assigns channels on request to avoid (un)intentional collisions.
Frame rate isn't necessarily related to latency. 30FPS, or even 24FPS, is a sufficient frame rate as long as the end-to-end latency is low enough.
Pure BS. Let your son run 30fps progressive instead of 30fps interlaced (60Hz half-frames) and ask his opinion ;) Alternatively you can ask virtually any FPS/race gamer what is better for games: 30 FPS or 120 FPS. Kinda obvious answer, right?
-
Pure BS. Let your son run 30fps progressive instead of 30fps interlaced (60Hz half-frames) and ask his opinion ;) Alternatively you can ask virtually any FPS/race gamer what is better for games: 30 FPS or 120 FPS. Kinda obvious answer, right?
We're talking about two different things. You're talking about refresh rate. I'm talking about signal path latency. The latter is not related to the bandwidth of the data (read: refresh rate) in the signal path. A low refresh rate video signal can have lower signal path latency than a high refresh rate video signal, and vice versa. The discussions in this thread have been about signal path latency, e.g. delays imposed by compression and/or conversion to and back from digital.
-
This problem is almost always some pilot believing "It won't matter for just a quick second". They drop their transmitter power to "pit mode" (generally a couple of mW, whereas most racing is at 25mW these days), remove their transmitting antenna (never mind the SWR mismatch), etc. and "plug in for just a quick test".
EMC (Electromagnetic compatibility) is nothing new, and in fact something every designer/manufacturer must comply to if selling equipment, at least in Europe.
When testing equipment, say naval emergency transmitters, it is paramount not to disturb the open air frequencies - unless you want a visit from the coast guard and their huge chopper. So, smart ingenious people (or poorer ;)) came up with a shielded box (Faraday cage) - and filtered inputs/outputs.
https://www.google.com/search?q=emc+shield+box&source=lnms&tbm=isch&sa=X (https://www.google.com/search?q=emc+shield+box&source=lnms&tbm=isch&sa=X)
I know, organizers must have it and require its usage - but from the rules I have read it states that antennas must not be soldered on, so what is stopping the scene from utilizing these boxes, with screw on antenna cables + attenuation so the receiver input is not fried.
A great way to test your equipment, without disturbing others.
-
Any reason not to be thinking spread spectrum?
...
Advantage is that unless someone has set the same spreading code as you a jammer will take out at most a few randomly scattered video lines instead of the whole thing.
Won't any other VTX have the same effect, especially when there's ~8 of them in use.
-
I think you're in for a heck of challenge to successfully make a digital system that can replace the analog ones.
I'd just like to interject for a moment. What you're referring to as analog is in fact digital PCM + FM. Analogue is a corner case of digital modulation, the signal is generated by a DAC ... they just use a really weird encoding for legacy reasons.
Once you cut the legacy NTSC out of the chain you can trivially change the resolution/refresh rate. An OV5460 can do VGA at 90fps and QVGA at 120 for instance. Even if you encode it as PCM, you have much more control over it. You're not reliant on some blackbox circuitry to synch to the NTSC signal for instance. If you have signal interference there is no reason to get confused, keep counting and the next pixel you get will be in exactly the right place. Who needs sync signals every line when you have crystals? You can throw a reference pulse in there every couple of ms and calculate the path response and correct for it, etc.
-
Pure BS. Let your son run 30fps progressive instead of 30fps interlaced (60Hz half-frames) and ask his opinion ;) Alternatively you can ask virtually any FPS/race gamer what is better for games: 30 FPS or 120 FPS. Kinda obvious answer, right?
We're talking about two different things. You're talking about refresh rate. I'm talking about signal path latency.
When you know latency of existing analog FPV radios, you know that there's nothing to improve. (https://oscarliang.com/fpv-camera-latency/) That's my point which I already provided BTW:
If we want to beat analog TV transceiver latencyy using digital tech - we can't unless we increase frame rate. So, it shall be 120Hz or higher. Also whole frame buffering to compress/decompress is way too expensive waste of precious time. Analog TV transmits every scan line w/o buffering
Inter-frame delay time also shall be considered as latency - you want it or not.
-
Who needs sync signals every line when you have crystals? You can throw a reference pulse in there every couple of ms and calculate the path response and correct for it, etc.
Crystals are NOT perfect, and will change frequency when subjected to g-changes (acceleration, deceleration, rotation). Ah relativity :box:
An extensive subject, already looked into by TI, and not to be forgotten - it is not only reflection/multi-path/dobbler effect which affects system performance in this case.
When designing a clock oscillator into a system, the impact of vibration is often overlooked. However, as demonstrated above, oscillators with similar specifications can xhibit vastly different behaviors when subject to vibration or shock, possibly resulting in the addition of tens of picoseconds of jitter and violation of design requirements. Very important when using mixers and/or multipliers - http://www.ti.com/lit/an/snaa296/snaa296.pdf (http://www.ti.com/lit/an/snaa296/snaa296.pdf)
Acceleration Sensitivity Characteristics of Quartz Crystal Oscillators - http://www.greenraytoday.com/beta1/wp-content/uploads/2017/02/AccSens.pdf (http://www.greenraytoday.com/beta1/wp-content/uploads/2017/02/AccSens.pdf)
-
With 166 nanoseconds between each symbol you can swallow a few picoseconds.
-
"Impounding" vehicles is a non-starter... folks have to work on/repair their aircraft between races. Props break, wires get sliced, PCB's and/or their components get dinged... it's a lot like an airborne demolition derby. Impound aircraft and the sport will cease to exist overnight.
Just disallow any power sources in controllers and drones except during racing or specifically allocated testing periods. Any time you power it on there is a non acceptable risk of interference, there is no need for that ... only want.
-
Back in the day when we were figuring out what existing analog VTXs to use for FPV it was common for some of them to be severely affected by vibrations indeed, to the point of having a barely viewable picture with common prop vibration.
EDIT: However, this only works if all pilots in the area switch to the new system. Otherwise the new system would be susceptible to a "old school" pure analog transmitter being powered up.
I suppose if we used the entire available ISM band for the spread the effect of a single analog channel powering up could be minimized, but realistically any SS system would have to be designed to accommodate multiple pilots simultaneously... and we're back to channelized allocation again
Why? RC radios use the whole band and can coexist just fine. OK lower bandwidth, but larger numbers too...
Just disallow any power sources in controllers and drones except during racing or specifically allocated testing periods.
Again for repair reasons you can't fully disallow any kind of power source, if you did not let the pilots test for repairs other than VTX-related during other races you'd then have to lose way too much time between races with testing periods.
For the VTX there's already a system for this made by a manufacturer, when checking in to a race the contest organizer essentially disables your TX so you can power it up all you want but it'll never transmit. When the participants are ready on the start line the race director goes by with the NFC programming wand to enable them until the next power off.
But that's not practical, because people also damage/break their TXs so they need to test them, and going to everyone between races to re-enable to allow testing then re-disabling before the next would be basically impossible. But there are easy ways to make that better, e.g. a longer range link similar to the R/C system built on the TX and a central "server" station managed by race directors, anytime someone powers up a TX it queries the server for whether it is allowed to transmit and only does so if answered positively.
-
Again for repair reasons you can't fully disallow any kind of power source
My intuition says that unless you create some kind of standard which allows a physical detachable interlock which referees can easily check/control to keep people off the racing bands, which batteries are in a way, I'd say the odds of interference are in the whole percentage points if someone does a powered test during a race.
So it's just a question of what you find acceptable risk ...
-
That's why I mentioned other solutions to prevent transmission that are reasonably tamper-proof and don't prevent aircraft testing.
-
Because you can't use inter-frame compression (due to latency) you loose much of the advantage of digital video, being the removal of redundant information (and ideally replacing it with error checking of the more important information). The existing analogue systems essentially transmit lots of redundant information, and the Mark One Human Brain does a lot of postprocessing to tease the image out of noise and interference and it's actually very good at it. From an information theory perspective there isn't a clear advantage of a digital system in this application.
Going digital does not in itself solve the two-transmitters-shouting-over-each-other problem. You are still working with the same slice of spectrum, and anything which degrades your SNR will degrade your received signal. You can have active spectrum management with some kind of clear channel assessment or "permission to transmit" token, but that works just as well as a low-bandwidth digital control channel on top of analogue video as it does on a digital stream.
If the real problem is suddenly cutting to the wrong stream then that's essentially a capture effect, some types of digital stream might avoid that, but the degraded SNR might also just kill it entirely which is hardly any better. It seems likely that cutting to the wrong stream is the combination of two things, the FM capture effect, and which set of video sync signals the display manages to lock to. Switching from FM to AM (as all UHF-band analogue video always was) and then focusing the rest of your effort on making the sync as robust as possible (transmit it separately on a different narrow-band channel? Use a PLL to regenerate the sync so it can free run for a few frames if the sync is lost? Both?) You'd then have the sum of your wanted signal in complete frame sync, and your pits signal, almost certainly rolling, and I'm pretty confident that a human brain will separate them. Edit: if you have full control over the video source you could add a pseudo-random dither to the syncs which pretty much guarantees that your spurious signal will be smeared into fuzz, that should help the brain do the separation.
-
With 166 nanoseconds between each symbol you can swallow a few picoseconds.
5.8GHz FM transmitter chip - RTC6705 datasheet page 7 (sky.geocities.jp/oumeastro/RTC6705-DST-001.pdf)
Synthesizer counter default setting( 5.8Ghz band:5865MHz)
For 5.8Ghz band, FRF = 2*(N*64+A)*(Fosc/R)
Example: default FRF=5865MHz, Fosc=8MHz, R=400
5865=(2*N*64+A)*8Mhz/400=2*(N*64+A)*20KHz
N=2291(=8F3xH), A=1(=01xH)
Output freq is 5865 [MHz], Crystal Oscillator frequency is 8 [MHz] - an input/output multiplier ratio of 733.125
So a few (1,2,10) picoseconds suddenly becomes nanoseconds. 10 [pS] * 733.125 = 7.331 [nS]
Edit - added following: This is 'baseband' center frequency - it shifts up/down when subjected to motion. Any receiver must be able to follow in phase, if it is using phase-modulation-demodulation as an information carrier (As Digital video transmission most commonly does)
-
Hey all, I have decided that I'm going to put this project on hold on my end. You guys have brought up so nsny good points that i had not considered that make this project damn near infeasible. Feel free to continue discussion, as I feel it is very constructive.
-
Again for repair reasons you can't fully disallow any kind of power source, if you did not let the pilots test for repairs other than VTX-related during other races you'd then have to lose way too much time between races with testing periods.
That's why I said earlier that this must be solved with REAL Engineering, not social engineering.
I too thought of the Faraday cage idea, so far I see two possibilities:
* Simply {grin} wrap the pit area in appropriately sized wire mesh. 5.x GHz would require some rather small pitch openings (haven't done the calcs). Perhaps just a grounded screen in the direction of the flight line and course itself. This *might* be feasible at a(n) (inter)national scale event, but lots of local events run by local clubs feed into those and I suspect most local clubs could not afford the screening material. As an example, a typical regional qualifier can involve a few dozen pilots and a total attendance of perhaps 100-200 people - enough to experience this problem but not so many that there's enough cash flowing to pay for expensive infrastructure.
* Have small dedicated test cages into which aircraft are placed during powered non-flight tests. This might actually be feasible, but it would have to be a glovebox like affair where hands and probes could have access while powered up and still providing shielding. When we're on the road we carry a decent set of test equipment - portable scope, meters, VTX spectrum analyzer, etc. - as we repair and test things during events that can last 1-4 days. Those probes and the hands that hold them or adjust things in real time would need access that doesn't defeat the purpose of the Faraday cage.
-
hexahedron - if you're willing to wait, and invest time and money, 5G network might be interesting for you:
https://en.wikipedia.org/wiki/5G#Advantages (https://en.wikipedia.org/wiki/5G#Advantages)
It could be a (very) long time before chipsets become publically avaliable however, and projects open sourced like the GSM Baseband software implementation OsmocomBB (http://bb.osmocom.org/trac/ (http://bb.osmocom.org/trac/))
Good luck, and happy droning :P
-
* Have small dedicated test cages into which aircraft are placed during powered non-flight tests. This might actually be feasible, but it would have to be a glovebox like affair where hands and probes could have access while powered up and still providing shielding. When we're on the road we carry a decent set of test equipment - portable scope, meters, VTX spectrum analyzer, etc. - as we repair and test things during events that can last 1-4 days. Those probes and the hands that hold them or adjust things in real time would need access that doesn't defeat the purpose of the Faraday cage.
Metallic mesh gloves exists in different sizes and lengths - https://www.google.com/search?q=metal+mesh+gloves&source=lnms&tbm=isch&sa=X (https://www.google.com/search?q=metal+mesh+gloves&source=lnms&tbm=isch&sa=X)
If you're really handy, you could hack a (large) microwave oven and cut some holes in its sides.
Mount the gloves, and secure them so they do not leak RF, and you have a shielded glove box you can look into - and it opens/closes easily.
Copper tape is used extensively in EMC compliance testing, if you don't want to use/rely solely on metallic screws and unweave the glove - it should be tight and tidy ;)
-
Because you can't use inter-frame compression (due to latency)
If you use 8x8 blocks, the only necessary latency is buffering 8 lines ... 0.19 ms at 720p60.
-
Because you can't use inter-frame compression (due to latency)
If you use 8x8 blocks, the only necessary latency is buffering 8 lines ... 0.19 ms at 720p60.
:-// ???
Doing a partial frame update/interlacing - is like filling a FIFO buffer, and only rendering the final picture frame when full. Great for slow/non-moving stuff.
The drone is moving (fast), rendering the data sent to the FIFO buffer 'useless' (that's what I understand anyways) as the next 8 lines will be nearly the same.
If you hit the 'right' speed, it will be exactly the same - unless you do a down-up rendering, and I think that would really mess with your perception (rolling shutter effect - https://en.wikipedia.org/wiki/Rolling_shutter)
Imho the 'right' solution is a super high frame-rate, very short packets sent on a low latency network - and 'post-processing' those packets onto a high frame-rate display.
(I have worked with NTSC, PAL, SECAM, (Videocrypt+D2-MAC), MPEG1, MPEG2, MPEG4 on various medias - so I have a bit of experience on video systems)
-
Imho the 'right' solution is a super high frame-rate, very short packets sent on a low latency network - and 'post-processing' those packets onto a high frame-rate display.
That would be ideal. Some challenges:
* The camera and its transmit-side electronics must be physically small and weigh well under 100 grams. A particular camera with which I'm familiar is 74 grams and considered grossly overweight by most pilots. Total aircraft dry weight is generally around 300 grams including video system, of which the four motors alone are 100-120 grams.
* Establish a low latency network (50mS is about the max) on an ISM band (so no licenses are required) that can support at least eight simultaneously active pilots.
...and the above does not address the problem of random idiots plugging in during a race, which we've kind of been dealing with as a separate question.
-
I still believe that the correct way to address the idiots in the pits who plug in during a race should be technical and not social. However, a solution that could be implemented today with existing technology would be a separate bank of receivers and directional antennas pointed AWAY from the course, and TOWARD the pits. When a race is to begin, start recording on all active channels. The nulls in the antennas should (mostly) reject the aircraft on the course. But if someone lights up in the pits, they should be nicely within the antenna pattern and be recorded, with the receiver(s) naturally selecting the then-stronger signal. Most pilots have their "handle" on their OSD's and are thus self-identifying. If a pilot reports video problems these recordings are reviewed and if an offender is found, they are ejected from the event. And maybe their equipment is doused in gasoline and burned too. (Tempers get pretty hot when someone does this. The chief official at an event a while back actually took the offender's aircraft and beat it to pieces with a hammer in front of the crowd to set an example.)
I'd still rather have a self-enforcing technical solution, but this might be a stopgap until the tech is ready.
-
* The camera and its transmit-side electronics must be physically small and weigh well under 100 grams. A particular camera with which I'm familiar is 74 grams and considered grossly overweight by most pilots. Total aircraft dry weight is generally around 300 grams including video system, of which the four motors alone are 100-120 grams.
* Establish a low latency network (50mS is about the max) on an ISM band (so no licenses are required) that can support at least eight simultaneously active pilots.
http://socialcompare.com/en/comparison/raspberrypi-models-comparison (http://socialcompare.com/en/comparison/raspberrypi-models-comparison)
Raspberry Pi Zero W - without headers: 9 grams
Raspberry Pi Zero WH - with headers: 12 grams
Some random 5.8 GHz AC WiFi high speed dongle - https://www.amazon.co.uk/Daffodil-Ultra-Fast-Dongle-LAN05/dp/B016QKQ1U0#productDetails (https://www.amazon.co.uk/Daffodil-Ultra-Fast-Dongle-LAN05/dp/B016QKQ1U0#productDetails)
Weight: 9 grams
5.8 GHz WiFi band is ISM, and has ping times at/below 1 mS if done correctly ^-^
I do not know if the Raspberry Pi is good enough to handle video encoding at breakneck speeds - fortunately other SBC's exists that does not use Broadcom , such as the Beagle Bone(s)
(I have not used those to video-encode, so I can't comment on it's usability):
https://www.mouser.com/new/seeedstudio/BeagleBone-black-vs-green/ (https://www.mouser.com/new/seeedstudio/BeagleBone-black-vs-green/)
The TI SoC has a Programmable Real-time Unit (PRU) - making it excellent for controlling hardware pins, timing, transfers, etc.
It is large (it is a development board) - its weight is around 40 grams.
I would say hardware wise, it is not impossible - it should be rather light, inexpensive, and easy to obtain.
Software - well, can't help you there
Edit - clarify "5 GHz band is ISM, and has ping times at/below 1 mS if done correctly ^-^"
-
Doing a partial frame update/interlacing - is like filling a FIFO buffer, and only rendering the final picture frame when full. Great for slow/non-moving stuff.
Most digital video sensors works same "scan line" way as oldskool TV - readout happens line by line. At 640x480 @ 60fps analog tranceivers will have roughly 1/60 sec = 16.666 msec "delay". Proposed 8-line buffer will add 8 * ( 1/(60*480) ) = 0.2777 msec delay. To me 1.66% frame time increase due to 8 line buffer does not look like much. Assuming that existing analog transceivers are ideal giving 16.66 msec best case latency, for digital video we use 120 FPS so summary interframe time + buffering delay is (16.666 + 0.2777)/2 = 8.472 msec. If we add kinda pessimistic 1 msec digital radio delay (doable even with WiFi), total latency is 9.472 msec. One is clear - radio is not the main contributor to (perceived) latency.
-
One is clear - radio is not the main contributor to (perceived) latency.
Agreed. The discussions of latency here have centered on various proposed data processing steps such as compression to obtain higher resolution, or FEC to handle RF dropouts. Today's pure analog systems certainly don't have much in the way of processing latency, AFAIK they don't even buffer a line - it's as real time as the semiconductors make possible.
-
Most digital video sensors works same "scan line" way as oldskool TV - readout happens line by line.
One is clear - radio is not the main contributor to (perceived) latency.
One thing is the sensor, another is the display.
The sensor output is 'serial' - the display output is 'not'. I hope you get where I am pointing at, as the screen has to be 'stationary long enough to show the entire picture', and then show the 'entire next picture long enough'
I too agree on the radio tech being fast enough.
Edit: Adding info:
Oldschool TV's/CRT work 'in serial mode', however the screen used phosphor which has a persistence glow, thus 'keeping the image on screen'
https://en.wikipedia.org/wiki/Cathode-ray_tube#Phosphor_persistence
-
One thing is the sensor, another is the display.
The sensor output is 'serial' - the display output is 'not'.
Are you sure? :)
-
It would be spamming doing a lot of replies:
S
o
Y
e
s
I
a
m
u
n
l
e
s
s
y
o
u
r
b v
r i
a s
i i
n o
n
w
o
r
k
s
d
i
f
f
e
r
e
n
t
l
y
t
h
a
n
m
i
n
e
-
I would say hardware wise, it is not impossible - it should be rather light, inexpensive, and easy to obtain.
Very promising, if the hardware can handle whatever processing is desired. Great way to prototype. Then a commercial solution could take the various components and integrate them onto a single small PCB, while eliminating all of the unnecessary headers and USB connectors. The only off-board connections are power (2) and camera (3, power plus video, so the camera cable need only go to a single point), presuming that the antenna has its own connector or soldered coax.
This is great, except for the following. Many configurations run the NTSC camera output through the (separate PCB) flight controller, and then the flight controller outputs NTSC to the video transmitter. This allows the flight controller to superimpose OSD text on the NTSC as I mentioned earlier in this thread. Pilot vision is entirely within the goggles, so they need OSD to monitor battery status and other real time flight data that the camera and video transmitter have no way of knowing. This is one reason why sticking to NTSC is quite important... the entire infrastructure of the sport, from on-aircraft avionics to the ground support monitors used by judges and crew, rely on the use of NTSC. I suppose this new proposed video transmitter could accept additional (analog?) inputs and do the OSD itself, but flight controllers would have to be redesigned to emit such signals AND that doesn't address the extensive existing ground equipment.
-
It would be spamming doing a lot of replies:
So you say that LCD does not take picture sequentially? - Meaning controller takes whole frame into RAM, then frame miraculously in no time appears on LCD screen? :-DD
Just searched YT for Arduino LCD topic to show how LCD update happens in slo-mo:
https://youtu.be/X8syt55ITUo?t=267 (https://youtu.be/X8syt55ITUo?t=267)
-
One thing is the sensor, another is the display.
The sensor output is 'serial' - the display output is 'not'.
Are you sure? :)
You're talking about the display INPUT, I presume
The output is being outputted, until new information is stored in display memory - and the screen refreshed - showing the new output for a 'long' time, much longer than it took for the data to inputted
-
You're talking about the display INPUT, I presume
The output is being outputted, until new information is stored in display memory - and the screen refreshed - showing the new output for a 'long' time, much longer than it took for the data to inputted
Before you spread your wizdom, make sure you got it right
[edit] I talk about this BS where you supposedly tell your beliefs about how LCD works:
I hope you get where I am pointing at, as the screen has to be 'stationary long enough to show the entire picture', and then show the 'entire next picture long enough'
-
I would say hardware wise, it is not impossible - it should be rather light, inexpensive, and easy to obtain.
Very promising, if the hardware can handle whatever processing is desired. Great way to prototype.
This is great, except for the following. Many configurations run the NTSC camera output through the (separate PCB) flight controller, and then the flight controller outputs NTSC to the video transmitter.
Sorry to cut some of your quote - I hope it's ok.
I have been working with the MinimOSD myself, a mix of an Arduino and the MAX7456 OSD chip. It didn't start out that way ^-^
http://ardupilot.org/copter/_images/MinimOSD_Micro.jpg (http://ardupilot.org/copter/_images/MinimOSD_Micro.jpg)
If a SBC is used to generate video (it interfaces directly to a digital camera), what is stopping it from measuring other vital flight information (Gyro, Battery voltage, etc) - which is then superimposed onto the generated video image - which is then sent to the pilots FPV goggles.
It is essentially a flight computer, capable of video+information handling - just like your existing system
-
You're talking about the display INPUT, I presume
The output is being outputted, until new information is stored in display memory - and the screen refreshed - showing the new output for a 'long' time, much longer than it took for the data to inputted
Before you spread your wizdom, make sure you got it right
[edit] I talk about this BS where you supposedly tell your beliefs about how LCD works:
I hope you get where I am pointing at, as the screen has to be 'stationary long enough to show the entire picture', and then show the 'entire next picture long enough'
How Not To Overclock:
https://www.youtube.com/watch?v=L8gLig2PnSs (https://www.youtube.com/watch?v=L8gLig2PnSs)
-
Because you can't use inter-frame compression (due to latency)
If you use 8x8 blocks, the only necessary latency is buffering 8 lines ... 0.19 ms at 720p60.
8x8 blocks is intra-frame compression surely? Or have I got my terminology completely wrong here.
What I meant was that most of the redundant information in a raw video stream is in the time domain, the parts of the image that either do not change from one from to the next or change only by simple translations. Any compression algorithm that accounts for that has to delay by a full frame, and a compression algorithm that doesn't can't remove the information that is redundant in time.
Oh, I see what I missed. Because you're comparing your 8x8 block to the previous frame is does not introduce additional delay beyond waiting for 8 lines, or a little more if that block has translated vertically. Good point.
-
How Not To Overclock:
So what? That picture do not even apply to what we are talking about. Most LCD displays immediately show pixels as they arrive through interface. If you wonder what VSYNC and HSYNC signals are for - they are pointer/coordinate reset so both LCD and controller agrees where on the screen pixel contents will be going to. Watch YT video I mentioned
-
I do not know if the Raspberry Pi is good enough to handle video encoding at breakneck speeds
No it's not, there's a popular project using the combination of Pi+wifi and while it works it is not satisfactory in terms of latency. Main reason is that the whole video pipeline and GPU of the Pi is completely closed so there is nothing in there you can optimize, your only way through that is call the provided API function to get a frame from the "black box", with unnecessary buffering. Most SoCs on popular SBCs have the same problem of either closed libs and/or unobtainium documentation.
So you say that LCD does not take picture sequentially? - Meaning controller takes whole frame into RAM, then frame miraculously in no time appears on LCD screen? :-DD
Just searched YT for Arduino LCD topic to show how LCD update happens in slo-mo:
https://youtu.be/X8syt55ITUo?t=267 (https://youtu.be/X8syt55ITUo?t=267)
That's a small, low performance SPI module, you just populate its internal buffer in the order you want and at the speed you want.
Normal fast high res displays (such as the ones with the common parallel 24bit RGB interface) of course take the data sequentially but also in order and at constant refresh rate, you can't just stop sending stuff while you wait for the next line or such, if you don't respect timings you get what is shown in the video above.
The standard way that computers and display controller boards use is to hold usually 2 full frame buffers that are sent to the screen alternatively using DMA at the screen clock aka out of sync with the incoming data so that the controller can populate the inactive buffer while the other is sent to avoid screen tearing and artifacts, but that obviously introduces 1-2 frames of latency.
Since the display panel is in the end the one that's driving the timing really minimizing latency would require lots of syncing so that the whole camera, encoder, transmitter and receiver work just right so the data arrives at the point where it is to be sent to the LCD as close as possible to the moment where it is to be sent to the display panel, with minimal buffering with only small FIFOs containing a fraction of a frame between each component instead of one or multiple frames. And some fast, logic-driven (hence FPGA again) that is capable of "filling in" the data to be sent to the display when a packet is missing at a small multiple of the pixel clock which is typically 25ns on a 800x480 panel.
That's what makes it hard to use off the shelf stuff. Almost all of it just puts buffers between each step so as not to have to bother with that, and it's why you get >100ms latency with most standard camera->computer->RF->computer->display chains.
-
Normal fast high res displays (such as the ones with the common parallel 24bit RGB interface) of course take the data sequentially but also in order and at constant refresh rate, you can't just stop sending stuff while you wait for the next line or such, if you don't respect timings you get what is shown in the video above.
Right. So this is essentially how analog TV updates screen. Only difference is digital input and "infinite persistence" due to built-in frame shadow buffer for refresh purposes (LCD's with built-in controller). I was opposing TheDane who implied that you shall put whole frame into LCD and only then it can be shown, which is untrue:
The sensor output is 'serial' - the display output is 'not'. I hope you get where I am pointing at, as the screen has to be 'stationary long enough to show the entire picture', and then show the 'entire next picture long enough'
-
Hey all, I have decided that I'm going to put this project on hold on my end. You guys have brought up so nsny good points that i had not considered that make this project damn near infeasible. Feel free to continue discussion, as I feel it is very constructive.
Not infeasible, just a lot of work. Lets say you go with the approach I suggest, here's roughly how ignoring unforeseen roadblocks I'd chop up the tasks :
- get preempt_rt and camera working on SBC and find out how to get low latency access to the framebuffer (unfortunately the H3 boards I mentioned before won't support a 60fps camera, just use a Raspberry Pi 3 for now)
- get EZ-USB and TLC7528 DAC working with SBC
- create monochrome NTSC signal from camera input, which is not that hard (http://bitluni.net/esp32-composite-video/). Feed it to the transmitter and you can fly at this point
- get EZ-USB and AD9200 ADC working with laptop running Linux with preempt_rt (easy to convert to SBC later on, but save yourself the headache for now)
- connect the ADC to the appropriate trace in the receiver, decode monochrome NTSC and put it on screen
Then you are ready to start experimenting with encoding, FEC, multi-path correction and compression. Then at the end of the line you can port the work to a lower weight SBC or even a FPGA.
-
Lots of confusion about how LCD display modules work.
The common graphics ones often have a few operating modes, typically you can have a microprocessor interface to a frame buffer, or an (often) 16/18 bit interface with Hsync and Vsync that may or may not involve the frame buffer depending on what you want to do, finally there is sometimes a microprocessor style option that also uses Vsync. MIPI, DSI and the like are basically Hsync/Vsync style interfaces just done over a serdes.
The tradeoff is that in non frame buffer mode you typically cannot do rotation in the screen itself, and quite a few of these screens are annoyingly setup for tall and narrow as their native scanning format. Obviously a rotation involves a frame of latency any way you cut it (and more typically two frames of latency).
For real low latency video there is still something to be said for a an old Sony BVM series grade 1 CRT....
73 Dan.
-
Marco just described complex, multimillion $ worth SDR development project like it's nothing. In short: if you have to develop digital radio transceiver - better drop whole "Digital FPV" project right now. If you can live with "off the shelf" digital radio ASIC (or module), then video processing part can be tried by bunch of dedicated developers with loads of free time.
unfortunately the H3 boards I mentioned before won't support a 60fps camera, just use a Raspberry Pi 3 for now
It is good idea to start with widely available stuff - rPI, it's camera and WiFi dongle to use it as prototype development system. BTW some forum chat says that rPI camera can run faster than 60FPS. Using existing software solutions/libraries you can build proof of concept remote display - to get taste of that huge pile you will be digging through.
Then at the end of the line you can port the work to a lower weight SBC or even a FPGA.
Right. I would plan FPGA from very beginning, use rPI just as development system for "low latency FPV video codec" and develop it with FPGA in mind. Video 480p or even 720p, >= 120FPS, camera with IMX219PQ or similar, FPGA (Zynq) for video compressor/decompressor, 5.8GHz 802.11ac (half-miniPCIe module) for transmission channel. When video development finished and it is worth to push radio latency into sub-1ms, *then* WiFi bells and whistles can be stripped down to just essential data transmission with minimal overhead, yet this alone is very complex task and most likely separate project on its own.
-
Marco just described complex, multimillion $ worth SDR development project like it's nothing.
You would be using an existing transmitter and receiver, you aren't touching anything over 6 MHz. The ADC/DAC's I mentioned are 1$ parts ... this is more equivalent to a RTLSDR software project than anything else.
Neither the existing transmitter nor receiver really cares about the signal being NTSC, that's why the original idea had merit.
Everything is so much more difficult in FPGAs, starting from attaching the camera. There's no real light weight Zynq boards and designing one is a big undertaking in and of itself. If you're doing compression on the ARM, what's the point of a FPGA any more any way? FEC does 100s of Mb/s on these kinds of processors, where exactly do you see the need for FPGA at all? The only thing a FPGA makes a little more compact is attaching the DAC/ADC, since you can forego the EZUSB workaround.
-
You would be using an existing transmitter and receiver, you aren't touching anything over 6 MHz. The ADC/DAC's I mentioned are 1$ parts ... this is more equivalent to a RTLSDR software project than anything else.
You most likely will *not* use existing transmitter and receiver, unless you want to have hilariously bad sensitivity specs or hilariously slow speed. Their SNR, linearity and phase noise specs are insufficient for high order modulations at high symbol rate resulting in > 2bits/1Hz spectral density. You keep persistently dreaming your uneducated dreams about plugging ADC and DAC into 6MHz baseband of video transceiver and easily demodulating 20Mbit or so, data with same ease as RTLSDR is showing spectrum and playing FM radio. All this to addition of video decompression using same CPU's BTW. You really shall study much more about what it takes to develop functioning hi-speed demodulator of digital radio.
Everything is so much more difficult in FPGAs, starting from attaching the camera.
Well.. obviously. After all you shall compress video coming at 640*480*120*2 = 73.7 MBytes second (120Hz, 16 bit color). 720p will require even higher bandwidth. H.264 video compressors of rPI uses GPU as coprocessor, but as I know, GPU of rPI is not Cuda compatible. So - good luck with rPI GPU development. I would love to see how easy as you say, is compression of such video stream using just ARM cores.
If you're doing compression on the ARM, what's the point of a FPGA any more any way?
Why would you suggest that in FPGA solution I offered video compression using ARM? It is kinda dumb to have FPGA but not use it :D
The only thing a FPGA makes a little more compact is attaching the DAC/ADC, since you can forego the EZUSB workaround.
That EZUSB workaround for ADC/DAC with SDR modem is no better than common USB WiFi dongle with proper radio/modem ASIC.
-
Well.. obviously. After all you shall compress video coming at 640*480*120*2 = 73.7 MBytes second (120Hz, 16 bit color). 720p will require even higher bandwidth. H.264 video compressors of rPI uses GPU as coprocessor, but as I know, GPU of rPI is not Cuda compatible. So - good luck with rPI GPU development. I would love to see how easy as you say, is compression of such video stream using just ARM cores.
The cores are unlikely to be fast enough especially not given the latency requirement (can't make good use of the multiple cores by spliting tasks), and as said the Pi's video pipeline / encoder and GPU are "black boxes" so you can't do fancy stuff with them like encoding a 8px high line from the frame at a time, you've fot to feed it one of the standard supported frame sizes as a whole.
But while a high res full frame encoder takes a lot of FPGA fabric we have the advantage that since we precisely want to encode small portions the encoder block can likely fit in a smaller area.
-
20Mbit or so
That's about an order of magnitude off from the last bandwidth I mentioned for video compression.
All this to addition of video decompression using same CPU's BTW.
Compression tends to be the harder part.
H.264 video compressors of rPI uses GPU as coprocessor, but as I know, GPU of rPI is not Cuda compatible. So - good luck with rPI GPU development. I would love to see how easy as you say, is compression of such video stream using just ARM cores.
It might be possible to encode slices with the videocore without adding too much latency (part of the OpenMAX API AFAICS). So say encode 128 lines at a time as a slice, that way you don't have to queue up an entire frame. Would require some experimentation. Still easier than getting video coding to work on a FPGA.
That EZUSB workaround for ADC/DAC with SDR modem is no better than common USB WiFi dongle with proper radio/modem ASIC.
It fulfils the design parameters set in the OP, in that it is better.
-
Who needs sync signals every line when you have crystals? You can throw a reference pulse in there every couple of ms and calculate the path response and correct for it, etc.
Crystals are NOT perfect, and will change frequency when subjected to g-changes (acceleration, deceleration, rotation). Ah relativity :box:
Late last night I remembered about MEMS - Microelectromechanical system oscillator - https://en.wikipedia.org/wiki/Microelectromechanical_system_oscillator (https://en.wikipedia.org/wiki/Microelectromechanical_system_oscillator)
When compared to a crystal, a MEMS oscillator is relatively immune to random vibration and has much better phase and jitter specs.
This might be a good replacement for the existing crystal oscillators on already flying drones, to improve video quality?
Some tests and conclusion - page 10 is about random vibration:
www.sitime.com/api/gated/AN10032-Shock-Vibration-Comparison-MEMS-and-Quartz-Oscillators.pdf (http://www.sitime.com/api/gated/AN10032-Shock-Vibration-Comparison-MEMS-and-Quartz-Oscillators.pdf)
Oh, just don't use them with your helium filled blimps - they will crash
https://www.youtube.com/watch?v=vvzWaVvB908 (https://www.youtube.com/watch?v=vvzWaVvB908)
-
20Mbit or so
That's about an order of magnitude off from the last bandwidth I mentioned for video compression.
2Mbit was not just example but actual offer of data bitrate? 640*480*120*2=73MByte/sec is 584 Mbits/sec coming out of camera. Even half that (8bit grayscale) is huge number. You offer to compress video with fast vertical panning using 292:1 compression? - You sound like those 21-st century teens who declare that there's no need for education/experience/skills other than trained thumbs and internet :)
I have to remind that 120fps h.264 cannot reach better latency than analog TV transceivers (60Hz half frames). By 100% saturating data channel you are adding one more frame time, so absolutely best delay of such h.264 120FPS system would be 3/120=1.5/60 sec while analog NTSC TV currently have better latency of 1/60 sec!
-
There is a lot of redundancy from one line to the next, so some sort of line-to-line delta compression might be worth investigating, as it could be very simple to encode and decode.
Also may be some mileage in multiple receivers working on a line-by-line basis, so as long as one receiver gets a good line, the whole frame will be correct. Receivers linked with gbit ethernet using raw packets (not TCP/IP) ought to be fast enough to handle this
-
There is a lot of redundancy from one line to the next, so some sort of line-to-line delta compression might be worth investigating, as it could be very simple to encode and decode.
Yes, indeed similar to h264 motion estimation can be used, but then low latency compressor becomes exponentially complex task.... unless video is ~80% centered crop of the sensor area, each frame is compressed in shifted coordinate system to match previous frame - so redundant image parts (if any) can be omitted, yet later frame is displayed in original coordinates. Kind of whole frame motion estimator. We may be on to something here.
[edit] This is more or less how digital steadycam is working, obviously it does not do "displayed in original coordinates" ;)
Also may be some mileage in multiple receivers working on a line-by-line basis, so as long as one receiver gets a good line, the whole frame will be correct. Receivers linked with gbit ethernet using raw packets (not TCP/IP) ought to be fast enough to handle this
Yes, "digital diversity" shall be included into feature list. Thou we talked about 8pix vertical slice so (M)JPEG compression can be used. For 480p video slice could be vertical 16pix so whole frame is split into 30 packets. At 20Mbit & 120 FPS it is ~694 bytes/packet.
-
You offer to compress video with fast vertical panning using 292:1 compression? - You sound like those 21-st century teens who declare that there's no need for education/experience/skills other than trained thumbs and internet :)
You sound like someone who hasn't actually tried to convert some action cam footage to 2 Mbps. It's not pretty, but then neither is NTSC. That's at less than 1 bit per symbol, which is a tad pessimistic ... noise isn't that high in the NTSC footage.
By 100% saturating data channel you are adding one more frame time
Nah, as I said you can probably slice encode. Get X lines, encode as slice, put on wire, decode as slice ... that's <3X lines delay.
120 fps isn't all that important, since displays which can actually do it are so sparse. The mobile phone panels which can do it are unobtainium, there's just some expensive square panels for VR glasses ... or you need to move up to 15.6" laptop panels.
-
You offer to compress video with fast vertical panning using 292:1 compression? - You sound like those 21-st century teens who declare that there's no need for education/experience/skills other than trained thumbs and internet :)
You sound like someone who hasn't actually tried to convert some action cam footage to 2 Mbps.
Yes, I admit that I did not try to compress 480p 120fps action cam footage into 2Mbps. Did you? Really? - You sound like you tried everything.
Nah, as I said you can probably slice encode. Get X lines, encode as slice, put on wire, decode as slice ... that's <3X lines delay.
This is not how h264 you are promoting, works.
120 fps isn't all that important, since displays which can actually do it are so sparse.
It is. You may need to re-read (or comprehend) my posts to understand why.
-
Would it not be easier to take the raw digital stream from the camera, encode it with something like Manchester 2 or similar telephone style 3 or 4 level polarity independent scheme, ie AMI or NRZI , transmit a sync word, and send keyed, AM, FM, Whatever, it with one of the sidebands suppressed by a simple filter? ie Vestigial Sideband like an old TV modulator... In other words, just MAKE IT PSEUDO ANALOG? Decode with a Costas Loop if needed, etc.. If need be mix with a pseudo-random stream to make it spread spectrum... You would have no error correction, no multipath protection, bandwidth is freaking huge, I know the downsides... Then just do diversity receive using a pilot carrier...
Might just work. I've seen far crazier stuff done in Ham Radio to get video across.
Expecting inbound laughter... in 5,4,3,2,1 ... But very low latency...
Steve
-
One of the requirements is having 8 systems working together in a relatively narrow ISM band, so "freaking huge" bandwidth won't cut it.
BTW that already exists (WHDI-like systems) but while RF latency is almost 0 the use of standard input/output interfaces and camera/displays still introduces quite a bit of it in the total. And the transmission performance is meh at more than short distances.
-
Yes, I admit that I did not try to compress 480p 120fps action cam footage into 2Mbps. Did you? Really? - You sound like you tried everything.
vlc -I dummy -vvv "xxx.mp4" --sout=#transcode{venc=x264,fps=60,vcodec=h264,vb=2048}:standard{access=file,mux=ts,dst=yyy.mp4}
Downloading a 60 fps movie off youtube is a bit tricky, all the new firefox extensions suck ... but YouTube Video and Audio Downloader can do it (you have to play a bit of a guessing game which it is).
This is not how h264 you are promoting, works.
Yes it does (https://en.wikipedia.org/wiki/Network_Abstraction_Layer). Slices are in principle designed for parallel encoding/decoding, but they work fine to reduce latency too, x264 went down the same path when they were paid to create a low latency encoder for network gaming.
It is. You may need to re-read (or comprehend) my posts to understand why.
I know why higher framerate lowers latency, but it's simply not an option for now in a system at sensible cost/size. So framelocked encoding/decoding chasing/leading the scanline at 60 Hz is the best you can do.
-
Would it not be easier to take the raw digital stream from the camera, encode it with something like Manchester 2 or similar telephone style 3 or 4 level polarity independent scheme, ie AMI or NRZI , transmit a sync word, and send keyed, AM, FM, Whatever, it with one of the sidebands suppressed by a simple filter?
To do it cheaply it's by far the easiest to use the existing 5.8 GHz chipsets ... in which case you get an amplitude only input/output for a FM encoder/decoder.
For what it's worth, the RTC6715 seems pretty good at AGC at least, just out of hand rejecting it as crap is presumptuous.
-
Hey again! I just had a fantastic idea, let's see what will be thought of it! Pure digital is for sure not going to be very easy to accomplish, but what if we just improved the analog standard? Obviously ntsc has a lot of issues when it comes to how color is encoded, and I could fix that, but I have a better idea. What if we were to apply the discrete cosine tranform to 8x8 blocks, but them transmit them as an analog signal? We could assign more time to the more important parts of the image (IE: low frequencies and brightness) and on the other end, we could do a ton of over sampling. This would result in high frequency parts of the image being lost first, giving the effect of higher levels of jpeg compression as signal quality is lost. Thoughts?
-
So framelocked encoding/decoding chasing/leading the scanline at 60 Hz is the best you can do.
What's the point of building 60Hz digital system if by design it's latency cannot be lower than existing NTSC (60Hz half-frames)?
[edit] Thank you for h.264 slices info. It makes sense to use just "standard" h264 for this application. Most likely you wanted to post following link: https://en.wikipedia.org/wiki/Flexible_Macroblock_Ordering (https://en.wikipedia.org/wiki/Flexible_Macroblock_Ordering). I wonder - does rPI h264 encoder have slicing support?
-
What's the point of building 60Hz digital system if by design it's latency cannot be lower than existing NTSC (60Hz half-frames)?
Even if you just did a pure PCM YUV encoding it would be an improvement on NTSC, it's not bandwidth efficient. The blackbox synch algorithms can almost certainly be improved upon too.
-
but what if we just improved the analog standard? Obviously ntsc has a lot of issues when it comes to how color is encoded,
Just switch the camera to PAL. :)
For a not too difficult thing to try, I quite like the idea of PLL recreating the video syncs between the RX and LCD monitor, so that they're still there for the monitor during the bad video to no video times. It's in this post:
https://www.eevblog.com/forum/projects/digital-fpv-video-for-drone-racing/msg2139877/#msg2139877 (https://www.eevblog.com/forum/projects/digital-fpv-video-for-drone-racing/msg2139877/#msg2139877)
-
Thoughts?
I'd spend a quarter of the bandwidth on interlaced HQVGA YUV at 60 Hz, then the rest on digital. If digital fails for a block interpolate the "analogue" signal.