Author Topic: Need to build On-screen display for VGA  (Read 5417 times)

0 Members and 4 Guests are viewing this topic.

Offline issoTopic starter

  • Contributor
  • Posts: 31
  • Country: us
Need to build On-screen display for VGA
« on: February 05, 2019, 04:13:21 pm »
I need to build an On-screen display module that will sit in between of a graphics adapter and monitor with VGA input and will modify the RGB video stream by inserting simple graphics or characters. Target resolutions are up to 1920x1080. Are there any specialized ICs for this purpose? I found STV9937 which seems to fit the job, however it's discontinued. Or any other ideas how to implement this using general purpose FPGA or MCU? Thanks
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 5050
  • Country: si
Re: Need to build On-screen display for VGA
« Reply #1 on: February 05, 2019, 04:31:54 pm »
For 1080p it will be indeed difficult to find purpose built chips for the job, but it is a pretty easy task for a FPGA.

The FPGA just has to look at the Hsync and Vsync signals to determine what location on the screen is being drawn at. When it wants to draw something at that location you can have a digital output pin flip a fast analog MUX chip to disconnect the original video stream and instead connect the output to the FPGA pins where you generate the color you want to show. You can get 8 available colors in the OSD text by just using 3 digital pins each driving its own color, or if you add a simple 3 bit R2R DAC cirucit to the pins you can push that to 512 colors.

Pretty ideal task for a FPGA really.
 

Offline issoTopic starter

  • Contributor
  • Posts: 31
  • Country: us
Re: Need to build On-screen display for VGA
« Reply #2 on: February 05, 2019, 04:56:35 pm »
Thanks for fast response Berni,

I have no experience with FPGA, so maybe a dumb question, but would the solution that you described require very fast RAM where the OSD data would need to be stored?
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 5050
  • Country: si
Re: Need to build On-screen display for VGA
« Reply #3 on: February 05, 2019, 05:14:12 pm »
Well expect a significant learning curve in programing FPGAs then, but it is a useful skill to have in your toolbox, so if you are not on a tight deadline its worth giving it a shot.

The fastest RAM is the one inside the FPGA itself and its way way faster than needed for 1080p video. As for if you need external RAM is mostly a question of how much RAM you need. Reasonably priced FPGAs tend to have about 10 to 300 KB of built in RAM. So if you want to overlay a picture into the video that is 1920x1080 in 24bit color you will need external RAM to hold it as that's about 6MB. But if you quadruple each pixel(use 1/4 the resolution of 1080p for the OSD) that brings it down to 389KB, if you settle for 256 colors that brings it down to 128KB.

But FPGAs also have a significant amount of processing power available so you can actually generate the OSD image on the fly as needed. This means that the FPGA can hold a array of ASCII text and process it into a OSD image as its being drawn from a font array. This brings the memory requirement to 20KB for a 100x200 field of text and about 32KB to hold a reasonably high resolution bitmap font. This might make it fit into a even smaller and cheaper FPGA and if the text is being updated by an outside MCU it saves it a lot of work because it can just spit a string at it rather than drawing text by flipping individual pixels in the FPGAs memory. Pretty much all computers from the 80s used this trick because it was easy to do and it saved them a lot of RAM and CPU power to get things on a screen.
 

Offline issoTopic starter

  • Contributor
  • Posts: 31
  • Country: us
Re: Need to build On-screen display for VGA
« Reply #4 on: February 05, 2019, 05:42:36 pm »
Thanks a lot, that's very useful information.

One more question: do you think any MCU could handle the same task? I realize it would require a very fast MCU, however maybe with some tricks (like those you described) and additional ICs the requirement can be lowered?
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 5050
  • Country: si
Re: Need to build On-screen display for VGA
« Reply #5 on: February 05, 2019, 06:41:09 pm »
Yes it can, but might require some clever coding to meet the timing requirements at 1080p.

Hardware timers can be used to trigger from Hsync and Vsync signals to provide an accurate interrupt call at the time the correct location is being drawn to the screen, this might then trigger a DMA transfer that shovels a large RAM array into a GPIO port where you connect your MUX signal and OSD color signals. Perhaps using 8bits on the GPIO port to have 1bit MUX signal, 2bit red, 3bit green, 2bit blue. The speed of the DMA could be tuned to make 1 transfer for every 4 pixels drawn to make fat long pixels and the DMA could be triggered to pump out the same array 4 times in a row to make 4x4 sized OSD pixels on the screen.

While the DMA is pumping out pixels this gives the CPU 4 video lines worth of time to fill up another array in RAM with the data for the next set of lines. Once the 4 lines are done the DMAs source address is then switched to the new array location to pump out the new pixels while the CPU starts preparing the next one, this repeats until the whole video frame is over and once vertical blanking starts the CPU now has a few microseconds of time to do anything else it needs to do, like perhaps user interface stuff, once a new video frame starts being drawn it all repeats again.

How much resolution and color depth you can afford depends on how well the code is optimized and how fast the CPU is. Maybe you need huge 16x16 sized pixels to have enough CPU time to do it. Or maybe you optimize the hell out of it and use the SPI bus running at 130MHz to get high speed output and get 1x1 sized pixels but only in a single color. Depends on what you optimize for.

Another trick that might be possible is to get a very powerful MCU that is capable of outputting 1080p video on a 24bit RGB bus. This bus is designed to connect a LCD to it, but with some hardware interrupt and register twiddling trickery you can perhaps synchronize the start of the frame on this video output bus with the start of frame on VGA. In that case you can overlay a high resolution 24bit color 1920x1080 picture over the original video by simply using MUXing between the video coming from the MCU and from VGA by using one of the color bits as the "transparency flag" to switch the mux at the right moment. Tho chips that can output 1080p video tend to also run Linux
 

Offline issoTopic starter

  • Contributor
  • Posts: 31
  • Country: us
Re: Need to build On-screen display for VGA
« Reply #6 on: February 05, 2019, 06:54:30 pm »
Wow, that was some awesome ideas! Thanks a lot!
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15796
  • Country: fr
Re: Need to build On-screen display for VGA
« Reply #7 on: February 05, 2019, 07:01:18 pm »
An FPGA would probably be the best fit here.

But since you are dealing with VGA, which is purely analog, I guess you don't really have to neither decode the full video input nor generate a video output at up to full hd video clock.

You may be able to use even a modest MCU, use the  HSYNC and VSYNC signals to synchronize your picture generator (which can run at a much lower clock rate than the original video), and mix the input and output video (R, G, B signals) on a purely analog level. Wouldn't that work?

Edit: for instance, with this kind of IC: https://www.analog.com/en/products/lt1675.html
this should be relatively straightforward if you don't need transparency, but opaque OSD is alright? It's a fast MUX. You could switch between your own RGB signals and the source signals, and it has integrated output amplifiers. Sounds doable with a reasonable MCU.
« Last Edit: February 05, 2019, 09:22:30 pm by SiliconWizard »
 

Offline issoTopic starter

  • Contributor
  • Posts: 31
  • Country: us
Re: Need to build On-screen display for VGA
« Reply #8 on: February 06, 2019, 01:50:55 pm »
Hi SiliconWizard,

Thanks for your suggestion, and LT1675 looks like a very useful component. I was thinking along the same lines re MCU, however I agree than FPGA is a better choice, partly because I need high quality OSD picture which is hard to get with MCU.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9319
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Need to build On-screen display for VGA
« Reply #9 on: February 06, 2019, 02:39:26 pm »
What's your budget? You can go as low end as the ICE40 series of FPGAs or as high as the Artix 7 series, the latter of which can do 1080p HDMI. There are also HDMI ASICs with overlay support but they tend to not be well documented to the general public. (Is there a specific reason to use VGA? It's been obsolete for years and some of the latest GPUs don't even have analog outputs anymore.)
Another trick that might be possible is to get a very powerful MCU that is capable of outputting 1080p video on a 24bit RGB bus. This bus is designed to connect a LCD to it, but with some hardware interrupt and register twiddling trickery you can perhaps synchronize the start of the frame on this video output bus with the start of frame on VGA. In that case you can overlay a high resolution 24bit color 1920x1080 picture over the original video by simply using MUXing between the video coming from the MCU and from VGA by using one of the color bits as the "transparency flag" to switch the mux at the right moment. Tho chips that can output 1080p video tend to also run Linux
Has anyone managed to genlock the parallel RGB output of a Raspberry Pi to an external source?
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline Scrts

  • Frequent Contributor
  • **
  • Posts: 800
  • Country: lt
Re: Need to build On-screen display for VGA
« Reply #10 on: February 06, 2019, 03:04:08 pm »
Look into TechWell (now Renesas?) chips. They have video processors that do video overlay - either graphics or text. I had TW8836 evaluation kit and it worked well. See here: https://www.renesas.com/us/en/www/doc/datasheet/tw8836.pdf

The TW8844 is widely used in the vehicles for overlay generation of text and graphics:

 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #11 on: February 06, 2019, 04:04:33 pm »
Yes it can, but might require some clever coding to meet the timing requirements at 1080p.

Hardware timers can be used to trigger from Hsync and Vsync signals to provide an accurate interrupt call at the time the correct location is being drawn to the screen, this might then trigger a DMA transfer that shovels a large RAM array into a GPIO port where you connect your MUX signal and OSD color signals. Perhaps using 8bits on the GPIO port to have 1bit MUX signal, 2bit red, 3bit green, 2bit blue. The speed of the DMA could be tuned to make 1 transfer for every 4 pixels drawn to make fat long pixels and the DMA could be triggered to pump out the same array 4 times in a row to make 4x4 sized OSD pixels on the screen.

While the DMA is pumping out pixels this gives the CPU 4 video lines worth of time to fill up another array in RAM with the data for the next set of lines. Once the 4 lines are done the DMAs source address is then switched to the new array location to pump out the new pixels while the CPU starts preparing the next one, this repeats until the whole video frame is over and once vertical blanking starts the CPU now has a few microseconds of time to do anything else it needs to do, like perhaps user interface stuff, once a new video frame starts being drawn it all repeats again.

How much resolution and color depth you can afford depends on how well the code is optimized and how fast the CPU is. Maybe you need huge 16x16 sized pixels to have enough CPU time to do it. Or maybe you optimize the hell out of it and use the SPI bus running at 130MHz to get high speed output and get 1x1 sized pixels but only in a single color. Depends on what you optimize for.

Another trick that might be possible is to get a very powerful MCU that is capable of outputting 1080p video on a 24bit RGB bus. This bus is designed to connect a LCD to it, but with some hardware interrupt and register twiddling trickery you can perhaps synchronize the start of the frame on this video output bus with the start of frame on VGA. In that case you can overlay a high resolution 24bit color 1920x1080 picture over the original video by simply using MUXing between the video coming from the MCU and from VGA by using one of the color bits as the "transparency flag" to switch the mux at the right moment. Tho chips that can output 1080p video tend to also run Linux

Forget any SW solution at 1080p pixel clock speeds.  >:(

You need a dedicated HW for that and at 1080p data bandwidths, no SW hack will produce any good results.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 5050
  • Country: si
Re: Need to build On-screen display for VGA
« Reply #12 on: February 06, 2019, 04:21:42 pm »
Well with a software MCU solution obviously the OSD would be of lower resolution than the 1080p video its overlaying on. But it is perfectly possible with some skillful C programming and a good bit of time studying the MCUs documentation to abuse the peripherals in the right way People are doing similar stuff on a 8bit MCU while also generating sound trough PWM at the same time(Tho in those cases its all hand optimized assembler with instruction cycle counting)

But yeah if you want to overlay a 1920x1080 resolution full color OSD image onto 1080p 60Hz video it will be the easiest to just bite the bullet and use a FPGA to do it properly.

As for genlocking the video output of a Raspberry Pi, i think it would be very difficult to do because video output comes from the internal GPU that is running closed source firmware and drivers. There are some work in progress open source drivers to run the GPU, but they are quite complex stuff and im not sure if they go low level enough to allow you to trick the GPU to restart a frame on command. Also getting the sort of timing accuracy under the Linux OS can be rather tricky and timings in general are not very deterministic on these big boy ARM chips. There are lots of buses, memory controllers, caches, pipelining etc. that cause the execution speed and interrupt latency to jitter around. Smaller ARM chips execute directly from internal flash/ram and don't pipeline and as such can provide much more tightly controlled execution timing (in a lot of cases cycle accurate). To pull off this sort of trickery you likely require cycle accurate execution timing or your image would shake all over the place.
« Last Edit: February 06, 2019, 04:23:39 pm by Berni »
 

Offline GeorgeOfTheJungle

  • Super Contributor
  • ***
  • !
  • Posts: 2699
  • Country: tr
Re: Need to build On-screen display for VGA
« Reply #13 on: February 06, 2019, 04:23:46 pm »
1080p @60Hz HSYNC frequency is ~ 1080*60= 65 kHz, pixel clock ~ 1080*60*1920= 125 MHz, 1080p @30Hz half that: 32.5 kHz and 62.5 MHz.

All you need is one parallel in serial out shift register than can run at those pixel clock frequencies (*). Or two...

Any µC can count HSYNCs @ 65 kHz, and load the shift register(s) between lines.

(*) Edit: not really, the OSD can run at its own pixel rate.
« Last Edit: February 06, 2019, 10:52:36 pm by GeorgeOfTheJungle »
The further a society drifts from truth, the more it will hate those who speak it.
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #14 on: February 06, 2019, 05:39:59 pm »
Shift registers = dedicated hardware.

I stay behind what I have stated. Forget SW solution.  Not gonna happen at these speeds, this ain't no PAL.  Even the slightest timing jitter will become clearly visible, rendering the result likely not very appealing.

Of course one can argue with "clever programing", clever this or that.  But dedicated HW will be probably much less complex as a solution and will give you much more flexibility.

Even though I am not any kind of  FPGA expert and I tend to stick to MCUs as much as possible, in this case I clearly recommend using a cheap FPGA or even larger CPLD to help with the job.

There are tons of cheap FPGAs available that would likely fit the job with less hassle and bucks spent, than a MCU with "just the right clever this or that peripheral abused" for the job.



 

Offline GeorgeOfTheJungle

  • Super Contributor
  • ***
  • !
  • Posts: 2699
  • Country: tr
Re: Need to build On-screen display for VGA
« Reply #15 on: February 06, 2019, 07:56:52 pm »
To make an OSD he's going to need a µC anyway.
The further a society drifts from truth, the more it will hate those who speak it.
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #16 on: February 06, 2019, 08:14:43 pm »
I do not think that making an OSD is a requirement for uC ;)

Text and graphics rendering can pretty sure be done even in the FPGA. Depends on the application of course.

Just wanted to make a point, that a a $5 FPGA would make for a pretty OSD overlay generator.  $5 MCU would be likely struggling to do the job properly, with you losing precious time on creating hacks over hacks to make that work.
 

Offline Scrts

  • Frequent Contributor
  • **
  • Posts: 800
  • Country: lt
Re: Need to build On-screen display for VGA
« Reply #17 on: February 06, 2019, 09:20:46 pm »
1080p @60Hz HSYNC frequency is ~ 1080*60= 65 kHz, pixel clock ~ 1080*60*1920= 125 MHz, 1080p @30Hz half that: 32.5 kHz and 62.5 MHz.

All you need is one parallel in serial out shift register than can run at those pixel clock frequencies. Or two...

Any µC can count HSYNCs @ 65 kHz, and load the shift register(s) between lines.

1080p pixel clock is 148.5MHz usually. I haven't seen a microcontroller that can handle such resolutions. Not even Spansion Traveo series.
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #18 on: February 06, 2019, 10:36:37 pm »
1080p @60Hz HSYNC frequency is ~ 1080*60= 65 kHz, pixel clock ~ 1080*60*1920= 125 MHz, 1080p @30Hz half that: 32.5 kHz and 62.5 MHz.

All you need is one parallel in serial out shift register than can run at those pixel clock frequencies. Or two...

Any µC can count HSYNCs @ 65 kHz, and load the shift register(s) between lines.

1080p pixel clock is 148.5MHz usually. I haven't seen a microcontroller that can handle such resolutions. Not even Spansion Traveo series.

That is for a rather unusual 85Hz refresh rate.   But anyway, the pixelclock is above 100MHz for sure, in any Full HD mode.

You will get in trouble trying to hack any MCU for making OSD, at such frequencies. Even the quite high performance MCUs (400MHz Cortex M7, with dedicated video hardware) are way slow for any image processing at such resolutions and pixel clock speeds.
You could barely find a peripheral, that could be hacked to accept a 100MHz+ external clocking.
Slapping together tons of glue logic (74 series and such) around the MCU will become a major pain at 100+ MHz anyway.

Not to begin with the fact that you would need a device to do the TMDS data deserialization and serialization. There aren't many devices on the market to do this. Many of them also closed under NDA.

The FPGA is the obvious solution here. Comparatively even cheap and quite simple to use.



 

Offline GeorgeOfTheJungle

  • Super Contributor
  • ***
  • !
  • Posts: 2699
  • Country: tr
Re: Need to build On-screen display for VGA
« Reply #19 on: February 06, 2019, 10:47:07 pm »
The shift register is the only thing that runs at the pixel clock frequency. And you can choose the pixel clock frequency, it doesn't have to be the same as that of the underlying image, can be whatever you want, a lower or higher pixel rate. The µC's job is to load the pixels into the shift register before the scan line begins, at HSYNC frequencies, it's easy-peasy: no assembly hacks needed whatsoever.

By the way, the OP wants RGB VGA, no LVDS, no TMDS, no things of that kind.
« Last Edit: February 06, 2019, 11:00:36 pm by GeorgeOfTheJungle »
The further a society drifts from truth, the more it will hate those who speak it.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 10385
  • Country: nz
Re: Need to build On-screen display for VGA
« Reply #20 on: February 06, 2019, 10:49:46 pm »
DMA of SPI bus running as fast as it can is the way to go if you want to make a MCU solution.

If you have 3 SPI buses you can even do RGB and get some colours.

This is the method used by ATMega's and STM32's to do an OSD over the top of composite video cameras for drones/RC planes.
VGA is similar except It's faster, the V/H sync is already broken out and colour is separated instead of a single B&W signal with a colour burst.

The trick is to code it in the right way so you have zero timing jitter.
If you use an interrupt it you will get jitter because instructions have different length. So whatever instruction is currently executing when the interrupt occurs has to finish and this timing changes depending on what the instruction was.

« Last Edit: February 06, 2019, 10:53:38 pm by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #21 on: February 06, 2019, 10:52:50 pm »
I am yet to see a comparatively cheap MCU, that can run SPI clocks at 100MHz, with the serdes circuitry around.

1080p TMDS is not analog PAL.  Forget the software hackery bullshit.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 10385
  • Country: nz
Re: Need to build On-screen display for VGA
« Reply #22 on: February 06, 2019, 10:54:58 pm »
In almost all cases you dont need to run the SPI at pixel clock frequency.
Most of the time you just want to draw some large lines/text and so your needed pixel size is much bigger than the native pixels size.
(I shouldn't really call them pixels. 1920x1080 is lines of 1920 divisions or 1920TVL)

Obviously you still need to fire the DMA for every line though. But that's triggered from the Sync.

As long as the MCU can keep up with the SYNC you could draw pixels on the display in 10x10 grid with quite a slow spi clock.

Its all about timing from the start of SYNC to the SPI output data

If your MCU has enough ram you can have all the SPI data in ram for the entire frame, that way you're not rushing to move anything around between DMA ops.
Then you can update pixels in ram as slow as needed for the MCU to keep up. Obviously you'd want to double buffer it.
« Last Edit: February 06, 2019, 11:08:05 pm by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3930
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Need to build On-screen display for VGA
« Reply #23 on: February 07, 2019, 06:14:26 am »
Fair point! However how will you keep the MCU clock (the SPI clock) in sync with the HDMI video stream?

You need to cross between two clock domains.  If you think your both clocks (MCU and HDMI) are perfect, then no, they aren't and we are back to what I have said: Jitter. You will get garbage data sampled at the imprecisely timed video data transitions.  Any vertical edges in the OSD will become jittery.

Keep in mind HDMI is a synchronous clock driven protocol, unlike PAL, where there is no requirement to the pixel clock be in sync with anything. The timing just needs to be without jitter - analog video has no pixels to begin with.
 

Offline Psi

  • Super Contributor
  • ***
  • Posts: 10385
  • Country: nz
Re: Need to build On-screen display for VGA
« Reply #24 on: February 07, 2019, 06:20:40 am »
When doing composite video it doesn't seem to matter to keep STM32 CPU clock and video pclock in sync.
I guess the jitter is too small to see.

But yes, i think at 1920x1080 you might run into at least some jitter issues due to the clocks drifting in and out of sync.
Maybe a 260mhz STM32F4-7 would be fast enough to be too small to see.

If you really wanted to you could PLL to the sync pulses and generate a fast clock to run the MCU from that.
« Last Edit: February 07, 2019, 06:25:01 am by Psi »
Greek letter 'Psi' (not Pounds per Square Inch)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf