Author Topic: HDMI image grab with microprocessors. Is it feasible? About HDMI Demystification  (Read 9375 times)

0 Members and 2 Guests are viewing this topic.

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Hi dear friends,

This will be my blog message about my ongoing learning curve about HDMI and electronics. I will send all of the useful materials i will come up with on this topic; of which i think that will help me and to other beginners in their research. 

I would like to ask you, if it will be possible to grab a single frame with the use of an Evaluation Board in the link below?

http://www.analog.com/media/en/technical-documentation/user-guides/UG-338.pdf

by using any type of a microprocessor's pins?

Any simple documentation on HDMI single frame receiving concept is also greatly appreciated

What i would like to achieve is to grab a single frame from my DVB-C STB's HDMI 1080HD output, with the use of any available pins of any type of a microprocessor.

Thank you very much.
« Last Edit: August 05, 2018, 10:46:13 am by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
An ongoing learning process ...
Which useful links i have found so far.

To understand the HDMI cable better, let’s first take a look at all the signals travel
inside one HDMI cable:
HDMI 1.4 Demystified
http://www.luxielectronics.com/attachments/File/HDMI_1_4_Demystified.pdf

HDMI Demystified
https://www.fpga4fun.com/files/HDMI_Demystified_rev_1_02.pdf

HDMI 2.0 and the push for 4k
http://luxielectronics.com/attachments/File/HDMI_2_0_and_the_push_for_4k.pdf

Pushing the 4K/UHD Digital Envelope 18Gbps and Beyond
https://nordost.com/downloads/Pushing%20the%204K.pdf

VHDL Implementation of TMDS encoder for the transmission of video signals in serial communication (International Journal of Advanced Research in Computer Engineering & Technology (IJARCET))
http://ijarcet.org/wp-content/uploads/IJARCET-VOL-4-ISSUE-4-1576-1579.pdf
« Last Edit: August 15, 2018, 09:02:26 am by donkey »
 

Offline dmills

  • Super Contributor
  • ***
  • Posts: 2093
  • Country: gb
I am not seeing HDMi over GPIO being possible (at least not directly), the data rates are stupidly high and I would be reaching for an FPGA to do the deserialisation and frame storage, then use whatever you like to get the data over into the micro.

Regards, Dan.
 
The following users thanked this post: NiHaoMike, donkey

Offline DaJMasta

  • Super Contributor
  • ***
  • Posts: 2298
  • Country: us
    • medpants.com
I sort of doubt this is possible as well.  You could get a chip that takes the HDMI bus of CML differential pairs and converts it to a wide LVTTL bus or something that would be signal compatible with a micro, but with 1080p60 video, you're still talking 3.2Gb/s data rates.... so even deserialized running at the minimum 165MHz character rate, it would take a pretty powerful micro just to be able to read the output.  This evaluation board uses a later hdmi standard with a faster data rate and includes two HDMI streams onto the single interconnect bus, increasing the speed and storage size demands.  Then you have to read enough of it in quick enough succession to store a frame of it it in memory - and then have to do any processing you need to get it into a usable form (though this part wouldn't necessarily need to be fast).

You could think of what you need similar to the frontend of a digital oscilloscope: you need something capable of a very fast data capture directly into memory on a trigger, then you need your micro to be able to sort through what was stored to find what you want.  Because of limitations of the speed of the micro, it's probably easiest to capture everything in a very quick storage subsystem, then have your micro sort through it after the fact, rather than trying to pass all the information through the micro at the full data rate.  Of course this means you will only be able to capture a second frame after the first in the time it takes for the micro to do what it needs and for the memory system and trigger to be reset, but short of the massive amount of computing realtime processing would take, it's what can be done.  A Pi probably wouldn't have trouble managing it, but I don't think the GPIO pins are suited for fast data transfer, so that's probably where your bottleneck would be.
 
The following users thanked this post: donkey

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9015
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
You could get a HDMI capture device that connects over USB or Ethernet.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
There's no way this will work with a microcontroller alone. Using a suitably capable FPGA it could be done but GPIO on a microcontroller will not be anywhere near fast enough.
 
The following users thanked this post: donkey

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Thank you very much for all of your answers.

I know that there are some HDMI capture cards to buy and to capture a frame.

The aim of this project is to understand the correct connection and communication procedure between an HDMI receiver IC and a system (FPGA, Microprocessor etc.) to manage to grab a single frame from it.

I will keep updating the post until i will find a successful way to manage to grab a single frame capture over HDMI.

Thank you!
« Last Edit: July 30, 2018, 06:26:41 pm by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
What i have found so far on the subject ...

TMDS33
DVI and HDMI specify that the same differential signalling is used on all the high-speed lines. Differential signalling is a very old and good technique for increasing signal to noise and resitance to common-mode noise. In the olden days transmission lines using this method were called "balanced".

Instead of sending the signal on one wire (referenced say to 0v), you send the signal on two wires, known as + and -. The - signal is the same signal as +, but inverted (so if + is 1, - is 0 and vice versa). When the signals are combined, "common-mode" noise, noise that appears on both signals the same, tends to be cancelled out. And when differential signals are routed for long lengths, the wires are twisted togther so their emissions also tend to cancel.

HDMI needs more bandwidth than can be sent on one differential pair. So they use 3 data pairs and one clock pair, plus some auxiliary signals. (Two of the auxiliary signals are also in a twisted pair physically, but they are not TMDS33 or related to the HDMI data stream.) In total there are 19 conductors in the familar HDMI cable.


HDMI Cable wiring

In DVI and HDMI the 3 data lanes and the clock lane are differential pairs using TMDS33 levels. TMDS requires 51R pullups to 3.3V at the receiver. However the differential voltages themselves are much smaller, on the order of 100 - 200mV. Smaller levels are quicker to reach and allow faster transmission rates.

The presence of these termination pullups is detected by the transmitter and it may suppress data transmission until they are seen, known as "receiver detection".

My 'scope is too weak to record the data faithfully at 742.5MHz clock used in 720p, but below gives you an idea of what the differential signals look like when pulled up to 3.3V and transmitting.


TMDS33 signal levels

It also shows how inadequate a 200MHz bandwidth scope is for looking at this, there are actually ten bit-times shown between the two vertical lines (roughly one 74.25MHz period). HDMI receivers (and the FPGA used here) have higher bandwidth inputs that can resolve the individual bits properly.

TMDS33 Channels

The DVI or HDMI cable itself carries the clock pair and three data pairs that transmit in parallel.


HDMI data transmission

Although the actual clock rate is very high, to ease carrying the clock on real cables and reduce RF emissions, the clock that is sent on the HDMI clock differential pair is 1/10th of the rate of the data on the data pairs. This makes the HDMI clock period represent one pixel period, in other words considering there are three data channels, there are 30 bits transmitted per HDMI clock period (== per pixel).

A PLL in the receiver reconstructs the x10 clock and uses it to capture the data on the channels.

TMDS codings
Unlike a VGA or earlier cable, there are no wires reserved to carry the sync signalling. Instead, syncs are carried as part of the 10-bits per pixel data from each channel, and they are carried differently according to what else is being sent.

The reason for this is that HDMI is derived from the earlier DVI standard, which had a very simple plan for carrying the syncs using reserved symbols during the whole of the blanking period. But HDMI builds on DVI by allowing new "data islands", using a different coding scheme, to randomly be carried during blanking as well as "control periods", and makes explicit "video periods" that contain the active video data: since DVI doesn't have these extra concepts it means HDMI ended up with two (syncs cannot change during active video data) completely different ways to express HSYNC and VSYNC state in the stream.


DVI + HDMI codings

The direct logical codings in HDMI then are

Active video data using 10b8b coding (providing 8b each for RGB in RGB mode)
Control periods using one of four special 10-bit control codings to express the 4 possible HSYNC + VSYNC states (10b2b)
Data islands using TERC4 (10b4b) coding
During the active video region where pixel data is being sent and 10b8b coding is used, there is actually a choice of two codings per byte. Once choice has more zeros and the other more ones. The transmitter selects which coding to use per pixel based on whether it has sent more ones or zeros lately, it keeps a running count and selects the coding to keep the ratio of ones to zeros at 50:50 overall.

TERC4 is a sparse 10b4b coding that is used to carry both HSYNC and VSYNC data and generic data such as HDMI audio samples. TERC4 has its own structure and subchannels, and inside the overall 36-byte packets sent using it, various packet types can be found (including PCM or other audio samples).

Although the Conrtrol Period reserved symbols are unique, actually you can't interpret HDMI data overall without parsing what has been going on before to eg, understand you are in an active video period or data island and track it using a state machine.

Types of stream sync in HDMI
At the high data rates of HDMI, skew between clock and data, or between data channels, or between differential pair members on a single channel or clock can destroy data integrity. These skews depend on the transmitter and cable as much as the receiver, and they vary somewhat with temperature.

So receivers typically have to hunt for the best clock phase with the least error rate... FPGAs have some support for either delaying to clock or individual data channels by small amounts to compensate. All high speed serial buses have some need for this including DDR, SD and PCIe who call it "tuning" or "training".

If we are able to collect bits with low error rate, we need to decide which on the 10 bits per HDMI clock is the first bit. This can be done by trying all 10 and finding out which exhibited the most valid control period symbols.

At that point we are aligned enough we can receive the 30 bit pixel data as it was sent.

The reserved control symbols are then used to align to higher level decoding state to track where we are in the raster, since they don't appear in either TERC4 or 10b8b data.

Refresher on generic video timing
The structure of video streams is still basically the same as used in the first black and white TVs, with CRT type displays.


Generic Video Timing

Basically, there are two kinds of period, active video (black) and blanking (grey). Normally only the active video part is shown on your display, but for our purposes, we are interested in capturing ALL of it.

Originally the blanking was used as the time required to swing the electron beam illuminating the CRT phosphor back to the start of the next line (on HSYNC) and back to the top left (on VSYNC) and it had no other purpose. On analogue TVs audio is sent separately on a different but related frequency. When Colour was added in analogue video, a chroma burst used to sync the phase of a chroma subcarrier was added in the back porch of each line. The sync pulses were encoded as different analogue voltage excursions not used by the video part.

Digital video formats kept the basic HSYNC and VSYNC timing and the concept of blanking intervals. So they still have front and back porches and HSYNC and VSYNC durations. One of the reasons the blanking interval won't die is because it allows you to tune the refresh rate of the video at a given pixel rate without changing the active region; this is how it is possible for HDMI to do both 720p50 and 720p60 using the same 74.250MHz pixel clock... the active video area is the same but the amount of blanking pixels is traded off with the time needed to send ten extra frames per second: 720p50 has an artificially large blanking period as you can see from a real capture below.


720p50 Active Region vs Blanking

It actually uses 1980px for each line even though only 1280px contain active video.

Once it was decided that there was an unused blanking interval basically as padding, in a digital protocol naturally minds turn to using the data passed there for something good. On DVI it simply passes 10b2b control period codes that encode the HSYNC and VSYNC state. On HDMI you can still do that, but you have the option to place "Data Islands" in the blanking (these are the red areas in the picture above). After error correction codes are removed, these consist of a 3-byte header and 4 x 7 byte subpackets.


Generic Video Timing

The three-byte header is defined in HDMI to contain a packet type, version and length information, and there are a list of packet types such as those carrying PCM Audio samples: at 24bps, the 4 x 7 bytes is enough to carry the 8 channels supporred by HDMI with 4 bytes left over.

Because HDMI sends these data islands using TERC4 (10b4b) on 2 of the three channels, and including error correction, 36 bytes must be passed, each data island packet "costs" 36 pixel-times of blanking plus some overhead to start and stop a data island in the protocol. During this time, the other data channel is used to send HSYNC and VSYNC state also in TERC4.

HDMI defined Data Island Packet Types
HDMI defines the following packet types... it's legal to just ignore them and act like it's DVI, if your HDMI sink doesn't need features like audio. However we are interested in audio and the other information.


0x83 (SPD) is quite interesting, the source device can send a vendor and product description in ASCII. On my laptop, it sends "Intel" and "IntegratedGfx".

Reference : https://warmcat.com/
« Last Edit: July 31, 2018, 09:40:04 am by donkey »
 
The following users thanked this post: ebclr, Richard Crowley

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Uncompressed HDMI Video Input to Raspberry Pi in OpenFrameworks
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Dissecting HDMI
Developing open, FPGA-based capture hardware for conference & user group recording

« Last Edit: July 31, 2018, 11:41:54 am by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
In this video also some HDMI concepts are become more clear
https://ksr-video.imgix.net/projects/850748/video-353869-h264_high.mp4

In the video stated that a 720p video format timings are:
Active video: 1280x720
Total video:  1650x750
Hsync : 40 ( Front porch: 110,  Back porch : 220 )
Vsync : 5 ( Front porch: 5,  Back porch : 20 )
Interlaced : 0
« Last Edit: August 01, 2018, 10:09:01 am by donkey »
 

Offline amyk

  • Super Contributor
  • ***
  • Posts: 8270
If you manage to bitbang HDMI from an MCU, you'll probably have achieved a world-first.

Using HDMI-DVI converter IC, it might be barely possible since you can work at pixel clock instead of bit clock.
 
The following users thanked this post: Richard Crowley, donkey

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Thank you amyk for your reply. From the replies i already assume that the answer of my question is <<No, it is not feasible! >>

I am aware of that this topic is a little bit gone beyond my question's scope but i decided to keep posting some useful matarials on this topic which i will come up with in my learning curve.

I am willing to write a conclusion and an explanation of my findings to answer my own question at the end.

As a beginner myself, i hope some of the materials i will post here may also help for other beginners to answer their own questions in their own similar research curve.

Again, thank you for your reply.
« Last Edit: August 01, 2018, 04:02:54 pm by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Development & Testing of HDMI 2.0 Interfaces | Tektronix



Understanding Bandwidth, Rise Time and Signal Fidelity
https://www.ece.ubc.ca/~robertor/Links_files/Files/TEK-Understanding-Scope-BW-tr-Fidelity.pdf
« Last Edit: August 15, 2018, 07:02:08 am by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
« Last Edit: August 15, 2018, 07:02:42 am by donkey »
 

Offline hendorog

  • Super Contributor
  • ***
  • Posts: 1617
  • Country: nz
I know nothing about HDMI, but I wonder if some sort of equivalent time sampling arrangement might allow you to progressively grab a frame.



 

Offline Lukas

  • Frequent Contributor
  • **
  • Posts: 412
  • Country: de
    • carrotIndustries.net
I know nothing about HDMI, but I wonder if some sort of equivalent time sampling arrangement might allow you to progressively grab a frame.
I did that for VGA some years back, sampling one column per frame using the builtin ADC.
After deserializing HDMI to parallel video, the same thing should be possible by the timer triggering a DMA transfer from a port to memory.
Apart from that, some MCUs like the bigger STM32 can interface to cameras, using an interface comparable to parallel video. Given the pixel clock isn't too high, that may work as well.
 
The following users thanked this post: hendorog, donkey

Offline hamster_nz

  • Super Contributor
  • ***
  • Posts: 2803
  • Country: nz
I've read extensively on DVi-D, HDMI, DisplayPort and so on, and know most of the older standards well (the newer ones like HDMI 2.0 are under NDAs).

I'm happy to clarify any questions you might have.
Gaze not into the abyss, lest you become recognized as an abyss domain expert, and they expect you keep gazing into the damn thing.
 
The following users thanked this post: donkey

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7733
  • Country: ca
You can always buy a SBC with HDMI input and HDMI output.  Like one of these:
http://linuxgizmos.com/snapdragon-600-sbc-offers-an-hdmi-input-and-dual-hdmi-outputs-for-99/
Grab frame, grab video, even realtime compress grabbed video...
 
The following users thanked this post: donkey

Offline Zucca

  • Supporter
  • ****
  • Posts: 4308
  • Country: it
  • EE meid in Itali
Nice post!

I am still dreaming to convert this old crap



into HDMI without an external ADC sampling with like those ones:



I can't stand the idea to take digital data (source VGA) and then get the analog to convert again in digital...  :horse:

It would be so nice to re-design this Keysight DSOXLAN LAN/VGA module



into a LAN / HDMI one or to be able to tap in the VGA output circuits and mod them into HDMI output ones

VGA must die!
« Last Edit: August 06, 2018, 11:09:23 am by zucca »
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 
The following users thanked this post: donkey

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
I like VGA, there are plenty of old displays around that have it and it pretty much always just works. It's a bit like RS-232, it's obsolete but ubiquitous and easy to use, so it still has value. Aside from the HDCP hassle, HDMI is generally superior and I use it if it's available but I'd still like to see VGA stick around.
 

Offline firehopper

  • Frequent Contributor
  • **
  • Posts: 408
  • Country: us
Nice post!

I am still dreaming to convert this old crap



into HDMI without an external ADC sampling with like those ones:



I can't stand the idea to take digital data (source VGA) and then get the analog to convert again in digital...  :horse:

It would be so nice to re-design this Keysight DSOXLAN LAN/VGA module



into a LAN / HDMI one or to be able to tap in the VGA output circuits and mod them into HDMI output ones

VGA must die!

actually vga is analog from what I understand, the rgb and sync signals are all analog, thats why the need of a A2D converter.
 

Offline BrianHG

  • Super Contributor
  • ***
  • Posts: 7733
  • Country: ca

actually vga is analog from what I understand, the rgb and sync signals are all analog, thats why the need of a A2D converter.
Yes, though the H&V syncs are digital, they are just clocked with the Analog DAC.
And, cheap small sample conversion boxes are crap and produce a crap HDMI image unless you go for a full blown quality scaler, doing a 1:1 outputting of the true source resolution & real high end VGA cables.  Otherwise, your text will be mush.

 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
Thank you for your courtesy

hamster_nz if you are the guy who own this page (http://hamsterworks.co.nz/mediawiki/index.php/Main_Page). You are my man! Great recourses for anyone who requires. Before asking any further questions, firstly i decided to try to clarify the concepts of the underlying technology by myself. I am a total beginner and i am even struggling with the elementry concepts of the technology. I have to try to answer some of my dumb questions on my own, to ask some less dumber questions to the forum.

If you are the guy who own the link(http://hamsterworks.co.nz/mediawiki/index.php/Main_Page). I want you to know that you did a such great work. Keep that great work and long live. I am also subscribed to your Youtube channel.

Thank you!
« Last Edit: August 15, 2018, 07:05:22 am by donkey »
 

Offline donkeyTopic starter

  • Regular Contributor
  • *
  • Posts: 51
  • Country: ru
A great product BrianHG. Thank you for your recommendation!
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf