Television Logo Killer

[johnwa]

Hi All,

I thought some people might be interested to see my current project, which is now approaching a reasonable degree of completion. It is an inline module for removing television station watermarks from live video in real time.

A number of TV channels here in Australia transmit their logos as a semitransparent graphic overlayed on the picture. It occurred to me that, provided that the logo has at least some degree of transparency, all of the information required to recreate the original picture is being transmitted. For any given pixel under the logo, the original pixel is simply being subjected to brightness and contrast changes - a simple linear function. Applying the inverse function will result in the transformation being ‘undone’. Of course, different pixel positions will require different inverse functions. These can be derived from a reference image of the logo being cancelled, and an alpha mask.

The module I have come up with has inputs and outputs in analogue RGB formats. The input signal is first digitised into three 8-bit values. A 16-bit SRAM framebuffer containing the reference logo and mask images is synchronised to the incoming video signal. All this data is fed to an FPGA, which implements the linear function described above. It also contains a simple digital filter, to blur any residual image of the logo that was not nulled out completely. Finally, the processed video data is converted back into an analogue signal.

The FPGA used is the Lattice iCE40HX1K device. Video ADC and DACs are an AD9983A and ADV7125KSTZ140, both from Analog Devices. There is an ATMega644A running the show, with an attached Micro SD card for loading the mask images. The address counters were done in discrete logic due to a lack of pins on the FPGA. The design maximum dot clock is 65MHz, though I am only running it at about 30 at the moment.

I will hopefully be able to put together a full description of the system at a later date, but I have attached a few pictures here to give an idea of the overall system. The video pass-through quality is quite good, albeit not perfect. The logo nulling is also fairly impressive, though it could be improved by allowing the strength of the cancelling function to be adjusted, and also possibly by using individual RGB or YUV mask images, rather than a single greyscale one.

Other than this, the only other issues are related to usability, as the template corresponding to the particular channel one is watching must currently be selected manually, and loading a new template takes around 15s. Faster loading could be provided, but is currently hampered by lack of resources in the FPGA. I had also  hoped to provide automatic recognition of which logo is being transmitted, and automatic selection of the corresponding template, but the algorithm for this requires further development, and implementing it would also require a new hardware revision.

I hope this is of some interest to readers. Let me know if you would like to see more about this project.

[Wilksey]

I would be interested to see more on this product.
+1 from me!

[LukeW]

Very clever.

And because the logo typically goes away during ads, if you detect when the logo goes away you might be able to do clever things like automatic channel hopping and then back to the original channel precisely when the ads stop, or automatic pausing of the VCR computer or PVR to remove ad breaks when recording video from the TV.

[johnwa]

Very clever.

And because the logo typically goes away during ads, if you detect when the logo goes away you might be able to do clever things like automatic channel hopping and then back to the original channel precisely when the ads stop, or automatic pausing of the VCR computer or PVR to remove ad breaks when recording video from the TV.

Now there is a thought!

I remember wondering years ago how one could automatically discern program content from advertising. Now, as a side effect of the networks' shameless self-promotion, they are effectively transmitting a signal that says 'this is the actual program content, it is safe to watch now'! I would imagine that the logos switch with a fairly deterministic delay from the start/end of the program, so with the aid of a few seconds of video delay, it should be possible to switch at the exact start and endpoints.

Regarding the logo recognition function, I did experiment with this a bit (using a PC-based implementation), and I got it half-working, but it was not reliable enough for casual viewing. However, there are plenty of established techniques in image processing, that I did not really look into then, and I don't think it would be too hard to improve the recognition function enough to make a logo-based ad zapper feasible.

You may have provided the inspiration for my next project LukeW, thanks!

[ecat]

Very clever.

And because the logo typically goes away during ads, if you detect when the logo goes away you might be able to do clever things like automatic channel hopping and then back to the original channel precisely when the ads stop, or automatic pausing of the VCR computer or PVR to remove ad breaks when recording video from the TV.

iirc logo detection is one of the tricka some video transcoding sw performs when removing advertisements from recorded programs. I'd need to hunt out the video app I'm thinking of but if anyone is interested just let me know.

Edit:
This is the app I was thinking of http://www.videoredo.com/en/Compare.htm "Automatic AdDetective commercial detection".

[Rory]

This would also be a good commercial audio muting application for live programming...

[rainking]

hi johnwa, have you tried YPbPr input too?
I'm using the AD9983A on a project (tv or dvd inputs to a tablet sized TFT output); works perfect with PC input but I can't get the colors right with 480i/p, and it's just all messed up at 720p/1080i...

[mariush]

I guess if you want to learn something, it's an interesting project.

There are however quite a lot of "logo removal" filters/plugins for software applications like Virtualdub or other video capture programs that can remove opaque, transparent, static or moving (rotating) logos ... some of them in realtime.
The opaque logo removal is usually done by analyzing a series of frames and using the motion vectors to predict what should be behind the logo...

[fcb]

Neat project - well done.

[denelec]

It would be great for plasma TV owners.
Some logos can cause burn-in.

[johnwa]

I would be interested to see more on this product.
+1 from me!

OK, I have got a bit of a writeup on the go, but it might be a little while before it is ready.

hi johnwa, have you tried YPbPr input too?
I'm using the AD9983A on a project (tv or dvd inputs to a tablet sized TFT output); works perfect with PC input but I can't get the colors right with 480i/p, and it's just all messed up at 720p/1080i...

No, rainking, I'm afraid I haven't tried YPbPr input. However, I did have terrible trouble getting the DC restoration on the chip to work properly. I was getting very dark, partially clipped images through the system. In the end, I think I had to use a mid-span biasing mode, which is actually intended for use with colour difference signals. I am not quite sure why this was required, or why it works, though the picture is quite acceptable.

I don't know if this is related to your problem at all. In what way exactly is it "all messed up"?

There are however quite a lot of "logo removal" filters/plugins for software applications like Virtualdub or other video capture programs that can remove opaque, transparent, static or moving (rotating) logos ... some of them in realtime.

Well, looks like I might be a bit behind the times... I was aware of MPlayer's filter, though this just does spatial interpolation over the logo region. It makes me wonder though, why haven't TV manufacturers implemented some of these algorithms already? I am sure current smart TVs would have enough processing power, and it would make a good selling point. Perhaps they don't think they can get away with it!

[zapta]

Reminds me the story of the color erases and the color anti eraser http://en.wikipedia.org/wiki/Color_killer#Color_eraser_.28mekhikon.29

I also saw in Japan once a TV that adjust down the volume during commercial, I think it was simply detect the higher sound volume and bump it down.

Anyway, very impressive project.

[steverino]

I already solved this problem...I stopped watching television 

Anyway, sounds like a nice bit of engineering.

[miguelvp]

I already solved this problem...I stopped watching television 

I validate your result, I also stopped watching television. My wife still wants me to watch shows with her for "quality" time I haven't give in yet. Quality and TV just doesn't go together.

[peter.mitchell]

a way of detecting when adverts start is by detecting when subtitles stop

[Grayfox]

a way of detecting when adverts start is by detecting when subtitles stop

Some ads have subtitles though.

Saw it when teletext was working.
IIRC it was a ad about hearing impairment, but these days it may be in every ad since video, audio and text advertising will make money.

But damn that is sweet, I take it you can add a program to change the location of the watermark removal added by the network.

And is it me or can you still see some part of the watermark if you pay real close attention.

[johnwa]

Reminds me the story of the color erases and the color anti eraser http://en.wikipedia.org/wiki/Color_killer#Color_eraser_.28mekhikon.29

Interesting, I hadn't heard of this. I think something similar happened here in Australia, before the "official" start of colour broadcasting.

 I take it you can add a program to change the location of the watermark removal added by the network.

Yes, you just need to capture a sample of the transmitted logo against a black and a white background. The mask image can then be computed from these pictures.

And is it me or can you still see some part of the watermark if you pay real close attention.

Yes, it is still visible if you look closely (see attachment - sorry about the out-of-focus). I would like to add some sort of gain control that could be adjusted in real time, for minimum visibility, but I have run out of room in the FPGA. There is also a slight colour shift occurring - this could be due to nonlinearities in the transmitted colour encoding, or I might be missing some colour weighting coefficients somewhere.

Anyway, thanks for the feedback everyone 

[rainking]

hey johnwa, if you're interested, i played with that chip a little more...
I use mid-scale clamping on Red and Blue when using a DVD player or cable box input, and clamp to ground on all channels when using a PC for input. This usually gives me the best color reproduction, but i still haven't gotten it perfect.
I went for a minimalist initialization and found the AD9983 defaults are almost good enough for most cases:

Code: [Select]

    write_i2c_device(0x01,0x73);//PLL Div MSB
    write_i2c_device(0x02,0x80);//PLL Div LSB
    write_i2c_device(0x03,0xc8);//VCO/CPMP
    write_i2c_device(0x05,0x02);//red gain offset
    write_i2c_device(0x06,0x00);
    write_i2c_device(0x07,0x02);//green gain offset
    write_i2c_device(0x08,0x00);
    write_i2c_device(0x09,0x02);//blue gain offset
    write_i2c_device(0x0A,0x00);
    write_i2c_device(0x20,0xc7);//Output Select 2
    write_i2c_device(0x3c,0x0d);
    //Initialize the AD9983A for YPbPr input
        write_i2c_device(0x1B,0x7b);//Clamp and Offset
        write_i2c_device(0x0B,0x40);//red channel offset
        write_i2c_device(0x0C,0x00);
        write_i2c_device(0x0D,0x02);//green channel offset
        write_i2c_device(0x0E,0x00);
        write_i2c_device(0x0F,0x40);//blue channel offset
        write_i2c_device(0x10,0x00);
        write_i2c_device(0x18,0x2a);//Coast and Clamp control: 20 for RGB, 2A for YPbPr
Using this code i've gotten:
1080i = ok
480p = ok
720p = bottom cut off
480i = split screens

Any idea if there's any settings on the  AD9983 to adjust properly for 720p or 480i or is this completely dependent on the output display?

[johnwa]

If anyone is still interested, I have finally got round to uploading a full description of this project at http://loopgain.net/logokiller/.

[Kilrah]

Very interesting to see a hardware solution for that!

[GK]

Could you do one of these that can make Amanda Jason's clothes disappear?

[SeanB]

You need to watch Ukranian TV for that, there are some which use news and weather women who do not need a budget for clothing at all, just for Victoria’s secret instead.

[Fungus]

It would be great for plasma TV owners.
Some logos can cause burn-in.

Most stations make the logo drift slowly around on screen to avoid this.

Try it: Mark the position of the logo with a post-it and watch TV for a while.

(now we find out which stations/countries don't give a damn about your plasma TV...  )

[German_EE]

Here is an example of Hungarian TV Weather Presentation



Other examples of her are available. The link above is safe for work but others may not be.

[johnwa]

It would be great for plasma TV owners.
Some logos can cause burn-in.

Most stations make the logo drift slowly around on screen to avoid this.

Try it: Mark the position of the logo with a post-it and watch TV for a while.

(now we find out which stations/countries don't give a damn about your plasma TV...  )

I didn't notice this  at all when I was developing the unit - the position was stable over a period of at least several weeks. And on ABC (the government broadcaster) too! Perhaps they reckon it will be good for the economy if everyone has to buy a new TV...