Author Topic: Audi automotive thermal imaging night vision camera teardown  (Read 44955 times)

0 Members and 1 Guest are viewing this topic.

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Audi automotive thermal imaging night vision camera teardown
« on: November 11, 2013, 10:33:21 am »
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline mos6502

  • Frequent Contributor
  • **
  • Posts: 537
  • Country: aq
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #1 on: November 11, 2013, 11:54:52 am »
Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?
« Last Edit: November 11, 2013, 11:57:01 am by mos6502 »
for(;;);
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 7754
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #2 on: November 11, 2013, 12:12:21 pm »
Mike,

You never cease to impress !

Thank you  :)
 

Online amyk

  • Super Contributor
  • ***
  • Posts: 5989
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #3 on: November 11, 2013, 07:38:21 pm »
Wow... is this the fourth or fifth one he's taken apart now? :-DD

If I remember correctly from reading some documents Motorola "accidentally" exposed many years ago, the 9S12 can be powered up in a special internal peripheral testing mode where it's possible to unsecure the flash and thus readout the firmware. ;)
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #4 on: November 11, 2013, 08:15:47 pm »
Next up on Mikeselectricstuff, Audi FLIR module connected to 1000 ipod displays mounted in a public place.

 :-+
« Last Edit: November 11, 2013, 08:57:07 pm by Stonent »
The larger the government, the smaller the citizen.
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #5 on: November 11, 2013, 08:38:59 pm »
What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?
I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline Stonent

  • Super Contributor
  • ***
  • Posts: 3824
  • Country: us
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #6 on: November 11, 2013, 08:58:43 pm »
What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?
I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

It doesn't by chance have a similar pin configuration to the E4 does it? Perhaps you could wake it up using the E4 as a host.
The larger the government, the smaller the citizen.
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #7 on: November 11, 2013, 09:48:02 pm »
What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?
I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

It doesn't by chance have a similar pin configuration to the E4 does it? Perhaps you could wake it up using the E4 as a host.
No totally different - When I get time I will try throwing a bunch of CAN at it to see if I can get it to run. Worst-case I may have to rent an i3 off ebay to see what the i-series sensor needs to be driven with.
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #8 on: November 11, 2013, 10:27:35 pm »
I suspect most of the protection is in the control unit, and the cam itself may just need a command to turn on - I can see some line/frame sync type signals in the serialised steam, so there's probably just s 'go' signal somewhere.
They _could_ have implemented some nasty challenge-response authentication though. The only obvious NV memory is the config flash  - not looked hard enough yet to determine if one of the 8 or 6 pin devices is an eeprom.
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Online amyk

  • Super Contributor
  • ***
  • Posts: 5989
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #9 on: November 11, 2013, 11:22:33 pm »
http://cvs.flir.com/tau2-electrical-idd
Does that 50-pin connector look familiar...?

http://cvs.flir.com/tau2-product-spec
This document says that all the outputs can be set to various options, one of which is "disabled". Maybe the camera module has just been configured with all the outputs disabled by default?

http://cvs.flir.com/tau2-quark-software-idd
Here details the commands that are used to change the settings, including the outputs.
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #10 on: November 12, 2013, 01:05:31 am »
http://cvs.flir.com/tau2-electrical-idd
Does that 50-pin connector look familiar...?

http://cvs.flir.com/tau2-product-spec
This document says that all the outputs can be set to various options, one of which is "disabled". Maybe the camera module has just been configured with all the outputs disabled by default?

http://cvs.flir.com/tau2-quark-software-idd
Here details the commands that are used to change the settings, including the outputs.
No - the connector is smalley. Look slike it has 4 LVDS pairs plus a few other oddments. Two of the LVDS likes look like clock and a datastream, but only sync present. I will take a look at the serial flash - it will be pretty obvious is there's anything in there apart from the FPGA bitstream.
It is also possible there's an eeprom - again won't be hard to find.

Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2198
  • Country: 00
    • My random blog.
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #11 on: November 12, 2013, 02:48:08 am »
How much was it Mike? I remember looking them up when you did E4 teardown and people wanted ridiculous amounts of money for scrapped Audi cameras (>$1000).

I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

I dont think there is anything stopping it from working, I think its the other way around - FPGA didnt initialize the sensor yet and is waiting for the command from the head unit.

Those Flir sensors look to be designed just like cmos camera sensors, with buildin ADC and all. I suspect there is I2C interface to the ADC inside and its waiting for commands (gain,exposure,start,stop). Without those commands it just sits in idle state.

We need ISC0601B pinout.

You did probe around E4 sensor, was there something resembling i2c bus on the sensor pins? Does changing levels in the camera change gain in the sensor? or is it fixed 16 bit output leaving all this stuff to camera cpu/fpga for postprocessing?
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #12 on: November 12, 2013, 02:56:50 am »
How much was it Mike? I remember looking them up when you did E4 teardown and people wanted ridiculous amounts of money for scrapped Audi cameras (>$1000).
There's tons of them on Ebay.de, around the 500-600 Euro mark

Quote

I dont think there is anything stopping it from working, I think its the other way around - FPGA didnt initialize the sensor yet and is waiting for the command from the head unit.

Those Flir sensors look to be designed just like cmos camera sensors, with buildin ADC and all. I suspect there is I2C interface to the ADC inside and its waiting for commands (gain,exposure,start,stop). Without those commands it just sits in idle state.

We need ISC0601B pinout.

You did probe around E4 sensor, was there something resembling i2c bus on the sensor pins? Does changing levels in the camera change gain in the sensor? or is it fixed 16 bit output leaving all this stuff to camera cpu/fpga for postprocessing?
Not probed the E4 sensor yet as it's hard to access - need to make a riser/breakout - I've ID'd the connector but couldn't find any stock of both parts.
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline equinoxe

  • Contributor
  • Posts: 23
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #13 on: November 12, 2013, 08:13:39 am »
Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

The module is also for sale here:
http://www.navi-tools.de/index.php?option=com_content&view=article&id=4&Itemid=5&lang=en
Sadly they don't really give any useful info (probably to protect their business).
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 11758
  • Country: gb
    • Mike's Electric Stuff
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #14 on: November 12, 2013, 08:24:04 am »
Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

The module is also for sale here:
http://www.navi-tools.de/index.php?option=com_content&view=article&id=4&Itemid=5&lang=en
Sadly they don't really give any useful info (probably to protect their business).

..well that probably rules out a complex enabling protocol - unfortunately I don't have access to a system installed in a car. When I get time I'll take a look at the CAN traffic & try throwing some commands at it
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 

Offline MrSquirrel

  • Contributor
  • Posts: 34
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #15 on: November 12, 2013, 08:51:51 am »
Great teardown - this is very interesting, think i'm going to have to get one of these myself.

I noticed the PCB had the marking "NV2":
NV2 = Second version of Autoliv Night Vision with a 24 degree Field of view, 320x240 pixels, 25u pitch.
According to some sales presentation stuff i found the camera is around $800 and image sensor alone $500. Ouch!

http://www.hanser-automotive.de/fileadmin/heftarchiv/2004/17700.pdf
Interesting block diagram on page 1. Camera simply pumps out raw video stream with CAN for control. Means camera module is somewhat universal, and processing engine can vary depending on that the OEM wants to do with it.

Looks like these guys had a go:
http://www.nightvisionforums.com/phpBB3/viewtopic.php?f=1&t=6799

Info on NV2 system, product roadmap etc:
http://www.chalmers.se/safer/EN/news/events/night-vision/downloadFile/attachment/Slides_Night_vision_presentation?nocache=1226396024.19

Teardown of the 1st Gen System
http://www.xolmatic.com/xprojects/XE65/Night_Vision_Camera.htm

More great 1st Gen info here, pics of 3rd Gen (NV3) board at bottom of page?: (oops, already posted earlier by mos6502)
http://www.mikrocontroller.net/topic/177639

And some tasty stuff here:
http://liu.diva-portal.org/smash/get/diva2:218938/FULLTEXT01
It is actually quite an interesting read in it's own right. :)

This document references a document called:
"Sundin Mats (2006) NV2 LVDS video link specification, Linköping: Autoliv Electronics AB"

Looks like that is the jackpot, but i can't find a copy. Perhaps e-mail the guys that wrote the thesis.  ;)

« Last Edit: November 12, 2013, 09:06:41 am by MrSquirrel »
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2198
  • Country: 00
    • My random blog.
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #16 on: November 13, 2013, 02:46:57 pm »
nice find, but as you probably noticed in teh pdf it relies on a car initializing the whole system
intercepting lvds data is "easy" (can read it before/after ldvs bridge chip) compared to guessing secret handshake headunit might have with ecu, or randomly fuzzing canbus between camera and headunit in hopes of initializing camera :(
This is the problem Mike has now.

Its probably as simple as headunit sending "#camera_address GO" to the camera unit.
Might be easier to just connect bare sensor to FPGA.
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Online amyk

  • Super Contributor
  • ***
  • Posts: 5989
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #17 on: November 13, 2013, 10:46:04 pm »
This document references a document called:
"Sundin Mats (2006) NV2 LVDS video link specification, Linköping: Autoliv Electronics AB"

Looks like that is the jackpot, but i can't find a copy. Perhaps e-mail the guys that wrote the thesis.  ;)
Notice the phrase "video link" - we already know the video data format so probably nothing interesting in there that isn't already covered in Flir's public datasheets. A control link spec on the other hand...
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 7754
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #18 on: November 16, 2013, 09:44:01 pm »
Slightly OT

An interesting document from Autoliv on the challenges of providing LW thermal cameras for the masses. Not applicable to the teardown specifically, but interesting to see what AutoLiv are up to and the hopes of the future. The lowering of the sensor and optics costs is detailed.....seems like the Flir E4 strategy.

http://www.autoliv.com/ProductsAndInnovations/Documents/Research%20Papers/4.%20Autoliv%20Research.pdf
 

Offline yogort1

  • Contributor
  • Posts: 18
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #19 on: December 15, 2013, 09:47:14 am »
Anyone found out what this secret handshake consists of?  :)  With this information it would be possible to make some fairly simple startup emulator without having to spend 250E for activator module  ;D
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2198
  • Country: 00
    • My random blog.
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #20 on: January 03, 2014, 08:11:55 am »
http://youtu.be/7h7LWeET1fI?t=19m01s

might work for powering this camera up
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline Lexman

  • Contributor
  • Posts: 6
  • Country: se
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #21 on: January 10, 2014, 06:51:20 am »
Hello everyone !
Great teardown and very interesting project. I have also order one cam from ebay.
I hope we can make some progress with this camera.
It will probably take a week or so until I got my camera.
I do not know if it's a NV1 or NV2. I could not see this in the pictures.
BTW I think we have to send a request via the CAN bus for each individual frame ... if I understand the docs. correct.

Cheers
//Lexman
 

Offline MrSquirrel

  • Contributor
  • Posts: 34
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #22 on: January 10, 2014, 10:56:53 am »

Did you buy the Audi or BMW variant?

I e-mailed these guys about their "enabler" unit for the BMW version:
http://carsystems.com.ua/en/intelligent-modules.html

Connection diagram attached for the NV1 and NV2 version. Looks like the vehicle only has basic integration with the control unit.

Unfortunatley i can't find anyone doing an equivalent for the Audi model, i wonder how similar the protocols are?

They want $250 for their module. Pretty reasonable as it potentially saves a lot of work and you can re-sell both it and the control unit on when the camera protocol is understood.

The BMW system seems to attract far higher prices on eBay, a bit too high IMHO for playing with. Audi units are a lot more available and realistically priced.
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 7754
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #23 on: January 10, 2014, 11:01:14 am »
IIRC the camera enabler only 'fools' the camera system controller into accepting the control commands from a genuine BMW central command bus. It cannot be used to fool the camera controller as a stand alone unit or to make a BMW camera work in an Audi etc.

Edit: This unit appears to be able to work in stand alone mode so I may be wrong about it still needing the BMW CAN Bus:

http://carsystems.com.ua/en/intelligent-modules.html

Definitely described as 'stand-alone' yet I read a reply from the company previously (In answer to a question about using it in another brand of car) and they said it would only work on a BMW of the correct model. Maybe they have improved it ?
« Last Edit: January 10, 2014, 11:07:56 am by Aurora »
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 7754
  • Country: gb
Re: Audi automotive thermal imaging night vision camera teardown
« Reply #24 on: January 10, 2014, 11:16:27 am »
If this has already been found, apologies. I attach an interesting document detailing the BMW system. Found here:

http://www.bimmerboard.com/members/jcarey/original/BMW%20Night%20Vision.pdf
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf