Audi automotive thermal imaging night vision camera teardown

[mikeselectricstuff]

[mos6502]

Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

[Fraser]

Mike,

You never cease to impress !

Thank you 

[amyk]

Wow... is this the fourth or fifth one he's taken apart now?

If I remember correctly from reading some documents Motorola "accidentally" exposed many years ago, the 9S12 can be powered up in a special internal peripheral testing mode where it's possible to unsecure the flash and thus readout the firmware.

[Stonent]

Next up on Mikeselectricstuff, Audi FLIR module connected to 1000 ipod displays mounted in a public place.

 

[mikeselectricstuff]

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

[Stonent]

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

It doesn't by chance have a similar pin configuration to the E4 does it? Perhaps you could wake it up using the E4 as a host.

[mikeselectricstuff]

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

It doesn't by chance have a similar pin configuration to the E4 does it? Perhaps you could wake it up using the E4 as a host.

No totally different - When I get time I will try throwing a bunch of CAN at it to see if I can get it to run. Worst-case I may have to rent an i3 off ebay to see what the i-series sensor needs to be driven with.

[mikeselectricstuff]

I suspect most of the protection is in the control unit, and the cam itself may just need a command to turn on - I can see some line/frame sync type signals in the serialised steam, so there's probably just s 'go' signal somewhere.
They _could_ have implemented some nasty challenge-response authentication though. The only obvious NV memory is the config flash  - not looked hard enough yet to determine if one of the 8 or 6 pin devices is an eeprom.

[amyk]

http://cvs.flir.com/tau2-electrical-idd
Does that 50-pin connector look familiar...?

http://cvs.flir.com/tau2-product-spec
This document says that all the outputs can be set to various options, one of which is "disabled". Maybe the camera module has just been configured with all the outputs disabled by default?

http://cvs.flir.com/tau2-quark-software-idd
Here details the commands that are used to change the settings, including the outputs.

[mikeselectricstuff]

http://cvs.flir.com/tau2-electrical-idd
Does that 50-pin connector look familiar...?

http://cvs.flir.com/tau2-product-spec
This document says that all the outputs can be set to various options, one of which is "disabled". Maybe the camera module has just been configured with all the outputs disabled by default?

http://cvs.flir.com/tau2-quark-software-idd
Here details the commands that are used to change the settings, including the outputs.

No - the connector is smalley. Look slike it has 4 LVDS pairs plus a few other oddments. Two of the LVDS likes look like clock and a datastream, but only sync present. I will take a look at the serial flash - it will be pretty obvious is there's anything in there apart from the FPGA bitstream.
It is also possible there's an eeprom - again won't be hard to find.

[Rasz]

How much was it Mike? I remember looking them up when you did E4 teardown and people wanted ridiculous amounts of money for scrapped Audi cameras (>$1000).

I would expect it to be similar to the E4 sensor, but at the moment something is stopping the sensor doing anything.

I dont think there is anything stopping it from working, I think its the other way around - FPGA didnt initialize the sensor yet and is waiting for the command from the head unit.

Those Flir sensors look to be designed just like cmos camera sensors, with buildin ADC and all. I suspect there is I2C interface to the ADC inside and its waiting for commands (gain,exposure,start,stop). Without those commands it just sits in idle state.

We need ISC0601B pinout.

You did probe around E4 sensor, was there something resembling i2c bus on the sensor pins? Does changing levels in the camera change gain in the sensor? or is it fixed 16 bit output leaving all this stuff to camera cpu/fpga for postprocessing?

[mikeselectricstuff]

How much was it Mike? I remember looking them up when you did E4 teardown and people wanted ridiculous amounts of money for scrapped Audi cameras (>$1000).

There's tons of them on Ebay.de, around the 500-600 Euro mark

I dont think there is anything stopping it from working, I think its the other way around - FPGA didnt initialize the sensor yet and is waiting for the command from the head unit.

Those Flir sensors look to be designed just like cmos camera sensors, with buildin ADC and all. I suspect there is I2C interface to the ADC inside and its waiting for commands (gain,exposure,start,stop). Without those commands it just sits in idle state.

We need ISC0601B pinout.

You did probe around E4 sensor, was there something resembling i2c bus on the sensor pins? Does changing levels in the camera change gain in the sensor? or is it fixed 16 bit output leaving all this stuff to camera cpu/fpga for postprocessing?

Not probed the E4 sensor yet as it's hard to access - need to make a riser/breakout - I've ID'd the connector but couldn't find any stock of both parts.

[equinoxe]

Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

The module is also for sale here:
http://www.navi-tools.de/index.php?option=com_content&view=article&id=4&Itemid=5&lang=en
Sadly they don't really give any useful info (probably to protect their business).

[mikeselectricstuff]

Here's an old thread on a German forum that deals with a BMW system:

http://www.mikrocontroller.net/topic/177639

If you scroll down for the pictures, you can see that the sensor module looks similar to yours. It's different inside, though. Different sensor, Altera FPGA.

Anyway, the upshot seems to be that at least the BMW system, the sensor module, the control module and the car's ECU are locked together. They do a little chat on powerup and exchange serial numbers and stuff. This prevents the camera from being stolen and used in a different car, even if it's the same model.

Here's an old listing on eBay for an "unlocking module":

http://www.ebay.de/itm/ws/eBayISAPI.dll?ViewItem&item=330774553378&ssPageName=ADME:B:SS:DE:1123

It connects as a man in the middle device between the car's CAN bus and the control module's CAN bus.

What kind of data does the actual sensor itself spit out? Would it be feasible to just hook it up to an FPGA board?

The module is also for sale here:
http://www.navi-tools.de/index.php?option=com_content&view=article&id=4&Itemid=5&lang=en
Sadly they don't really give any useful info (probably to protect their business).

..well that probably rules out a complex enabling protocol - unfortunately I don't have access to a system installed in a car. When I get time I'll take a look at the CAN traffic & try throwing some commands at it

[MrSquirrel]

Great teardown - this is very interesting, think i'm going to have to get one of these myself.

I noticed the PCB had the marking "NV2":
NV2 = Second version of Autoliv Night Vision with a 24 degree Field of view, 320x240 pixels, 25u pitch.
According to some sales presentation stuff i found the camera is around $800 and image sensor alone $500. Ouch!

http://www.hanser-automotive.de/fileadmin/heftarchiv/2004/17700.pdf
Interesting block diagram on page 1. Camera simply pumps out raw video stream with CAN for control. Means camera module is somewhat universal, and processing engine can vary depending on that the OEM wants to do with it.

Looks like these guys had a go:
http://www.nightvisionforums.com/phpBB3/viewtopic.php?f=1&t=6799

Info on NV2 system, product roadmap etc:
http://www.chalmers.se/safer/EN/news/events/night-vision/downloadFile/attachment/Slides_Night_vision_presentation?nocache=1226396024.19

Teardown of the 1st Gen System
http://www.xolmatic.com/xprojects/XE65/Night_Vision_Camera.htm

More great 1st Gen info here, pics of 3rd Gen (NV3) board at bottom of page?: (oops, already posted earlier by mos6502)
http://www.mikrocontroller.net/topic/177639

And some tasty stuff here:
http://liu.diva-portal.org/smash/get/diva2:218938/FULLTEXT01
It is actually quite an interesting read in it's own right.

This document references a document called:
"Sundin Mats (2006) NV2 LVDS video link specification, Linköping: Autoliv Electronics AB"

Looks like that is the jackpot, but i can't find a copy. Perhaps e-mail the guys that wrote the thesis. 

[Rasz]

nice find, but as you probably noticed in teh pdf it relies on a car initializing the whole system
intercepting lvds data is "easy" (can read it before/after ldvs bridge chip) compared to guessing secret handshake headunit might have with ecu, or randomly fuzzing canbus between camera and headunit in hopes of initializing camera
This is the problem Mike has now.

Its probably as simple as headunit sending "#camera_address GO" to the camera unit.
Might be easier to just connect bare sensor to FPGA.

[amyk]

This document references a document called:
"Sundin Mats (2006) NV2 LVDS video link specification, Linköping: Autoliv Electronics AB"

Looks like that is the jackpot, but i can't find a copy. Perhaps e-mail the guys that wrote the thesis. 

Notice the phrase "video link" - we already know the video data format so probably nothing interesting in there that isn't already covered in Flir's public datasheets. A control link spec on the other hand...

[Fraser]

Slightly OT

An interesting document from Autoliv on the challenges of providing LW thermal cameras for the masses. Not applicable to the teardown specifically, but interesting to see what AutoLiv are up to and the hopes of the future. The lowering of the sensor and optics costs is detailed.....seems like the Flir E4 strategy.

http://www.autoliv.com/ProductsAndInnovations/Documents/Research%20Papers/4.%20Autoliv%20Research.pdf

[yogort1]

Anyone found out what this secret handshake consists of?    With this information it would be possible to make some fairly simple startup emulator without having to spend 250E for activator module 

[Rasz]

http://youtu.be/7h7LWeET1fI?t=19m01s

might work for powering this camera up

[Lexman]

Hello everyone !
Great teardown and very interesting project. I have also order one cam from ebay.
I hope we can make some progress with this camera.
It will probably take a week or so until I got my camera.
I do not know if it's a NV1 or NV2. I could not see this in the pictures.
BTW I think we have to send a request via the CAN bus for each individual frame ... if I understand the docs. correct.

Cheers
//Lexman

[MrSquirrel]

Did you buy the Audi or BMW variant?

I e-mailed these guys about their "enabler" unit for the BMW version:
http://carsystems.com.ua/en/intelligent-modules.html

Connection diagram attached for the NV1 and NV2 version. Looks like the vehicle only has basic integration with the control unit.

Unfortunatley i can't find anyone doing an equivalent for the Audi model, i wonder how similar the protocols are?

They want $250 for their module. Pretty reasonable as it potentially saves a lot of work and you can re-sell both it and the control unit on when the camera protocol is understood.

The BMW system seems to attract far higher prices on eBay, a bit too high IMHO for playing with. Audi units are a lot more available and realistically priced.

[Fraser]

IIRC the camera enabler only 'fools' the camera system controller into accepting the control commands from a genuine BMW central command bus. It cannot be used to fool the camera controller as a stand alone unit or to make a BMW camera work in an Audi etc.

Edit: This unit appears to be able to work in stand alone mode so I may be wrong about it still needing the BMW CAN Bus:

http://carsystems.com.ua/en/intelligent-modules.html

Definitely described as 'stand-alone' yet I read a reply from the company previously (In answer to a question about using it in another brand of car) and they said it would only work on a BMW of the correct model. Maybe they have improved it ?

[Fraser]

If this has already been found, apologies. I attach an interesting document detailing the BMW system. Found here:

http://www.bimmerboard.com/members/jcarey/original/BMW%20Night%20Vision.pdf

[MrSquirrel]

They said it worked standalone and "provided all appropriate CAN messages" though i will ask specifically about this. English is not their main language.

[Fraser]

It was interesting to read in the BMW camera document that I referenced, that the controller effectively up-scales the camera 320x240 output to 640x480.....nice 

If the controller enabler is fully 'stand-alone', as described to MrSquirrel, and detailed on the web page, this is quite a powerful high frame rate TIC to own for a reasonable cost    Time to start hunting in the scrap yards for 'written off' high end BMW's ?    

[Lexman]

I bought the BMW variant but it is not the Pathfinder model.
Still waiting for the delivery.
It's made by Autoliv (from Sweden  ) and have 4 or 4+2 (6) pin connector.
So the night independent stand alone controller from carsystems.com will not work with this camera if it's only have 4-pin connector.
I think their second independent "solution" works only together with the Pathfinder from FLIR with 12-pin connector.
And I dont have a BMW..

Cheers
//Lexman

[Lexman]

I don't now if you have seen this video..

It's a cam from BMW E65 made by Autoliv "Autocam C1" and working as standalone.
What do you all Think about this cam ?
Looks like PathFinder from Flir.
Maybe this "old" model was not needed to be unlocked to work properly.

Cheers
//Lexman

[MrSquirrel]

It's a cam from BMW E65 made by Autoliv "Autocam C1" and working as standalone.
What do you all Think about this cam ?
Looks like PathFinder from Flir.
Maybe this "old" model was not needed to be unlocked to work properly.

Cheers
//Lexman

Yes that's the 1st generation model - this has a composite video output too so you should be able to have fun without much effort.

Edit: Some more info for you on the Gen 1 camera:

Spec sheet

http://www.flir.com/uploadedFiles/Brochure_PathFindIROEM.pdf

The PathFindIR cameras could have different firmware to the vehicle manufacturers models though? One way to find out.

The interface specification is mentioned here, but you have to sign an NDA:
http://www.flir.com/uploadedFiles/UserManual_PathFindIR.pdf?
http://cvs.flir.com/l/6132/PathFindIR-Interface-Cable-pdf/wb66k
http://cvs.flir.com/l/6132/-FLIR-PathFindIR-OEM-Cable-pdf/wb66m

So you just need a serial to LIN converter, to communicate with that camera. At an physical level you can bet they have just stuck a serial to LIN converter on top of an existing design. I wonder how similar the commands are to other Flir cores?

[MrSquirrel]

Just found this, loads of great info on how the 2nd Gen BMW system operates:

http://www.kneb.net/bmw/F01-02/06.3_F01%20BMW%20Night%20Vision%202.pdf

It is similar to the BMW manual Aurora posted, but for the 2nd gen, not 1st.

This suggests the 2nd gen camera is paired to the control unit:

Programming and Coding
Initializing Software
When replacing the camera, it is always necessary to initialize the software by entering an
enable code (FSC).

Note: The vehicle identification number (VIN) must always be entered when
ordering a new camera or a new control unit.

The camera is programmed by through of the control unit. The control unit receives the
programming data for the camera through PT-CAN. The control unit forwards this data to
the camera through the "private CAN-bus".

This is standard practise on with modern vehicles, hopefully nothing deeper? Often is simply a serial number check by a "master unit" on the bus. If the master notices one of the serial numbers (sometimes even a light cluster, or a door switch module) has changed it throws a wobbly.

However, it does not mean to say that if the end device is powered up on the bench and sent the correct CAN messages it won't come to life. I'm no expert on how BMW/Audi do it though, Honda is more my scene.

I think i would do a lot of sniffing before firing messages to either unit in case they get upset.

[Myself]

By the way, I don't know if any of the pages linked earlier in the thread have this info, but that round four-pin connector carrying the video signal is known as HSD:

http://www.rosenberger.com/en/products/automotive/hsd.php

http://www.bce.it/wp-content/uploads/2012/06/Amphenol-HSD-Connector-Introduction-BCE-06-12.pdf

http://www.te.com/en/industries/automotive/browse-products/infotainment/high-speed-data-connector-system-hsd.html

Unfortunately I don't know anywhere to source samples, but one could ask around. In the TE PDFs there's a part number for a female-female cable, 2141723, which their own site doesn't know about but Newark claims to carry it:
http://www.newark.com/te-connectivity/2141723-3/connector/dp/55W4915

All the other part numbers seem to be dead-ends, so perhaps calling one's local TE rep could yield fruit. Or just use solder -- the universal adapter!

[Lexman]

I got my BMW camera today. (Autoliv)
It was a NB2 version.
Part no: 66.54-9 205 899-01
HW:1
SW:0.1.36 / 0.0.39

I have not opened it yet but the connecting cable for the camera was included and it has ONLY 4 wires + woven shield.
The connector on the camera has 4 + 2 pins. The "extra" 2 pins is positioned on the side of the shielded round 4 pin connecter and are not used in the female cconnector (Cable side)
None of the four wires are connected to ground so I guess the neg power goes through the shield - through the shielded circular 4-wire connector.
That means we have a total of 5 wires to the camera.
I get confused ... 1 wire + shield for power, 2 wires for LVDS signal and 2 wires for the canbus = 5 wires + shield 
I have 120 ohms between two wires so they are likely the canbus.

Cheers
//Lexman

[Lexman]

Here is some pictures...

[lacatusuflorin]

I got my BMW camera today. (Autoliv)
It was a NB2 version.
Part no: 66.54-9 205 899-01
HW:1
SW:0.1.36 / 0.0.39

I have not opened it yet but the connecting cable for the camera was included and it has ONLY 4 wires + woven shield.
The connector on the camera has 4 + 2 pins. The "extra" 2 pins is positioned on the side of the shielded round 4 pin connecter and are not used in the female cconnector (Cable side)
None of the four wires are connected to ground so I guess the neg power goes through the shield - through the shielded circular 4-wire connector.
That means we have a total of 5 wires to the camera.
I get confused ... 1 wire + shield for power, 2 wires for LVDS signal and 2 wires for the canbus = 5 wires + shield 
I have 120 ohms between two wires so they are likely the canbus.

Cheers
//Lexman

The "extra" 2 pins is positioned on the side of the shielded round 4 pin connecter are use for 12 Vdc, 2 wires (red + / black -)

[mikeselectricstuff]

Do you have both ends of that cable?
From memory, the data parts are the same, but only the camera end has the power pins

[lacatusuflorin]

Do you have both ends of that cable?
From memory, the data parts are the same, but only the camera end has the power pins

I have the cable with the both end?
The end from the camera have 4+2 pins
andat  the other end have 2 terminals: 1 with 4 pins and  1 with 2 pins

From the service manual for audi the 2 pins are for power supply  with 12 Vdc

The core of this camera are similar with the FLIR TAU 2, and even have the power module of the camera similar with the tau photon replicator board for use with voltage of 12 vdc
The tau core have the posibilities to use  analog video signal NTSC / PAL, but i am not able to check the signal type for this connector.
I need only the ntsc signal from this camera to use with an external display

the 30 pins connector whwn power up have this kind of voltage levels, but i don't have an oscilloscope to see the signals forms, and i don't know how ntsc signals looks like

if you have any idea

[mikeselectricstuff]

Do you have both ends of that cable?
From memory, the data parts are the same, but only the camera end has the power pins

I have the cable with the both end?
The end from the camera have 4+2 pins
andat  the other end have 2 terminals: 1 with 4 pins and  1 with 2 pins

From the service manual for audi the 2 pins are for power supply  with 12 Vdc

The core of this camera are similar with the FLIR TAU 2, and even have the power module of the camera similar with the tau photon replicator board for use with voltage of 12 vdc
The tau core have the posibilities to use  analog video signal NTSC / PAL, but i am not able to check the signal type for this connector.
I need only the ntsc signal from this camera to use with an external display

the 30 pins connector whwn power up have this kind of voltage levels, but i don't have an oscilloscope to see the signals forms, and i don't know how ntsc signals looks like

if you have any idea

See the teardown vid - the camera itself outputs LVDS data, once you send it whatever magic commands it needs. The control box outputs composite video, but needs to think it's in a car.
I will be investigating it when I get time, but that won't be for a while

[Lexman]

I purchase the camera including its outer plastic housing. - see picture.
In the picture is the connecting cable visible, and its cut off.
The cable I have received and which were connected to the camera when I received it is also cut off but its longer than the cable in the Picture. Which explains it all.
It is not the right cable for the camera I have received. It's missing the DC power wires.
 
I have seen in the newest camera version from Autoliv (NV3), it has only 4 pin connector, the extra 2 pin for Power is missing. I wonder how the interface works in this camera. 

New info...
NV3 uses MAX9259 for the LVDS signal and the IC has also "full-duplex control channel" to control the camera. Canbus interface is no longer used.
The camera connector is 4-pin, 2-pin for Power and 2-pin för LVDS AND control.


Cheers
//Lexman

[LesioQ]

Howdy,
New here.
I also got NV2 IR cam core and would love to make good use of it. While hoping that Mike will crack the CAN init, I was wondering what B-plans can be assigned to the task.

I noticed there's an ARDUCAM pcb project, being capable to communicate with CMOS sensors. It's open to modifications, comes at $50 with QVGA LCD and snapshot storage on a card. Maybe this would be a good path to read image from ISC0601B IR sensor, once (if) we know the pinout.

Piotr.K

[TheEnd]

Hi guys,
Yes, it's someone else interested in an easy and cheap way of getting hold of a thermal imaging core with as little extra stuff tacked on.

Give me a nudge if there is another more detailed thread on this topic, otherwise I'm going to stick here as a data dump for those interested.

I have some BMW diag software, including a very basic program that will forward and return commands.

So- here are some of the possible commands and controls. Note- I think this is direct from the camera module itself!

Statuses

STATUS_KAMERA_TEMP camera temp
STATUS_HEIZUNG_TEMP heating temp (it has a heated front lens for clearing rain and condensation)
STATUS_KAMERA_SPANNUNG - spannung = voltage
STATUS_KAMERA_STUNDEN - operating hours
STATUS_SHUTTER_CYCLE -shutter cycle time in seconds
STATUS_ZEILE_LESEN - "Read data of a line from the camera.
             It is the one half of one line read (160 bytes)
             The camera image consists of 320 columns and 240 rows" - official function description
STATUS_VERDREHWINKEL - camera rotation angle, returns 1/10 deg angle
STATUS_OFFSETWINKEL - gives the offset angle, 8 bit data centred on 128decimal, and can read +/- 7.2 degrees
STATUS_DEAD_PIXEL - this returns the result of a dead pixel test, and gives the reply as a voltage(?!)
STATUS_KAMERA_BOARD - this reads ID data such as serial numbers and manufacturer codes


Controls and commands
STEUERN_TESTBILD "Displays test patterns NIVI the camera to
             Test Image 1 -> diagonal gray stripes
             Test Image 2 -> Vertical grayscale bar
             Test image 3 -> horizentale grayscale bar
             Test image 5 -> horizentale grayscale dark bottom light up"
STEUERN_SHUTTER- The following closing procedure can be set
             00 => shutter remains open
             01 => Shutter closes automatically every 2 min WITHOUT pixel calibration
             02 => Shutter closes automatically every 2 min WITH pixel calibration
             06 => shutter is closed (not Image Display)
STEUERN_WINKEL_SCHREIBEN - writes the rotation angle
STEUERN_HEIZUNG_AUS - switches heating off
STEUERN_HEIZUNG_EIN - switches heating on
STEUERN_ZOOM_AUS - zoom out
STEUERN_ZOOM_EIN - zoom in
STEUERN_TEST_PIXEL_EIN - switches on the dead pixel test, this takes 3.5 mins and outputs the total number of dead pixels - an off command is also available


Now some of the more interesting parts-
AUTHENTISIERUNG_START - for this, you need to give an "argument", in this case feed it the binary buffer, and the binary buffer is formed like this (in German, but fairly self explanatory, google translate is there if you need it!)
Als Argument wird ein vorgefuellter Binaerbuffer uebergeben
            Der Binaerbuffer hat folgenden Aufbau
            Byte 0              : Datentyp (1:Daten, 2:Maskendaten)
            Byte 1              : (unbenutzt) Wortbreite (1:Byte, 2:Word, 3:DWord)
            Byte 2              : (unbenutzt) Byteordnung (0:LSB zuerst, 1 MSB zuerst)
            Byte 3              : (unbenutzt) Adressierung (0: freie Adressierung, 1:Blockadressierung)
            Byte 4              : Authentisierungszeit in Sekunden
            Byte 5,6            : (unbenutzt) WordParameter 1 (low/high)
            Byte 7,8            : (unbenutzt) WordParameter 2 (low/high)
            Byte 9,10,11,12     : (unbenutzt) Maske (linksbuendig)
            Byte 13,14          : Anzahl Bytedaten (low/high)
            Byte 15,16          : (unbenutzt) Anzahl Wortdaten (low/high)
            Byte 17,18,19,20    : (unbenutzt) Wortadresse (low/highbyte, low/highword)
            Byte 21,....        : Schluesseldaten
            Byte 21+Anzahl Daten: ETX (0x03)


Fault code outputs
0xA788   Heater Camera
0xA789   voltage Supply Camera
0xA78A   Tamper Protection Camera uh oh! - it has a form of anti tamper
0xA78B   IR Sensor
0xA78C   Defective Pixel
0xA78D   Shutter
0xA78E   HW/SW error
0xA78F   Video out failure
0xA790   Camera not adjusted
0xA768   Camera Overtemperature
0xA769   Watching Reset (Camera)
0xFFFF   unbekannter Fehlerort


So, that might be making things overcomplicated, especially if someone is going to tap into the raw sensor data but it is explaining what the "private CAN" connections are about, and it shows the module has some brains and self diagnosis.

This particular info should be for the first version of the camera, but it looks like it applies to all of them.

I haven't got a camera, I'm not on the level of the hardware hacking that some people might be on, but if i can help, I will!

[LesioQ]

Hi,

That indeed sounds promising !
Do these commands have any hex representation ? Do You have a command id table or something useful when trying to communicate with bare cam via CAN ?
Piotr.K

[TheEnd]

Sadly it's doesn't go that far, the program is still a "dashboard" or front page, and the data will be interpreted and formed into a CAN command somewhere else.

Another translation table shows-
NR   MODE   MODE_TEXT
0x81   DEFAULT   DefaultMode
0x82   PT   PeriodicTransmissions
0x84   EOLSSM   EndOfLineSystemSupplierMode
0x85   ECUPM   ECUProgrammingMode
0x86   ECUDM   ECUDevelopmentMode
0x87   ECUAM   ECUAdjustmentMode
0x88   ECUVCM   ECUVariantCodingMode
0x89   ECUSM   ECUSafetyMode
0xFA   SSS_A   SystemSupplierSpecific (A)
0xFB   SSS_B   SystemSupplierSpecific (B)
0xFC   SSS_C   SystemSupplierSpecific (C)
0xFD   SSS_D   SystemSupplierSpecific (D)
0xFE   SSS_E   SystemSupplierSpecific (E)
0xXY   --   unbekannter Diagnose-Mode

So it looks like there is a command to ask what mode it is in, and the hex answer will define what it comes back as saying.
This could well be a standard table for all modules in the cars, but it might also mean it could be set to a development mode 0x86, if the command to change modes could be found.
I don't know much about CAN and the data side of things, but I'm guessing it'll make more sense to others.
If it's useful, maybe it'll mean a way to get into these devices, if it isn't, well, hey, you can never have too much information.

[LesioQ]

Meanwhile I noticed that my Audi NV2 responds to 0x301 to 0x303 frames (remote type) at 100kbps.
Does anyone know IF I can use CAN interface to talk to LIN device ? (BMW NV1 cam).
Piotr.K

[yogort1]

Anyone managed to run this camera?

[ogoun]

Greets all,

Has anyone got any further with this? I have both the cam and the controller, and have disassembled both, and have the serial eeprom from the camera (used with the Xilinx spartan3e FPGA). I will look at reading it (after making an adapter), to see if anything can be gleaned from its contents, as I suspect that the spartan3e has one of the Xilinx microcontroller cores in it, possibly a microblaze. If it does, then there may be some areas of the eeprom that have the code for this micro core. If so, it may shed some light on how the private can bus is used. Perhaps this will be enough to learn how to "wake up" the cam.

Cheers,
Pete

[ogoun]

Some interesting patents:

7470902 http://www.freepatentsonline.com/7470902.pdf

6812465 http://www.freepatentsonline.com/6812465.pdf

6028309 http://www.freepatentsonline.com/6028309.pdf

Gives a bit more info on the camera module and the FPA (320x240 focal plane array) chip

Cheers,

Pete

[therew]

Does automotive thermal imaging have something in common with thermal systems for tactical/hunting purposes? I have a couple of the finest thermal imaging scopes from http://www.atncorp.com/thermal-night-vision and I was wondering whether this technology is similar to what I can install on a car. My father got a FLIR thermal imaging night vision camera for a 99-03 Cadilac Deville, installed a 5" monitor above his radio and installed the camera behind the grill. Looks awesome.

[encryptededdy]

Does automotive thermal imaging have something in common with thermal systems for tactical/hunting purposes? I have a couple of the finest thermal imaging scopes from http://www.atncorp.com/thermal-night-vision and I was wondering whether this technology is similar to what I can install on a car. My father got a FLIR thermal imaging night vision camera for a 99-03 Cadilac Deville, installed a 5" monitor above his radio and installed the camera behind the grill. Looks awesome.

I'm not sure about the new ATN thermal sights (the one with WiFi), however their older ones used FLIR Tau2 cores which is very similar if not the same sensor wise compared to automotive systems like the FLIR PathfindIR

[nixxon]

See the teardown vid - the camera itself outputs LVDS data, once you send it whatever magic commands it needs. The control box outputs composite video, but needs to think it's in a car.
I will be investigating it when I get time, but that won't be for a while

Have you had a chance to investigate the Audi Night Vision any further, Mike?

[nixxon]

See the teardown vid - the camera itself outputs LVDS data, once you send it whatever magic commands it needs. The control box outputs composite video, but needs to think it's in a car.
I will be investigating it when I get time, but that won't be for a while

Have you had a chance to investigate the Audi Night Vision any further, Mike?

I forgot to mention, there is a guy in the neighborhood selling an Audi Night Vison lens/sensor unit assembly. (no control box). I was thinking of using the Germanium lens as a close up tool for my Fluke TiR. And if it is possible I want to make the Audi camera a stand alone working unit (with a separate LCD screen and battery pack). He is asking the equivalent of US$ 130 for it. I guess it is worth that amount. Any chance somebody will hack this equipment any further soon?

[Fraser]

They make good paperweights and a source of parts. Do not expect a working 'camera only' hack any time soon ;-)

Any that have had any form of hacking already tried on them will be in 'lockdown' so an even worse hacking proposition.

Fraser

[xmjacky]

I also have a BMW thermal imaging camera, the 2nd Gen BMW system,4 wire interface,and the research work has made little progress
1 camera interface definition.(1.JPG)
2, the decoder using MAX9260, as long as the build simple circuit can output RGB encoding 16bit, the logic analyzer results, video decoding is correct, but the camera does not output pixel data, should be the protection mechanism of the problem.
3, the use of PC through the MAX9260 to send the camera command (232, RX/TX), in some baud rate can receive replies, but I do not have the exact command, can not let the camera work properly.
If you have a BMW car or rent a car, you can grab data with a simple tool(logical analyzer or serial port tool), you can remove the protection mechanism.
Wiring mode is as follows

[xmjacky]

Hope that more people join the crack job

[Rasz]

Hope that more people join the crack job

So you are posting from Iran (at least you removed country from your profile now) asking for help with cracking ITAR controlled technology, and those are your only posts on this forum. Right.

[james_s]

Why on earth are these locked down in the first place? What are they protecting them from? I can't see why BMW or Audi would care if someone uses their parts in something else. Are they afraid of the camera modules being stolen off of parked cars?

[mikeselectricstuff]

Why on earth are these locked down in the first place? What are they protecting them from? I can't see why BMW or Audi would care if someone uses their parts in something else. Are they afraid of the camera modules being stolen off of parked cars?

BMW / Audi don't care -  it's a condition imposed by the camera manufacturer to ensure they stay within US export regs, to avoid them being repurposed

[Fraser]

As Mike says, it is a requirement of ITAR(USA) and Dual Use Technology regulations. These cameras are 320x240 pixels with a 60fps output. They therefore are a controlled product.

The reasons for such controls have been detailed in other threads but basically, a thermal camera can be a very effective weapon aiming or guidance system. The appropriate authorities set the frame rate as <9fps for cameras that do not require the tighter ITAR and DUT distribution restrictions.
9fps is not great on a vehicle travelling at high speed so Autoliv opted to use a FLIR high frame rate camera that was suitably locked down to comply with ITAR and DUT regulations. High frame rate thermal imaging can involve much red tape when it comes to export from one country to another. Not great for a car manufacturer.

Fraser

[BradC]

Why on earth are these locked down in the first place? What are they protecting them from? I can't see why BMW or Audi would care if someone uses their parts in something else. Are they afraid of the camera modules being stolen off of parked cars?

High speed IR cameras make great fine tuning guidance add-ons for high powered fireworks, so there are a number of places that try hard to keep them out of the hands of people who might like to use them like that.

The amount of paperwork required to get some medium speed units used for CCTV into Qatar a few years ago would make your head spin. I'm sure the automotive manufacturers are doing their part in making them difficult to use.

[james_s]

Surely anyone wanting to use such a thing for missile guidance would have the resources to make reverse engineering and re-purposing the cameras trivial? I mean it seems like designing an entire new board to interface directly with the sensor would be trivial compared to designing a workable guided missile.

I guess that's the usual approach though, put a gigantic expensive and inconvenient lock on the front door but leave a window open.

[Fraser]

You would have to ask those on the various Dual Use Technology commitees for their reasons and thought processes.

Ask yourself this though...... if a technology has proven to be a tactical advantage over your enemies, why on earth would you make it EASY for them to obtain the same ? The 9fps restriction is actually generous in my opinion. If I had needed to prevent weapons use of the technology I would have limited the cameras to 1fps or even lower.

As I say, this has been covered before and I do not want to quote chapter and verse about why a control is needed on this technology and go through all the counter arguments about ease of obtaining the technology on eBay etc. The regulations are extant and remain a legal requirement when selling this technology. That is all we really need to know.

Fraser

[james_s]

I just find artificial limitations annoying. I believe that if I own a piece of hardware then I am entitled to use it any way I please, and that includes unlocking the full potential of the hardware. I was similarly annoyed the way earlier GPS hardware was deliberately crippled but at least that requirement went away eventually. I don't care what their reasoning is, it's stupid, nefarious people will always find a way around such limitations leaving legitimate users as the only ones impacted.

[xmjacky]

Hope that more people join the crack job

So you are posting from Iran (at least you removed country from your profile now) asking for help with cracking ITAR controlled technology, and those are your only posts on this forum. Right.

I think you are misunderstood, I came from China, concerned about "EEVblog" has been a long time, I just haven't posted any posts, The screenshot of the logic analyzer can prove what I said.

Studying this is just out of interest, it's for fun. make it work instead of turning it into a scrap metal.  I think you should be able to understand that feeling

[xmjacky]

Surely anyone wanting to use such a thing for missile guidance would have the resources to make reverse engineering and re-purposing the cameras trivial? I mean it seems like designing an entire new board to interface directly with the sensor would be trivial compared to designing a workable guided missile.

I guess that's the usual approach though, put a gigantic expensive and inconvenient lock on the front door but leave a window open.

If it's for military use, I don't think it's going to come here at all.
Just a FPGA can do, no matter which country has more resources to handle this

[Rasz]

Hope that more people join the crack job

So you are posting from Iran (at least you removed country from your profile now) asking for help with cracking ITAR controlled technology, and those are your only posts on this forum. Right.

I think you are misunderstood, I came from China

Why did you initially put Iran in your profile ?   
to help you at this point would be crazy 

[mikeselectricstuff]

Just a FPGA can do, no matter which country has more resources to handle this

You know FPGAs are export controlled as well, right?

[KE5FX]

Ask yourself this though...... if a technology has proven to be a tactical advantage over your enemies, why on earth would you make it EASY for them to obtain the same ?

Because the bad guys will find a workaround, and everyone else will be inconvenienced.  Pure security theater.

[Fraser]

Your statement appears to be that of a disgruntled consumer. Those who NEED the technology in a friendly country can have it. That has ALWAYS been the case. Just contact FLIR, do the paperwork and pay your money. Nobody is stopping you owning a high resolution, high frame rate camera.

It should also be remembered that the ONLY reason we have such advanced and affordable thermal imaging cameras these days is Military use and the Government military budgets that paid for the development of the sensors. The civilian world benefits from this military development because we were permitted to buy and own this technology. It was never a right to own it. The situation could have remained as it was when I started using such technology...  high spec cameras were only provided to friendly Governments and scientific/Industrial institutions, and then, only after security checks.

Many will likely disagree with me, but we should honestly be grateful for what we have, rather than whinging about thermal camera prices, current technology limitations or Government restrictions. People appear to not know just how fortunate they already are !

If anyone wants a really nice high quality thermal imaging camera, contact FLIR and I feel sure that they will help you obtain the obje t of your desires.

If you want high performance thermal imaging on the cheap via a hacked Autoliv camera....... you are on your own. But that is surely part of the 'fun' of hacking for those who do such

You have never had it so good ......... enjoy

Fraser

[lukier]

In principle I agree with the need of export controls - while there are anti-radar measures and counter measures for other guidance systems, the thermal response is difficult to hide. The only thing I know of is the thermal masking on the prototype polish PL-01 tank.

I'm curious though if the export controls are effective. Somehow it didn't stop the North Korean and Iranian nuclear programs - see Siemens PLCs (Stuxnet), Mitutoyo's coordinate measuring machines and similar cases. I think the BIS export control lists are a bit outdated (mentioning tape devices etc), while sometimes having quite broad definitions (e.g. anything that digitizes analog signal with such and such parameters). For example 1 GHz oscilloscopes are export restricted (to some countries) - 1 GHz scope is not a big deal nowadays, even Rigol makes them.

[Fraser]

I agree that the export controls need to be revisited. They are outdated in many areas. The intention was good, and appropriate when written. The World of technology moves very fast. This may be why the regulations are 'broad brush strokes' to cover current and predicted future developments.

One thing I know about policies and regulations is that there will always be people who wish to circumvent them through legal loop holes. That fact makespolicy writers very prone to creating complex and comprehensive documents. I speak from the position of being a security policy author.

Thermal imaging is a powerful tool and so the regulations on its distribution were made quite draconian. There has been relaxation of the regulation content over the years and the policy makers included the commercial world in the meetings that created the current policies and regulations. This was essential now that thermal imaging has become important in many industries outside the Military.

Finally, there will always be failures in policy, regulations and people. Regulations do not prevent people on-selling a controlled product to an inappropriate or illegal destination. The regulations do enable a Government to prosecute that individual or company however. Just look at the Iraqi SuperGun and Missile system Batteries legal cases. Both defendants went to prison so they could do no further harm. That was only legally possible because of the legally binding regulations controlling such exports.

Fraser

[james_s]

It's the same with many things, as the saying goes "The road to hell is paved with good intentions." I very often encounter rules and regulations designed to stop various forms of abuse that instead only affect legitimate users. All forms of copy protection are like this, security dongles, HDCP, all of the various protections on CD and DVD media, it all creates hassle for paying customers but the pirates have no trouble cracking it. I also fundamentally hate intentional crippling, if the hardware is capable of doing certain things, I want to be able to utilize all of those features without any of it being locked out. Also I think people should be able to enjoy the cool technology the world has to offer regardless of what country they live in. The real danger in the modern world is small groups of individuals with very crude and often improvised weapons. The Oklahoma city bombing, 9/11, the subway bombings, the recent concert bombing, the list goes on and on. None of these attacks used any sort of fancy controlled technology whatsoever. The world has moved on and outdated export controls do not offer a lot of protection.

[xmjacky]

Hope that more people join the crack job

So you are posting from Iran (at least you removed country from your profile now) asking for help with cracking ITAR controlled technology, and those are your only posts on this forum. Right.

I think you are misunderstood, I came from China

Why did you initially put Iran in your profile ?   
to help you at this point would be crazy 

I don't know how it was set up in Iran

[xmjacky]

Just a FPGA can do, no matter which country has more resources to handle this

You know FPGAs are export controlled as well, right?

Some high-end models do export controls, but this is only "EP3C25U256" and can be purchased anytime in China.

[zxase258]

Hi. I am also from China.
Last year I tried to crack the BMW Night Vision Thermal Imaging Camera. It is a version with analog video output and LIN control. I am fortunate to have a complete set of components (CAMERA+CONTROL UNIT) and a possible VIN code. I purchased a programmer that can decrypt 9S12 from taobao. I tried to read the binary file in the control unit. But the programmer seems to be faulty and may have damaged the control unit microcontroller---When entering decryption, the programmer The internal power supply makes a lot of noise, there may be a short circuit, and the data cannot be read. Later I bought a new microcontroller from the other vendors for the programmer. The data has now been read, but it is not certain if he is undamaged. And the data read out has 2Mbyte. And the microcontroller memory doesn't have that much space...
With regard to those export restrictions, it should be aimed at extremely backward countries. For example, if they want to connect their cameras to a TV, they can be used as an excellent reconnaissance equipment. Rather than rebuilding the camera's control system.

[xmjacky]

Can you add me to chat? 我的微信号xmjacky0415