Author Topic: Autoliv NV2 on Raspberry Pi  (Read 33248 times)

0 Members and 1 Guest are viewing this topic.

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Autoliv NV2 on Raspberry Pi
« on: December 22, 2018, 02:42:54 pm »
Some people may be interested - I made the NV2 (famous from Mike's teardown) work on a Raspberry Pi.  https://debugmo.de/2018/12/autoliv-nv2-teardown/

 
The following users thanked this post: oPossum, Fraser, BravoV, railrun, lukier, Cat, zbqpig, dnhkng, VGN

Offline railrun

  • Regular Contributor
  • *
  • Posts: 113
Re: Autoliv NV2 on Raspberry Pi
« Reply #1 on: December 22, 2018, 04:14:43 pm »
This is

AWESOME!
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 13165
  • Country: gb
Re: Autoliv NV2 on Raspberry Pi
« Reply #2 on: December 22, 2018, 06:53:26 pm »
It certainly is an Awesome investigation and result :-+

Fraser
If I have helped you please consider a donation : https://gofund.me/c86b0a2c
 

Offline railrun

  • Regular Contributor
  • *
  • Posts: 113
Re: Autoliv NV2 on Raspberry Pi
« Reply #3 on: December 27, 2018, 11:09:18 am »
How did you get the dump from the flash? Or have you found it somewhere in the internet?

Cheers
Martin
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #4 on: December 27, 2018, 12:51:38 pm »
I desoldered the flash chip and dumped it in a programmer. (Previously I sniffed the SPI traffic and reconstructed the content; but this doesn't yield all data as some data is only read after the security challenge is completed.)
 

Offline oddbondboris

  • Contributor
  • Posts: 38
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #5 on: January 09, 2019, 12:04:32 am »
wow nice work, i picked one of these up on the super cheap and was working on interfacing the bolometer directly but this will be much easier to duplicate if i can ever find the time.
the fpdlink II docs are also welcome.
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #6 on: January 09, 2019, 08:03:43 am »
I really want to work on documenting the bolometer interface next; How far have you gotten there?

I'd like to use them in a more compact design, and if I could replace the PCBs with a custom PCB then one could build a self-contained camera in the original case (say with WiFi/USB/... output). Also I have a couple of NV3 that I would want to use, but interfacing with the high-speed serial stream is even more annoying.
 

Offline lukier

  • Supporter
  • ****
  • Posts: 634
  • Country: pl
    • Homepage
Re: Autoliv NV2 on Raspberry Pi
« Reply #7 on: January 09, 2019, 12:53:13 pm »
Awesome job tmbinc!

Back when I looked at this (2015) I've also downloaded the SPI flash, saw some strings there in the FW part, but I didn't know about cpu_rec and binwalk wasn't helpful to say which architecture it is so I kind of put this project on the shelf and moved to do something else. I'm glad you had more determination and skill :)

I'm thinking, is the FPGA on the receiving end necessary? Maybe just DS90C124 deserializer + GPIO to USB IC (like Cypress FX2/FX3) could do to get the pixels to the PC in real time?

Somebody on the forum approached the problem from the ROIC end and ignored the FPGA, CAN and security which is also an option I guess.

Did you figure out this dense SMD pin header on the power board? Maybe it is the raw 14 bit video like in Tau cameras, so one could tap there and don't bother with FPD Link?
 

Offline cq-317

  • Contributor
  • Posts: 38
  • Country: cn
Re: Autoliv NV2 on Raspberry Pi
« Reply #8 on: January 10, 2019, 12:27:01 pm »
Awesome job tmbinc!

Back when I looked at this (2015) I've also downloaded the SPI flash, saw some strings there in the FW part, but I didn't know about cpu_rec and binwalk wasn't helpful to say which architecture it is so I kind of put this project on the shelf and moved to do something else. I'm glad you had more determination and skill :)

I'm thinking, is the FPGA on the receiving end necessary? Maybe just DS90C124 deserializer + GPIO to USB IC (like Cypress FX2/FX3) could do to get the pixels to the PC in real time?

Somebody on the forum approached the problem from the ROIC end and ignored the FPGA, CAN and security which is also an option I guess.

Did you figure out this dense SMD pin header on the power board? Maybe it is the raw 14 bit video like in Tau cameras, so one could tap there and don't bother with FPD Link?

I think even if the dense SMD pin header on the power board is the 14 bit raw video, it also has no value, because this is just the sensor data, the FPGA also needs to calculate the final image, including various corrections and signal processing.
 

Offline oddbondboris

  • Contributor
  • Posts: 38
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #9 on: January 10, 2019, 06:05:59 pm »
I really want to work on documenting the bolometer interface next; How far have you gotten there?

I'd like to use them in a more compact design, and if I could replace the PCBs with a custom PCB then one could build a self-contained camera in the original case (say with WiFi/USB/... output). Also I have a couple of NV3 that I would want to use, but interfacing with the high-speed serial stream is even more annoying.
i've got one rigged up pretty cleanly, cpld and an fx2 is all you really need, just havent had the time to get the sensor working properly. if you want standalone, a beaglebone and a cpld should be about ideal with it's fast gpio and dedicated io processors, shouldn't have an issue implementing a 28 bit dual input shift register/level shifter in basically any cpld or fpga that supports 2.5v io
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #10 on: January 11, 2019, 07:59:40 pm »
Holy hell the madmen actually did it! Now I want one of those even more than I already did for their lens stack
 

Offline LesioQ

  • Regular Contributor
  • *
  • Posts: 67
  • Country: pl
  • Every king should be naked.
Re: Autoliv NV2 on Raspberry Pi
« Reply #11 on: January 13, 2019, 11:46:06 am »
I really want to work on documenting the bolometer interface next; How far have you gotten there?

I'd like to use them in a more compact design, and if I could replace the PCBs with a custom PCB then one could build a self-contained camera in the original case (say with WiFi/USB/... output). Also I have a couple of NV3 that I would want to use, but interfacing with the high-speed serial stream is even more annoying.

This is about where I postponed the work:

https://www.eevblog.com/forum/thermal-imaging/interfacing-isc0601b-bolometer/msg1495546/#new

Some static images captured, but without FPGA work - no chance for faster frame rate, I'm afraid.

Piotr.K
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #12 on: January 16, 2019, 07:23:53 pm »
Interesting, I wonder if it could be done on one of the Lattice series, considering it appears to be parallel, a few SERDES might do the trick.
 

Offline oddbondboris

  • Contributor
  • Posts: 38
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #13 on: January 17, 2019, 09:26:15 pm »
Interesting, I wonder if it could be done on one of the Lattice series, considering it appears to be parallel, a few SERDES might do the trick.
the gearboxes in lattice fpgas do roughly the same thing if fed with a proper clock, you can probably abuse the ddr3 deserializer

so i did a bit more research on fpdlink II and it seems ti makes a fpdlinkII to mipi csi bridge, somone well versed in writing csi drivers/firmware could ideally interface the i2c channel on the csi interface to the can with a cheap micro and get this working with hardware accellerated capture on a raspi or some other sbc with a csi connector for <$20 bom cost and with a proper fpdlinkII interface with all the noise immunity that brings.  even if an exploit can't be found for reading the key out of the camera without disassembly, (assuming the key isn't a static value), reading a configuration rom isn't exactly major surgery.
« Last Edit: February 04, 2019, 02:48:27 pm by oddbondboris »
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #14 on: March 27, 2019, 12:18:54 am »
I know you weren't looking to give out the key you used to complete the UDS challenge but I can't seem to repeat your process. I got the flash dumped, I can see strings at the end of the flash, including the few lines you showed, cpu_rec identified microblaze, but whenever I try to run data2mem it crashes. I've tried windows and linux, and 2 different versions of data2mem, they all give the same response.

Code: [Select]
tripp@MoonServ2:~/thermal$ /tools/Xilinx/SDK/2018.3/bin/data2mem -bt flash.bit -bm dummy.bmm -d


INTERNAL_ERROR:Data2MEM:45 - Memory allocation leak of 136 bytes at 0x0227E7A8 for a 'fileIODescriptorType' record.
    Total memory in use at allocation was 2400 bytes.
    Source file "FileUtils.c", line number 1853.

Memory contents:

 0227E7A8:   00 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00   ................
 0227E7B8:   10 EA 27 02 00 00 00 00 18 0F 28 02 00 00 00 00   ..'.......(.....
 0227E7C8:   A0 86 01 00 00 00 00 00 00 00 00 00 A0 86 01 00   ................
 0227E7D8:   A0 86 01 00 01 00 00 00 00 00 00 00 00 00 00 00   ................


INTERNAL_ERROR:Data2MEM:45 - Memory allocation leak of 33 bytes at 0x0227E8A8 for a StrDup.
    Total memory in use at allocation was 2536 bytes.
    Source file "FileUtils.c", line number 1858.

Memory contents:

 0227E8A8:   01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
 0227E8B8:   66 6C 61 73 68 2E 62 69 74 00 00 00 00 00 00 00   flash.bit.......
 0227E8C8:   00                                                .


INTERNAL_ERROR:Data2MEM:45 - Memory allocation leak of 100003 bytes at 0x02280F18 for 'char' data.
    Total memory in use at allocation was 2569 bytes.
    Source file "FileUtils.c", line number 1919.

Memory contents:

 02280F18:   FF FF FF FF AA 99 55 66 30 00 80 01 00 00 00 07   ......Uf0.......
 02280F28:   30 01 60 01 00 00 00 7C 30 01 20 01 00 20 31 E5   0.`....|0. .. 1.
 02280F38:   30 01 C0 01 01 C2 E0 93 30 00 C0 01 00 00 00 00   0.......0.......
 02280F48:   30 00 80 01 00 00 00 09 30 00 20 01 00 00 00 00   0.......0. .....

 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #15 on: March 27, 2019, 04:04:46 pm »
Hi,

I realized I hand-waved a little bit too much here, sorry - you need to convert the .bin file (from the flashdump) to .bit, for example by pre-pending this header:


00000000: 0009 0ff0 0ff0 0ff0 0ff0 0000 0161 001a  .............a..
00000010: 746f 702e 6e63 643b 5573 6572 4944 3d30  top.ncd;UserID=0
00000020: 7846 4646 4646 4646 4600 6200 0d33 7331  xFFFFFFFF.b..3s1
00000030: 3230 3065 6674 3235 3600 6300 0b32 3031  200eft256.c..201
00000040: 382f 3036 2f32 3900 6400 0931 363a 3535  8/06/29.d..16:55
00000050: 3a34 3200 6500 0753 94                   :42.e..S.

 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #16 on: March 28, 2019, 09:22:33 pm »
That header worked perfectly. Though now I'm struggling to find the actual bootloader, it looked like it might just be inverted since data2mem dumps row 13 first, but I can't get any reasonable looking instructions. So far I've tried it in the original order from data2mem, making the rows go from 0-13, and doing byte reversal, but nothing matches, is there a standard entry point for the bootloader? I tried just seeing if I could find where it was loading from spi but I can't find anything documenting what the standard register addresses are. It's not in the IP ref or in the spartan datasheet.

I at least got somewhere with the flash, looks like all the code starts at 0x7E0000, but it doesn't look like there's an obivous main loop, so I'm going to have to trace it until I find that bit check you mentioned or the can bus stuff if I can figure out the registers.

On another note it looks like the cameras are likely keyed to thier modules and it's not a universal key for all the cameras. Really it's not surprising since this has been standard for quite some time with ECU's and immobilizers. In the strings output from the flash I found a VIN matching a 2013 Audi S6, followed by the model and sw of the camera, and then the serial of the camera as indicated on the sticker. While there's a private bus between the camera and the module, the module is still on the main can bus for the vehicle.
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #17 on: March 28, 2019, 09:51:48 pm »
It should be 2 cols x14 rows, and Microblaze typically starts with a number of "BRI" (or something) instructions, i.e. 0xB8 ... No inversion is necessary, just pick the right rows and order them correctly.

I have a number of NV2s, from both Audi and BMW (I don't have one from Mercedes), and they all use the same key. The VIN is not used for unlocking in the camera.

If someone has the firmware from the ECU (decrypted; all I have is the encrypted firmware update) I'd be interested. Otherwise I need to try to dump the S12x...
 
The following users thanked this post: ArsenioDev

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #18 on: April 16, 2019, 03:42:45 am »
Finally got the camera to wake up, ended up throwing me for quite a few loops. Mainly there's 2 massive capacitors on the can lines inside the camera that were causing massive ringing making it impossible to talk to. After removing those capacitors from the circuit I was able to figure out that the camera wanted 100kbit/s and it finally started talking. I'm going to build up a proper deserializer board using the companion TI chip for the serializer in the camera and go from there instead of the fpga route. I think it might still be easier overall than it was figuring out the can bus issues and disassembling the microblaze code.

Before cap removal:


After cap removal:


Capacitors in question:
« Last Edit: April 16, 2019, 03:45:07 am by Treehouseman »
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #19 on: April 16, 2019, 07:07:51 am »
Huh, interesting, the caps never posed a problem. I've used all of a Lawicel CANUSB, 8devices USB2CAN, a non-isolated custom CAN board as well a cheap mcp2515 (modified to 3.3V for raspberry PI) without having trouble. The nice thing is that all ports on the camera are 12V tolerant - which means you hook up any permutation of these wires to GND, 12V, CANH, CANL, LVDS+, LVDS- without any damage to the camera. (This includes for example 12V on LVDS ports, or inverted polarity on the power).

I would be interested in a deserializer board as well, maybe coupled to a small FPGA or directly going into a SoC (if possible).
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #20 on: April 16, 2019, 11:51:25 am »
Yeah, I had tried multiple drivers and chips, mcp2515, esp32, NEC 1050, mcp2561, and finally I ordered the same ti chip as the camera and put it on a mcp2515 board (I ordered mcp2515 boards after a breadboarded one didn't work, it wasn't until I removed those caps it magically had a proper signal, I spent more time stuck on can bus than the fpga.

It's nice to know all pins are 12v tolerant, makes me a bit more reassured I'm not going to easily damage it, I'm still not 100% on how video decoding is going to work, but I figured the right deserializer chip was a good place to start, I don't have your exact fpga to try your code on and it should just spit the parallel data out once connected, I just need something to read it in.
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #21 on: May 14, 2019, 04:51:29 am »
Alright guys, I finally got a ping on my radar of a DEAL on the NV2 and since I already have my ICEBreaker FPGA on hand, I figured why not go for it!
Threw down the cash and now it's a waiting game while I download the software to extract the key and grab the docs I need. Worst case I can resell the camera on here to one of you all
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #22 on: June 15, 2019, 04:05:44 am »
Progress has been made! A deserializer daughterboard attached to the FT2232H dev board is dumping data in a usable fashion, this is just a raw dump of the port with the upper 6 bits opened in gimp as a raw image.
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #23 on: June 16, 2019, 06:00:47 am »
And we now have 14 bit data coming in via the FTDI board! Images have to be manually assembled right now, but the data is coming through as valid. There's some issues with the decode, not sure if it's the FTDI or if it's the deserializer. Still some major progress, now to just work out software to automatically stitch images together.
 
The following users thanked this post: railrun

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #24 on: June 16, 2019, 07:58:57 am »
\o/

Great to see this working!
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #25 on: June 17, 2019, 02:24:41 pm »
Got a bit more progress, I'm really starting to fight signal loss, I'm going to have to rework the connection between the deserializer and the camera, I'm not sure what makes the difference between a good capture and a bad one, without touching the camera I can get a good 30 second capture and then instantly running a second one get maybe one or two good frames out of the entire thing.

 

Offline LesioQ

  • Regular Contributor
  • *
  • Posts: 67
  • Country: pl
  • Every king should be naked.
Re: Autoliv NV2 on Raspberry Pi
« Reply #26 on: June 18, 2019, 02:53:04 pm »
Is this video without any shutter operation ? Not bad if without calibrations.
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #27 on: June 18, 2019, 03:44:47 pm »
The shutter isn't really used much here, but you can see it actuate in the first capture in the video, there's some "blank" frames but it you look at it frame by frame you can see the shutter sweep in and out. If you look at a Flir E4 or Seek as a "photometric" camera this is a camcorder. This doesn't supply metered values or at least it's not intended to. It may be possible to "calibrate" it's output and tell what temp something it but it's intended to only show differences in heat, it's only used as a people/animal/car detector at night to show things out of range of your headlights. The shutter does actuate on this but I don't know if it's the sensor requesting it for internal stuff, or if it's the FPGA doing it and maybe the FPGA is doing some crude internal auto exposure or maybe just sanity checks. The shutter can be controlled via can bus commands per Tmbinc's documentation but I haven't figured out anything other than the UDS challenge.
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #28 on: June 18, 2019, 06:49:01 pm »
For shutter operation, you need to send on can_id 0x401 the following data:

"c8 64 18 00 04 00 00 00"
"c8 64 18 00 00 00 00 00"

Byte 7 defines what's shown during the shutter operation (black, uncorrected, test pattern).
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #29 on: June 18, 2019, 06:57:13 pm »
Do you have a list of what commands you saw in the microblaze code? Or at least where the block of code is that was processing the messages? I never actually found that, I got the key from simply finding the 16 bytes in an out of place location and seeing that there was a ref to it. I was just doing trial and error with high entrophy 16 byte blocks.
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #30 on: June 18, 2019, 08:30:35 pm »
The key is in the boot block (initialized BRAM from the bitstream); the CAN handlers are in the firmware.

This is what I have there: (All names from me of course)

ROM:2200E970 00 00 04 01     CanFuncTable:   .int 0x401              ; DATA XREF: _InitCan:loc_22001B6C↑o
ROM:2200E974 22 00 1A 10                     .int CanCamFuncControl
ROM:2200E978 00 00 03 04                     .int 0x304
ROM:2200E97C 22 00 19 20                     .int CanCamSync
ROM:2200E980 80 00 03 13     CanFuncTable2a: .int 0x80000313         ; DATA XREF: _InitCan+6C↑o
ROM:2200E984 22 00 04 88                     .int CanCamStatus
ROM:2200E988 00 00 03 11                     .int 0x311
ROM:2200E98C 22 00 05 FC                     .int CanCamSerialNumber
ROM:2200E990 00 00 03 12                     .int 0x312
ROM:2200E994 22 00 05 70                     .int CanCamInitRequest
ROM:2200E998 80 00 03 03     CanFuncTable2:  .int 0x80000303         ; DATA XREF: _InitCan+78↑o
ROM:2200E99C 22 00 04 88                     .int CanCamStatus
ROM:2200E9A0 00 00 03 01                     .int 0x301
ROM:2200E9A4 22 00 05 FC                     .int CanCamSerialNumber
ROM:2200E9A8 00 00 03 02                     .int 0x302
ROM:2200E9AC 22 00 05 70                     .int CanCamInitRequest


So, 0x401 is handled in CanCamFuncControl (22001A10).
 

Offline oddbondboris

  • Contributor
  • Posts: 38
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #31 on: June 19, 2019, 01:26:58 pm »
Got a bit more progress, I'm really starting to fight signal loss, I'm going to have to rework the connection between the deserializer and the camera, I'm not sure what makes the difference between a good capture and a bad one, without touching the camera I can get a good 30 second capture and then instantly running a second one get maybe one or two good frames out of the entire thing.


if you have a cheap fx2 based logic analyzer (or any streaming output logic analyzer) you might wanna hook it up to the deserializer output and see if what the ftdi chip is spitting out matches.
 

Offline Musclor

  • Contributor
  • Posts: 19
  • Country: es
Re: Autoliv NV2 on Raspberry Pi
« Reply #32 on: July 15, 2019, 12:59:17 am »
This is pure genius and if related to the old thread where someone had asked if such thing was even possible, even better. Absolutely inspiring! Bravo!
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #33 on: May 13, 2020, 01:47:16 am »
Using dsPIC33EV256GM102
<click> <click>
Thanks to tmbinc, Treehouseman, Lennie, and Johan for their help with this.

 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #34 on: May 17, 2020, 10:28:30 am »
Using the ubiquitous ELM327 cable

 

Offline conmega

  • Contributor
  • Posts: 23
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #35 on: May 20, 2020, 10:18:31 pm »


Well this is the best I have gotten out of it so far.
Still some noise issues due to me just hooking this straight to the FPGA using its internal termination.

Thanks to oPossum and Treehouseman for starting me down this rabbit hole.
Also thanks to tmbinc. I took the Migen implemented de-serializer and  decoder and was able to get it ported over to 7-Series parts (mostly changing the SERDES units over to the new ones and dealing with different clocking resources).
I took just the fpdrecv.py and basically built it out as top then implemented some clock domain crossing/fifo and LCD controlling in VHDL and glued the two together.
Its been a heck a week working on this.
« Last Edit: May 20, 2020, 10:23:57 pm by conmega »
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #36 on: May 20, 2020, 10:27:56 pm »
Conmega, that is one IMPRESSIVE quick build you have there. Only wish I could poke around in it, been thinking of how to port it to the ECP5 boards I have, should be enough LUTs to drive a smaller display and do the CANBUS on board.
 

Offline conmega

  • Contributor
  • Posts: 23
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #37 on: May 21, 2020, 12:04:25 am »
Thanks, but most of the heavy lifting was done by others, especially the decoder/deserializing written by tmbinc.
I simply had to learn a bit of Migen to get it move from a Spartan-6 target to 7-Series parts.

My design as it currently sits uses around 700~ LUTs, 1100~ FlipFlops, and two half BRAM units, which are 36x1024 and can be used as 18x1024 "half" BRAMs.
For reference this is only about 2%~ of the entire Artix-7 100T... So I bet if you had the bandwidth or implemented an even lower sample rate over-sampler you could probably get away with running this on something much smaller and simpler like an ICE40, I will probably plan to just target a small Artix-7 or Spartan-7 myself.

The big thing is to use tmbinc's decoder/deserializer directly without modification. You need at-least a 1080Mbps capable SERDES. Looking at Artix-7 and Spartan-7 it looks like speed grade 1 always is 950Mbps capable and speed grade 2/3 is 1250Mbps.

Of-course this is assuming you don't want to modify tmbinc's sampler code which over-samples by 7.7x as described on his website.
You could probably modify it to work with 6x oversampling (840~Mbps) to allow it to work on more common/cheaper/smaller parts.
But its easiest to just use a speed grade two 7-series part. I had to change the clocking.py to implement 7-series primitives for the MMCM and clock buffers. And the Serdes change was a bit more complex but not really all that bad. To get 1080Mbps you use the ISERDES2 in "OVERSAMPLE" mode, which has a 4 bit output but allows for QDR sampling. So you feed a 2 times data rate clock, in this case to match tmbinc's code which expects a 7.7x sampling it would be a (135Mhz*2) 270Mhz clock then you get 4 bits every 270Mhz which you put into an 8-bit register every 135Mhz and you get your 8-bit 7.7x sample data to feed into the sample.

So its not that bad, honestly it took me a few days to learn enough Migen to get stuff built and then a few days to understand the timing coming out of the sampler and to just write sane VHDL... I am still quite new to FPGAs so there was a good 4-5 days of buggy glitchy video on the screen :)

I will probably try and get his sampler down to only a 6x oversample rate so I can use a common and cheap FPGA board (like the Cmod S7) with a speed grade 1 part on it for implementing this into a hand-held device.
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #38 on: May 21, 2020, 01:08:53 am »
The good news is even the basic ECP5 Serdes are 3.2Gb/s! Hmm, guess it's time to nose to the grindstone on the Migen and Litex stuff eh? Honestly I'm trying to get this all running on FOSS flows, gonna do the designs for a custom breakout of the FPDlink. Prob gonna make a few and give em to friends then open source the design.
 

Offline conmega

  • Contributor
  • Posts: 23
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #39 on: May 22, 2020, 10:27:38 pm »


And now glitch free video with hardware histogram equalization!

This uses the current frame to build a histogram in BRAM1 then between frames using downtime to built up an equalized histogram in BRAM2 (in this case equalized to 6-bits for the 6-bits of green available).
Then using the new equalized histogram in BRAM2 we apply that to the next frame's pixel data on the fly while building up a new histogram for the next frame in BRAM1 again. This is while not 100% accurate from frame to frame its close enough and extremely low latency. It is also space efficient requiring about 1/3rd the amount of BRAM to store both histograms as it would take to store one entire frame. Not to mention the logic to say offload this to a microprocessor for processing via software :)

I also adjusted the focal point of the camera by unscrewing the lens outward a few turns as Treehouseman found will bring the focal distance in.
I was about 2ft~ away from the camera and I had gotten it to about 18~in or so before!
 

Offline conmega

  • Contributor
  • Posts: 23
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #40 on: May 23, 2020, 04:06:25 pm »


And now with palette support!
Thanks to oPossum for generating a few palettes for me to test out.

Also decided to show off the histogram equalization a bit with the Metcal iron :)
 

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #41 on: June 28, 2020, 05:33:39 pm »
I'm not asking for a key, but is there a dump of the flash uploaded somewhere? It seems silly that we all have to desolder the SPI chip over and over again.

And thanks to all the posters on this thread so far, awesome work!

EDIT:
Note: I don't have an NV2, so I'm doing this without feedback.

Not made much progress yet, but I will post a (growing) list of things that DON'T seem to work  |O

I *assume* this is how the unlock protocol works: the camera sends you a 'Seed', as per Tmbinc's blogpost description, and this data is 12-bytes random plus 4-byte serial number. You have to use a secret 'key' and do an HMAC-MD5 digest of the 'Seed' and send it back. The camera has done the same thing and compares its calculation with yours, and if they are the same, it unlocks itself. No secret key, no unlock. It seems simple enough...

I tried using the screenshots from oPossums post, where he showed 2 seed/key pairs, 16 bytes each.
Code: [Select]
seed:    8D E9 AD 49 DC 8E 68 9E CE F6 89 20 3C 25 DB 53
Key:      90 CC 53 17 64 A1 66 1C 58 0A 88 16 EB 39 50 52

Code: [Select]
seed:    7E 71 63 31 D5 5E EF 26 A1 9C AA A1 3C 25 DB 53
 Key:      BA CC 53 C6 8A 79 CF C2 46 AD 56 AE C3 B6 A4 68

Then, because I'm dumb, I just brute-forced iterated through the whole BIN file, reading every sequential 16-byte block as a potential private key.
I used the standard python hmac library.
Code: [Select]
import hmac
output = hmac.new(password, seed).hexdigest()

then I tried to match the posted 'Key' with the calculated output, and the whole process of 8 million MD5-HMAC digests (the bin file is about 8 Mb) takes under a minute, but no luck finding the key  |O  I might try iterating backward next, because why not?

Just in case there is data in a different format, I did the same again for the Data2mem output file, after extracting out the binary data. That didn't work either.

I was a bit confused about the BRAM columns/rows memory layout (I have used a MicroBlaze core before, but I just drew in wires from the Microblaze to the BRAM controllers in Vivado, I have no idea how they work under-the-hood), but I tried to take the 4 rows that kind of looked interesting and interleaved them together in various ways. From the discussion here, it seems that they have to be interleaved somehow (maybe the MicroBlaze reads across down rows?). The most interesting 4 rows have most of the data at the beginning, so maybe stacking the rows and then reading in columns makes sense...  Eyeballing they merged HEX data it didn't look like much beyond looking at a The Matrix screensaver. I didn't find a nice, simple, stand-alone MicroBlaze disassembler, plus I'm too lazy to learn IDA for a single project  :--

EDIT:

I found the key last night using the approach above, and the tip from tmbinc and oPossum below (thanks guys!). Just by knowing the XOR ordering (x4, as you don't know the byte offset), and overlaying that with the permutations on interleaving the 4x BRAM rows (x24), running over every possible private key in the data2mem dump (2Mb) shouldn't take more than half an hour brute force (~200 million potential keys). This should be faster than installing all the software you need to go the disassembler route. Using the extra tip from oPossum makes the calculation a bit faster. The timing below including passing the Data2Mem file for BRAM data too :)
Code: [Select]
34.2 ms ± 453 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
« Last Edit: June 30, 2020, 12:17:50 pm by dnhkng »
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #42 on: June 29, 2020, 05:18:32 pm »
The seed/key pairs are correct. Your code should be correct as well.

However just iterating over the flash file will not work because it only stores the XOR value (55AA33CC) from the data in BRAM. If you xor them together, and put them in the right order, it should produce an 16-byte ASCII string with two names.
 
The following users thanked this post: railrun, dnhkng

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #43 on: June 29, 2020, 05:19:14 pm »
By the way, if someone wants to post the key, I'm fine with that, but _I_ will not be that person :).
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #44 on: June 29, 2020, 05:41:07 pm »
Some hints...

Row 06 XOR CC
Row 09 XOR 33
Row 07 XOR AA
Row 04 XOR 55

Reply #33 (not the screen cap)


 
The following users thanked this post: railrun, dnhkng

Offline railrun

  • Regular Contributor
  • *
  • Posts: 113
Re: Autoliv NV2 on Raspberry Pi
« Reply #45 on: June 30, 2020, 08:21:29 am »
So many hints, but still no approach.  |O
@dnhkng Thanks for your mail. Unfortunately your mailbox is full and I couldn't answer your last mail.
 

Online Fraser

  • Super Contributor
  • ***
  • Posts: 13165
  • Country: gb
Re: Autoliv NV2 on Raspberry Pi
« Reply #46 on: June 30, 2020, 11:42:51 am »
I have one of these cameras and it’s associated original controller languishing in my “may be useful someday” cupboard  ;D

This is a very interesting thread but I am no coder and much of it is beyond me. Maybe one day someone will offer an adapter board and code to release the Autoliv camera from its shackles. If that day comes, I will be a customer for the solution. Until then, the Autoliv camera remains an interesting paperweight in my collection  ;D

To the knowledgeable participants in this development work.... I salute you  :-+

Fraser
If I have helped you please consider a donation : https://gofund.me/c86b0a2c
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #47 on: June 30, 2020, 08:10:49 pm »
I have one of these cameras and it’s associated original controller languishing in my “may be useful someday” cupboard  ;D

Well for the NV2 there's not much need for the controller at this point ;)

Now the NV3, we have a camera, we've started looking at it and tried to figure out how to pull the softcore out of the flash, but it's proving to be quite a task. A controller for that might prove to be helpful, or at least a capture of the comms from when it unlocks to see what commands it's using.
 

Offline conmega

  • Contributor
  • Posts: 23
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #48 on: July 02, 2020, 10:53:18 am »
I have one of these cameras and it’s associated original controller languishing in my “may be useful someday” cupboard  ;D

This is a very interesting thread but I am no coder and much of it is beyond me. Maybe one day someone will offer an adapter board and code to release the Autoliv camera from its shackles. If that day comes, I will be a customer for the solution. Until then, the Autoliv camera remains an interesting paperweight in my collection  ;D

To the knowledgeable participants in this development work.... I salute you  :-+

Fraser

I have been developing a little board that allows one to hack an RJ-45 onto an Autoliv2 and then my board has an S7-Mini FPGA board, RJ-45 for the camera allowing one to use any standard ethernet cable, and a USB-PD spoof board allowing one to power the entire thing off a USB-C PD capable power bank or supply. It also has a 320x480 LCD on it to allow for viewing of the full resolution.

Now I have been working on this board but have been hesitant to want to sell them to others for the same reason tmbinc is hesitant to release a key...
I mean I could sell it without a key but entering a key over a 4-button keypad or having to rebuild a bitstream to throw it in both seem like silly things. So likely I will just be keeping this between me and a few people who have been hacking on this with me.
But you can see what I have made here:
https://twitter.com/ConnorKrukosky/status/1271046037742989313

I could also sell the boards with them artificially locked to 9-fps with some challenge to get 30-fps...
Probably something more complex than a konami code or something though ;)

The big challenge ahead on the NV2 is getting temp data correlated to what comes out of the camera... That is where I am currently procrastinating... Otherwise I have a RiscV softcore doing the unlock as seen in that tweet and I have all the palette application and histogram done in hardware. I also have the RiscV able to load other palettes off SPI.

Honestly though it would be smartest to design a board that had everything integrated and not using an expensive FPGA dev board on it... Its an overkill FPGA for the application too.

But I was just focused on getting something I can have one of and use as a tool. Not something I planned on selling.
 
The following users thanked this post: railrun

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #49 on: July 02, 2020, 11:21:32 am »
Really glad to see all this progress here!

The two elephants in the room are "signal cleanup" and "temperature extraction".

For the "signal cleanup", it would be interesting for example to look at a Flir E40 (which uses the same sensor), and see which steps they do. The NV2 has a lot of "structured" noise that I think the Flir E40 is able to clean up (not sure if at the expense of sensitivity).

For the "temperature extraction", I'm pretty sure we have all the data we need - there's calibration data stored on the device that can be read via UDS, and there's a number of temperature sensors that is transmitted via CAN and via LVDS.

The NV2 does automatic FPC, however it's not very good at it, there's a significant "remain" of previous image data. I can't judge why really. I've tried to poke FPGA registers to receive the raw sensor data but couldn't do it. It would be great if we could figure out what corrections the FPGA actually does. We can grab the intermediate NUC pictures from either flash or RAM, and see how they compare to the sensor calibration data stored in Flir Exx cameras. I suspect the algorithm to be identical or very similar - likely it's just an offset and a scale?

Let's turn the NV2 into a top-notch thermal camera!
 

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #50 on: July 02, 2020, 08:12:04 pm »
I'd like to try working on the image clean up. That more my thing. Conmega, the S7-mini board looks great, I'll order one from Trenz. It seems like a fair price to avoid having to lay out and solder a chip like that.

I think I might have the same parallel LCD from your first video in my parts-box too. Can you access image data easily over USB with your S7-mini setup? I'll need lots of data to play with for image correction. Maybe a linear fit in enough, but from the temperature sensors I have used in the past, it's been a more complex model.

To get a good cleanup, it might be good to really defocus the lens to build a pixel-correction array.  Defocused imaging a piece of melting ice and boiling water gets you 2 fixed points to work with.

EDIT:
I've written micropython code for the unlock (HMAC-MD5/MPC2515 driver), which should be great on FPGA using FuPy https://fupy.github.io/. With a serial interface into the REPL, you can mess around on the CAN bus directly (no messy compile/upload steps).  I'm using as MCP2515 board with SPI, because they are cheap and easy. Maybe a CAN bus IP on the FPGA connected to a transceiver IC (MCP2551), and hooked up to FuPy on a RiscV core is a nicer way to go.

Before and after 'clickclick' traces  :popcorn:
« Last Edit: July 07, 2020, 04:59:52 pm by dnhkng »
 

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #51 on: July 19, 2020, 09:52:41 pm »
Kinda got some pictures out of the camera!

well not really...

I hacked together a hand-soldered monstrosity, wired up to a fake Saleae Logic thingy, to capture the upper 5 bits of thermal data, and the h/v-sync and clock lines, then chucked it all into a numpy array to poke around. I haven't figured out the logic of the video signal yet, so I just grab every line I can that has 326 pixels between syncs, and I get something that looks like my hand holding a lighter over and over (I captured 300 mS of data, so 10ish or so frames I hope).

It looks like either blocks of rows are scrambled or just missing entirely, kinda weird...

@Treehouseman - Is this the same issue you had?



« Last Edit: July 19, 2020, 09:55:10 pm by dnhkng »
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 58
Re: Autoliv NV2 on Raspberry Pi
« Reply #52 on: July 19, 2020, 10:10:39 pm »
I'd say same symptom and same general fault but likely different cause. I was suffering capture rate issues, with things being pushed to the limit and resulting in temp fluxuations in the camera causing me to lose sync. For your issue your'e not consistently capturing data it seems, looks like you're getting bursts of valid data. I know we were looking at the FX2 (aka your clone) for possible data dumping, but it would have required custom firmware for all the bits,
 
The following users thanked this post: dnhkng

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #53 on: July 20, 2020, 08:38:44 am »
There are empty lines between the valid lines, which you need to filter out. I think this is done to keep the output data rate constant, so it doesn't need to be locked with the sensor scan rate.
 
The following users thanked this post: dnhkng

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #54 on: July 20, 2020, 09:33:49 am »
Got it!  It seems the ground wires are in fact useful... 
@ Treehousea and tmbinc - Thanks for the tip! I thought the data was sent bursty, but it seems if you hook up the ground wires its a continuous stream!  :-+

Still a bit confused about the frame sync bit, I seem to get more than I would expect (bit 15 goes high 4 times per frame, but its a regular pattern), but I'll work it out, it's a fun puzzle.

Pretty happy so far, all done in Python, for less than $20 in bits and pieces. Haven't had to dust off my Arty A7 yet!
« Last Edit: July 20, 2020, 11:19:51 am by dnhkng »
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #55 on: July 20, 2020, 11:56:12 am »
Bit 15 is high for the first line and line number 20 (not the 20th line). So twice per frame.
 

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #56 on: July 20, 2020, 07:41:53 pm »
Am I crazy or the homebrew NV3 thread just vanished?
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
 

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #58 on: July 20, 2020, 08:33:26 pm »
Thank you and sorry for the panic attack :)
 

Offline kmike

  • Regular Contributor
  • *
  • Posts: 59
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #59 on: August 06, 2020, 07:37:38 am »
Hello,

if I understand correctly, the bits 16 to 23 are not used after deserialisation? I plan to use the DS90C124 for the task and would like to save some FPGA pins  :)

Br,
mike
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #60 on: August 06, 2020, 07:45:53 am »
Bit 14 and 17 to 23 can be ignored.

0 to 13 contain video or line/frame number
15 and 16 are sync
 
The following users thanked this post: kmike

Offline kmike

  • Regular Contributor
  • *
  • Posts: 59
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #61 on: August 06, 2020, 08:01:19 am »
thanks!

just for my planning: how much current does the camera need on the 12V rail, when it is operating and unlocked?

Br,
miike
« Last Edit: August 06, 2020, 08:37:44 am by kmike »
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #62 on: August 09, 2020, 09:24:25 am »
Deserializer and MCU board. Currently connected to a QMTech Cyclone 10 dev board.




First light


 

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #63 on: August 09, 2020, 04:27:32 pm »
Deserializer and MCU board. Currently connected to a QMTech Cyclone 10 dev board.


Congratulations. When will you launch it on Tindie? :)
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #64 on: August 11, 2020, 09:33:09 am »
Low frame rate (~5 fps) video via serial port.


 

Offline kmike

  • Regular Contributor
  • *
  • Posts: 59
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #65 on: August 14, 2020, 10:57:17 am »
would someone be so kind and send me a dump of the spi flash?

My camera looks pretty new, and I dont want to disassemble it

Thanks in advance!

br,
mike
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #66 on: August 18, 2020, 02:11:36 pm »
Full frame rate (30 fps) and full resolution (324 x 256) using an FTDI FT2232H in 245 async mode.
Video shows palette selection, dynamic palette (DP), and histogram equalization (HEQ).


 

Offline Imolteor

  • Newbie
  • Posts: 1
  • Country: gb
Re: Autoliv NV2 on Raspberry Pi
« Reply #67 on: December 26, 2020, 11:28:58 pm »
Hi, I realise I'm a bit late to this party. I'm also brand new around here (please excuse me if I have muddled any etiquette rules). I'm a postgraduate at a university in the UK looking to use thermal imaging for wildlife conservation.

We can't afford much (given life) and this sensor is awesome, plus the work thats been done.

I'm going through the thread looking to piece together what I need but I'd really appreciate a summary to getting started.

If I can get one of these off the ground on my end I'll be pumping a lot of work into using it. Any software I write to use it I'll make sure to post here for you guys as well, in case you find it useful.
 

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #68 on: January 06, 2021, 01:34:38 pm »
You need just need 2 things:
  • A way to unlock the camera
  • Some fast hardware to get data off the camera module (FPGA)

The first is super hard to figure out, but it's been solved, and there are hints throughout the thread.
The second is an open question. Some people have posted demo's, but I don't think there is public FPGA code out yet?

Best of luck!

EDIT: I have some rough FPGA code available soon:

If you cheat a bit, you can use the gap between the valid lines as the H-Sync, and the gap between frames as the V-Sync.
There are sequential lines with no gaps (about every 7 lines), so you lose those lines (there's no time for an H-sync pulse). This compresses the image vertically, but as the position of those double line jumps around, the image looks pretty stable overall.

But, what you get in return is super low resource requirements:
- No BRAM or SDRAM used at all (we don't need a frame buffer)
- only 58 LUTS and 21 registers used in total for the most basic implementation (as in the image)
- about 300 LUTS using histogram shifting and a colourmap (https://ai.googleblog.com/2019/08/turbo-improved-rainbow-colormap-for.html)
- you can drive this directly from the recovered clock on the DS90C124 (CPLD without a crystal?)
- probably drive a 320*240 RGB really nicely!

As its a PITA (at least for n00bs like me) to get a softcore on an FPGA, and ESP32's are super cheap, I will use that for the unlock, and try and get it to act as a WIFI video streamer :)

1150452-0
« Last Edit: January 26, 2021, 06:49:53 pm by dnhkng »
 
The following users thanked this post: ArsenioDev, Imolteor

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 250
Re: Autoliv NV2 on Raspberry Pi
« Reply #69 on: January 21, 2021, 12:46:01 am »
Nice!

By the way, I have demo code on https://github.com/tmbinc/fpdlink but it uses an OpenVizsla so it's not exactly the most practical setup. It doesn't need the DS90C124, but uses the FPGA LVDS receiver. There has been some other work to bridge this to a RasPi over SPI but getting SPI to work at a high performance is not trivial.
 
The following users thanked this post: dnhkng

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #70 on: January 26, 2021, 02:30:17 pm »
As perfect is the enemy of good, here's some (very imperfect/not very good) minimal HDL code to at least generate an image on an LCD at 30FPS, using the DS90C124 decoder IC:
https://github.com/dnhkng/NVCAM-readout

You need to just wire the DS90C124 up to the cheapest FPGA you can find that has enough pins. Using the deserializer with a tiny FPGA (< 1k LUTs)  is at least hand-solderable stuff, and won't cost more than a few dollars more than an FPGA with SERDES. I used this cheap dev board:
https://www.seeedstudio.com/Sipeed-TANG-PriMER-FPGA-Development-Board-p-2881.html

and used this screen:
https://www.seeedstudio.com/5-Inch-Display-for-Sipeed-Tang-Nanno-p-4301.html

because it's cheap and they fit together.

This particular screen does NOT give a shit about regular timing signals/clock speeds. I have a hard time giving it a signal that doesn't cause something reasonable to appear on the screen. Due to the timing stuff I mentioned in the post above, you get some extra weirdness on the right-hand side of the screen, due to the internal line buffer in the LCD controller hardware.
There are a few demo projects for that FPGA, including a demo for the LCD. Use the IO config file from that project for the LCD IO's :
https://github.com/Lichee-Pi/Tang_FPGA_Examples

The FPGA has 20K LUTs and 64MBit SDRAM etc, so there is lots of room to improve things, like add a frame buffer, or colour-maps, or SPI stuff to raspberry pi, or even add in a RISC-V core for the unlock.

You'll have to perform the camera unlock separately of course.
« Last Edit: January 26, 2021, 03:28:30 pm by dnhkng »
 
The following users thanked this post: ArsenioDev

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #71 on: February 04, 2021, 12:44:25 pm »
I've added some more stuff:



- I now use a framebuffer
- Use multiple colourmaps
- Do either histogram normalisation (not equalisation, to keep the values linear), or show the lowest 8-bits for maximum contrast
- Plot the histogram of the un-normalised data, so once we figure out the calibration, we have a histogram of true temperatures.

This is currently all done on FPGA without a RISC-V core, because I can't figure out how to use one... I should also learn how to write test benches...  And how to structure VHDL...




ToDo:
- figure out the calibration
- add an optional crosshair, and display the centre temperature on the histogram plot
- write an SPI controller, and interface with an ESP32
- write the video streamer on the ESP32 web server

« Last Edit: February 04, 2021, 12:55:43 pm by dnhkng »
 
The following users thanked this post: RO

Offline smason

  • Newbie
  • Posts: 2
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #72 on: February 14, 2021, 07:50:18 am »
Great work so far! I purchased one of these cameras in hope i will be able to buy a kit in the near future to retrofit this into my car!

I will keep an eye on this thread for your progress!
 

Offline twizzter

  • Contributor
  • Posts: 20
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #73 on: September 05, 2021, 10:01:59 am »
Hello from yet-another-scrapyard-nv2-owner!
Got flash dump, tried to perform an initial analysis and i stumbled onto weird cpu_rec output:
Code: [Select]
mint@mint:~/Desktop/nv2$ python3 cpu_rec.py nv2.bin
nv2.BIN
full(0x800000) None                               chunk(0x1b800;55)   6502     
which seems to be invalid, as i haven't expected 6502 code here.
 
binwalk shows:
Code: [Select]
mint@mint:~/Desktop$ binwalk nv2.bin

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Xilinx Virtex/Spartan FPGA bitstream dummy + sync word
54420         0xD494          Intel x86 or x64 microcode, sig 0x04a00004, pf_mask 0x822000, 20C0-01-10, rev 0x-3ffefdf0, size 657408
1984064       0x1E4640        MySQL MISAM index file Version 4
3210374       0x30FC86        MySQL ISAM index file Version 8
5051898       0x4D15FA        MySQL ISAM compressed data file Version 6

Is it possible this dump could be encrypted? Or maybe i forgot to add some command line flags when using cpu_rec?
 

                                                                   


 

Offline Wazowski

  • Newbie
  • Posts: 1
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #74 on: December 07, 2021, 04:55:16 am »
Is there a kit available for someone with zero coding experience. I would love to use this device in my car. It has huge potential for good. Excellent job so far but it has to be dumbed down for the rest of us.
 

Offline railrun

  • Regular Contributor
  • *
  • Posts: 113
Re: Autoliv NV2 on Raspberry Pi
« Reply #75 on: January 02, 2022, 12:54:43 pm »
Hi, I'm still trying to start my camera. I'm using an ELM327 but didn't receive any datas. Any idea what I'm doing wrong?
Code: [Select]
>ATZ
ELM327 v1.5
>ATSPB
OK
>ATPBE005
OK
>ATAL
OK
>ATCM000
OK
>8B02270155555555
NO DATA
 

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #76 on: January 02, 2022, 03:44:20 pm »
ATZ                              // Reset
ATPBE005                    // 100kbps for user protocol B
ATSPB                         // Select protocol B (User1 CAN)
ATSH000657               // Set transmit address to 0x657
ATAL                           // Allow long messages
ATCM000                    // Address mask
8B02270155555555    // Request seed
« Last Edit: January 02, 2022, 03:51:13 pm by oPossum »
 
The following users thanked this post: railrun

Online oPossum

  • Super Contributor
  • ***
  • Posts: 1415
  • Country: us
  • Very dangerous - may attack at any time
Re: Autoliv NV2 on Raspberry Pi
« Reply #77 on: January 02, 2022, 03:51:51 pm »
C++ code for ELM327 attached
 
The following users thanked this post: railrun, newex

Offline railrun

  • Regular Contributor
  • *
  • Posts: 113
Re: Autoliv NV2 on Raspberry Pi
« Reply #78 on: January 02, 2022, 04:01:41 pm »
ATZ                              // Reset
ATPBE005                    // 100kbps for user protocol B
ATSPB                         // Select protocol B (User1 CAN)
ATSH000657               // Set transmit address to 0x657
ATAL                           // Allow long messages
ATCM000                    // Address mask
8B02270155555555    // Request seed
Thank you
I forgot the ID.
I feel a little bit dumb  :palm:
 
The following users thanked this post: Denis_K

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #79 on: April 27, 2022, 02:09:32 pm »
Is anyone interested in building their own cheap fast Thermal Camera?

I have a pretty nice DIY solution finished (check back a page of two), although its nowhere near as nice as the OpenIRV
https://www.eevblog.com/forum/thermal-imaging/openirv-isc0901b0-(autoliv-nv3-flir-e4568)-based-opensource-thermal-camera/

But, it's cheap as it uses a Chinese FPGA module  (<20 euro), a Raspberry Pi Pico with micropython for unlocking, and a handful of components that can be hand soldered (the Decoded chip is a little tricky).

I was thinking about selling a PCB board, as a kit on Tindie/Etsy, together with all the code.

Is anyone interested? If so, let me know if you would prefer:

   a) A 4.5 inch 480x320 LCD, and have a hand-held device that can run on a USB battery pack
   b) A 7 inch 800v480 LCD, and have a hand-held device that can run on a USB battery pack
   b) an adapter from the NVCAM to 480P HDMI for display on any HDMI monitor, or with a USB capture dongle as a webcam

A kit (PCB and the parts in a bag) for maybe 120 euro, and assemble for 150, BYO NVCAM

Any feedback?
« Last Edit: April 27, 2022, 02:32:37 pm by dnhkng »
 
The following users thanked this post: tmbinc, RO, sephalix

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #80 on: April 29, 2022, 10:53:23 pm »
 :For me the cheapest option whith code that can later be used to tailor it to my needs.
 

Offline dnhkng

  • Contributor
  • Posts: 16
  • Country: de
Re: Autoliv NV2 on Raspberry Pi
« Reply #81 on: May 02, 2022, 06:46:31 am »
Here are some images of the 7" display prototype.

The display is in a laser-cut housing, and the PCB lives in a small 3D printed enclosure. It runs on an external USB battery and seems to last for ages (hours!). 

@dmendesf:  Sure, it would come with access to the Repo, with VHDL and the Micropython code for the Raspberry Pi Pico.  The cheapest option would just be the PCB and access to the code, BYO FPGA/RPi Pico and IC's/passives. Is that something you want?
« Last Edit: May 02, 2022, 06:58:05 am by dnhkng »
 

Online ArsenioDev

  • Regular Contributor
  • *
  • Posts: 236
  • Country: us
    • DiscountMissiles: my portfolio and landing page
Re: Autoliv NV2 on Raspberry Pi
« Reply #82 on: May 02, 2022, 01:56:28 pm »
I'll bite; how much you askin for base price?
 

Offline RO

  • Contributor
  • Posts: 45
  • Country: nl
Re: Autoliv NV2 on Raspberry Pi
« Reply #83 on: May 06, 2022, 07:18:11 pm »
I too would be very interested!

I actually bought the tang primer and display a while ago, kind of hopeful I could copy your displayed progress in this thread.
(Monkey see monkey do ;)

But i can't really wrap my head around the camera unlock part.
Any help would be much apreciated
 

Offline newex

  • Contributor
  • Posts: 23
  • Country: 00
Re: Autoliv NV2 on Raspberry Pi
« Reply #84 on: May 12, 2022, 12:28:19 pm »
I would also order psb. Interested adapter to HDMI.
 

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #85 on: May 13, 2022, 08:35:10 pm »
Yes, i'm interested. Will send a PM.
 

Offline LurkerN0mor3

  • Newbie
  • Posts: 2
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #86 on: October 30, 2022, 12:38:07 am »
Hello,

   I hate to be that guy whos first post is asking for something but  "hey atleast I registered finally" lol.
would it be possible to get a flash dump from the canbus pins on the nv2 or should be looking for the data  else where on the board? if none of that is possible would it be at all possible for someone to provide me with the file dump?  I finally was able to buy my first FPGA and alot of the neccessary components  (thanks to all the information in this thread and others). Unfortunately, My hot air gun is broken ( to be honest i dont trust myself enough to remove the chip anyway).
thank you

 

Offline sztelf

  • Newbie
  • Posts: 3
  • Country: pl
Re: Autoliv NV2 on Raspberry Pi
« Reply #87 on: December 29, 2022, 11:11:06 pm »
Hi. I brought nv2 camera from audi some time ago, but "NV2 on Raspberry Pi" project is still too hard for me to implement based on info on forums. Could you share more information for beginners like me to run this camera on rpi? I can solder, disassemble camera, know python (mid), and have some exp with electronics, but as I wrote before it's too complicated for me.
 

Offline LurkerN0mor3

  • Newbie
  • Posts: 2
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #88 on: January 22, 2023, 05:57:12 am »
hey
 you need to get an fpga, a canbus reader etc

while there are many cheap ones i honestly would get which ever one is being used by tmbic ( i think ) this way you can use the github stuff.
its easier to implement ( in a way ). i am in the same boat as you as far as being a beginner.  there are many many clues on this post. however, with out knowing what you are reading you wont pick up on it.
for example the unlock code is on here but it wasnt until many many rabbit holes that i figured that out.

compared to the NV1 it doesnt seem like you run the risk of locking the camera down from to many failed attempts.
i doubt this helps but i hope it somewhat does as a newbie myself it sucks not knowing where to even begin.

 

Offline turbbn

  • Newbie
  • Posts: 1
  • Country: cn
Re: Autoliv NV2 on Raspberry Pi
« Reply #89 on: March 30, 2023, 05:54:52 am »
Hello, I'm also trying to hack this camera. I wrote a program, but it's not working well. Can any experts provide me with some seep and key combinations so that I can adjust the program?
 

Offline sztelf

  • Newbie
  • Posts: 3
  • Country: pl
Re: Autoliv NV2 on Raspberry Pi
« Reply #90 on: April 03, 2024, 07:08:28 pm »
I think this project is slowly losing its point a little bit because there are already very cool thermal imaging cameras on usb-c available for around $250 that work without any work.... nevertheless if someone would like to sell the howto I would be happy to buy, because the camera from the audi is lying aimless to me....
 

Offline dmendesf

  • Frequent Contributor
  • **
  • Posts: 320
  • Country: br
Re: Autoliv NV2 on Raspberry Pi
« Reply #91 on: April 05, 2024, 06:45:27 pm »
Put me on that list. I´m also paying for something that works.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf