Author Topic: Autoliv NV2 on Raspberry Pi  (Read 5204 times)

0 Members and 1 Guest are viewing this topic.

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 53
Re: Autoliv NV2 on Raspberry Pi
« Reply #25 on: June 17, 2019, 02:24:41 pm »
Got a bit more progress, I'm really starting to fight signal loss, I'm going to have to rework the connection between the deserializer and the camera, I'm not sure what makes the difference between a good capture and a bad one, without touching the camera I can get a good 30 second capture and then instantly running a second one get maybe one or two good frames out of the entire thing.

 

Offline LesioQ

  • Regular Contributor
  • *
  • Posts: 52
  • Country: pl
  • Every king should be naked.
Re: Autoliv NV2 on Raspberry Pi
« Reply #26 on: June 18, 2019, 02:53:04 pm »
Is this video without any shutter operation ? Not bad if without calibrations.
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 53
Re: Autoliv NV2 on Raspberry Pi
« Reply #27 on: June 18, 2019, 03:44:47 pm »
The shutter isn't really used much here, but you can see it actuate in the first capture in the video, there's some "blank" frames but it you look at it frame by frame you can see the shutter sweep in and out. If you look at a Flir E4 or Seek as a "photometric" camera this is a camcorder. This doesn't supply metered values or at least it's not intended to. It may be possible to "calibrate" it's output and tell what temp something it but it's intended to only show differences in heat, it's only used as a people/animal/car detector at night to show things out of range of your headlights. The shutter does actuate on this but I don't know if it's the sensor requesting it for internal stuff, or if it's the FPGA doing it and maybe the FPGA is doing some crude internal auto exposure or maybe just sanity checks. The shutter can be controlled via can bus commands per Tmbinc's documentation but I haven't figured out anything other than the UDS challenge.
 

Offline tmbinc

  • Regular Contributor
  • *
  • Posts: 201
Re: Autoliv NV2 on Raspberry Pi
« Reply #28 on: June 18, 2019, 06:49:01 pm »
For shutter operation, you need to send on can_id 0x401 the following data:

"c8 64 18 00 04 00 00 00"
"c8 64 18 00 00 00 00 00"

Byte 7 defines what's shown during the shutter operation (black, uncorrected, test pattern).
 

Offline Treehouseman

  • Supporter
  • ****
  • Posts: 53
Re: Autoliv NV2 on Raspberry Pi
« Reply #29 on: June 18, 2019, 06:57:13 pm »
Do you have a list of what commands you saw in the microblaze code? Or at least where the block of code is that was processing the messages? I never actually found that, I got the key from simply finding the 16 bytes in an out of place location and seeing that there was a ref to it. I was just doing trial and error with high entrophy 16 byte blocks.
 

Offline tmbinc

  • Regular Contributor
  • *
  • Posts: 201
Re: Autoliv NV2 on Raspberry Pi
« Reply #30 on: June 18, 2019, 08:30:35 pm »
The key is in the boot block (initialized BRAM from the bitstream); the CAN handlers are in the firmware.

This is what I have there: (All names from me of course)

ROM:2200E970 00 00 04 01     CanFuncTable:   .int 0x401              ; DATA XREF: _InitCan:loc_22001B6C↑o
ROM:2200E974 22 00 1A 10                     .int CanCamFuncControl
ROM:2200E978 00 00 03 04                     .int 0x304
ROM:2200E97C 22 00 19 20                     .int CanCamSync
ROM:2200E980 80 00 03 13     CanFuncTable2a: .int 0x80000313         ; DATA XREF: _InitCan+6C↑o
ROM:2200E984 22 00 04 88                     .int CanCamStatus
ROM:2200E988 00 00 03 11                     .int 0x311
ROM:2200E98C 22 00 05 FC                     .int CanCamSerialNumber
ROM:2200E990 00 00 03 12                     .int 0x312
ROM:2200E994 22 00 05 70                     .int CanCamInitRequest
ROM:2200E998 80 00 03 03     CanFuncTable2:  .int 0x80000303         ; DATA XREF: _InitCan+78↑o
ROM:2200E99C 22 00 04 88                     .int CanCamStatus
ROM:2200E9A0 00 00 03 01                     .int 0x301
ROM:2200E9A4 22 00 05 FC                     .int CanCamSerialNumber
ROM:2200E9A8 00 00 03 02                     .int 0x302
ROM:2200E9AC 22 00 05 70                     .int CanCamInitRequest


So, 0x401 is handled in CanCamFuncControl (22001A10).
 

Offline oddbondboris

  • Contributor
  • Posts: 29
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #31 on: June 19, 2019, 01:26:58 pm »
Got a bit more progress, I'm really starting to fight signal loss, I'm going to have to rework the connection between the deserializer and the camera, I'm not sure what makes the difference between a good capture and a bad one, without touching the camera I can get a good 30 second capture and then instantly running a second one get maybe one or two good frames out of the entire thing.


if you have a cheap fx2 based logic analyzer (or any streaming output logic analyzer) you might wanna hook it up to the deserializer output and see if what the ftdi chip is spitting out matches.
 

Offline Musclor

  • Contributor
  • Posts: 16
  • Country: es
Re: Autoliv NV2 on Raspberry Pi
« Reply #32 on: July 15, 2019, 12:59:17 am »
This is pure genius and if related to the old thread where someone had asked if such thing was even possible, even better. Absolutely inspiring! Bravo!
 

Offline oPossum

  • Super Contributor
  • ***
  • Posts: 1007
  • Country: us
  • Save the possum. Save the world.
Re: Autoliv NV2 on Raspberry Pi
« Reply #33 on: May 13, 2020, 01:47:16 am »
Using dsPIC33EV256GM102
<click> <click>
Thanks to tmbinc, Treehouseman, Lennie, and Johan for their help with this.

 

Offline oPossum

  • Super Contributor
  • ***
  • Posts: 1007
  • Country: us
  • Save the possum. Save the world.
Re: Autoliv NV2 on Raspberry Pi
« Reply #34 on: May 17, 2020, 10:28:30 am »
Using the ubiquitous ELM327 cable

 

Offline conmega

  • Contributor
  • Posts: 16
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #35 on: May 20, 2020, 10:18:31 pm »


Well this is the best I have gotten out of it so far.
Still some noise issues due to me just hooking this straight to the FPGA using its internal termination.

Thanks to oPossum and Treehouseman for starting me down this rabbit hole.
Also thanks to tmbinc. I took the Migen implemented de-serializer and  decoder and was able to get it ported over to 7-Series parts (mostly changing the SERDES units over to the new ones and dealing with different clocking resources).
I took just the fpdrecv.py and basically built it out as top then implemented some clock domain crossing/fifo and LCD controlling in VHDL and glued the two together.
Its been a heck a week working on this.
« Last Edit: May 20, 2020, 10:23:57 pm by conmega »
 

Offline ArsenioDev

  • Regular Contributor
  • *
  • Posts: 53
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #36 on: May 20, 2020, 10:27:56 pm »
Conmega, that is one IMPRESSIVE quick build you have there. Only wish I could poke around in it, been thinking of how to port it to the ECP5 boards I have, should be enough LUTs to drive a smaller display and do the CANBUS on board.
 

Offline conmega

  • Contributor
  • Posts: 16
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #37 on: May 21, 2020, 12:04:25 am »
Thanks, but most of the heavy lifting was done by others, especially the decoder/deserializing written by tmbinc.
I simply had to learn a bit of Migen to get it move from a Spartan-6 target to 7-Series parts.

My design as it currently sits uses around 700~ LUTs, 1100~ FlipFlops, and two half BRAM units, which are 36x1024 and can be used as 18x1024 "half" BRAMs.
For reference this is only about 2%~ of the entire Artix-7 100T... So I bet if you had the bandwidth or implemented an even lower sample rate over-sampler you could probably get away with running this on something much smaller and simpler like an ICE40, I will probably plan to just target a small Artix-7 or Spartan-7 myself.

The big thing is to use tmbinc's decoder/deserializer directly without modification. You need at-least a 1080Mbps capable SERDES. Looking at Artix-7 and Spartan-7 it looks like speed grade 1 always is 950Mbps capable and speed grade 2/3 is 1250Mbps.

Of-course this is assuming you don't want to modify tmbinc's sampler code which over-samples by 7.7x as described on his website.
You could probably modify it to work with 6x oversampling (840~Mbps) to allow it to work on more common/cheaper/smaller parts.
But its easiest to just use a speed grade two 7-series part. I had to change the clocking.py to implement 7-series primitives for the MMCM and clock buffers. And the Serdes change was a bit more complex but not really all that bad. To get 1080Mbps you use the ISERDES2 in "OVERSAMPLE" mode, which has a 4 bit output but allows for QDR sampling. So you feed a 2 times data rate clock, in this case to match tmbinc's code which expects a 7.7x sampling it would be a (135Mhz*2) 270Mhz clock then you get 4 bits every 270Mhz which you put into an 8-bit register every 135Mhz and you get your 8-bit 7.7x sample data to feed into the sample.

So its not that bad, honestly it took me a few days to learn enough Migen to get stuff built and then a few days to understand the timing coming out of the sampler and to just write sane VHDL... I am still quite new to FPGAs so there was a good 4-5 days of buggy glitchy video on the screen :)

I will probably try and get his sampler down to only a 6x oversample rate so I can use a common and cheap FPGA board (like the Cmod S7) with a speed grade 1 part on it for implementing this into a hand-held device.
 

Offline ArsenioDev

  • Regular Contributor
  • *
  • Posts: 53
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #38 on: May 21, 2020, 01:08:53 am »
The good news is even the basic ECP5 Serdes are 3.2Gb/s! Hmm, guess it's time to nose to the grindstone on the Migen and Litex stuff eh? Honestly I'm trying to get this all running on FOSS flows, gonna do the designs for a custom breakout of the FPDlink. Prob gonna make a few and give em to friends then open source the design.
 

Offline conmega

  • Contributor
  • Posts: 16
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #39 on: May 22, 2020, 10:27:38 pm »


And now glitch free video with hardware histogram equalization!

This uses the current frame to build a histogram in BRAM1 then between frames using downtime to built up an equalized histogram in BRAM2 (in this case equalized to 6-bits for the 6-bits of green available).
Then using the new equalized histogram in BRAM2 we apply that to the next frame's pixel data on the fly while building up a new histogram for the next frame in BRAM1 again. This is while not 100% accurate from frame to frame its close enough and extremely low latency. It is also space efficient requiring about 1/3rd the amount of BRAM to store both histograms as it would take to store one entire frame. Not to mention the logic to say offload this to a microprocessor for processing via software :)

I also adjusted the focal point of the camera by unscrewing the lens outward a few turns as Treehouseman found will bring the focal distance in.
I was about 2ft~ away from the camera and I had gotten it to about 18~in or so before!
 

Offline conmega

  • Contributor
  • Posts: 16
  • Country: us
Re: Autoliv NV2 on Raspberry Pi
« Reply #40 on: May 23, 2020, 04:06:25 pm »


And now with palette support!
Thanks to oPossum for generating a few palettes for me to test out.

Also decided to show off the histogram equalization a bit with the Metcal iron :)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf