Author Topic: Multiple LVDS channels acquisition with PIC32 from CMOS sensor  (Read 1865 times)

0 Members and 1 Guest are viewing this topic.

Offline s1m0n3tTopic starter

  • Newbie
  • Posts: 8
  • Country: it
Hi, I'm trying to acquire data from a CMOS image sensor (NOIP1SN1300A) with a 32bit microcontroller (PIC32MZ2048EFM100).

I'm studying the datasheet of this sensor and for this version (P1-SN/FN) there are 3 options for output data stream: 1, 2 or 4 LVDS channels in multiplexing, in addition to sync and clock channels (page 39 of datasheet).

The output data rate is 720Mbps for 10bit acquisition and 576Mbps for 8bit (page 35).
If I undestand correctly, with 4 channels multiplexing option and 8bit image depth, each LVDS channel will have an output datarate of 576/4=144Mbps.
I'm trying to understand if I can handle this stream with a PIC32 (with a maximum clock of 252MHz).

I think that I don't have any possibility to read 6 144Mbps serial channels at the same time with a "simple" PIC32 with 252MHz clock. Probably the best way to manage this CMOS sensor is an FPGA, but if it's possible, I would prefer to avoid it. In fact, I don't need a high framerate: also a single frame per second can be sufficient. I should be able to understand if I can configure the sensor to stream all the data in a single LVDS channel (plus sync and clock) with a low data rate, manageable also by the PIC. I checked all the configuration bytes of the sensor (configurable by SPI input), but I don't found any configuration output data rate.

The CMOS sensor accepts a clock signal as input, with 2 options:
1) LVDS input clock with 360MHz frequency for 10bit mode or 288MHz for 8bit mode
2) TTL input clock with 72MHz (with internal PLL that multiply the frequency by 4 or 5)
On page 11 a “typical input clock frequency” is indicated, but is not clear if is possible to supply a lower clock, with a proportional reduction of output data rate.
On page 5, in “Electrical Interface − P1 − SN/SE/FN LVDS” table, the values of PLL input clock (72MHz) and LVDS input clock (360MHz) are indicated only as maximum value. No typical or minimum value are indicated. This would let me think that maybe it is possible to decrease the input clock as desired, proportionally modulating the output data rate.

If it's possible, maybe I can manage the output channel with an appropriate input port, through an LVDS interface (like MAX9122 receiver). I should verify also if the stream protocol is compatible with UART or SPI standard, instead I have to receive the stream in a generic i/o port and manage the protocol manually.

Do you have any ideas, different point of views or advices?

You can find the datasheet of the sensor here:

https://www.onsemi.com/pub/Collateral/NOIP1SN1300A-D.PDF

Thank you in advance for any suggestions!
 

Online mikeselectricstuff

  • Super Contributor
  • ***
  • Posts: 13695
  • Country: gb
    • Mike's Electric Stuff
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #1 on: August 03, 2019, 07:16:49 pm »
No you won't be able to do it with a PIC32 unless you can slow the clock down  - you'll need an FPGA.

If you can live with skipping pixels ( e.g. every other pixel) there are some tricks you can do with counters and DMA to get fairly high rates. If doing this you will need to be clocking the sensor and PIC from the same source.
I did a video about doing this with alow-res sensor on a lower - end PIC, the ideas should be scalable to the PIC32MZ to some extent
https://youtu.be/rQYByorpoFk
Youtube channel:Taking wierd stuff apart. Very apart.
Mike's Electric Stuff: High voltage, vintage electronics etc.
Day Job: Mostly LEDs
 
The following users thanked this post: s1m0n3t

Offline diegogmx

  • Contributor
  • Posts: 19
  • Country: dk
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #2 on: August 04, 2019, 12:25:05 am »
No you won't be able to do it with a PIC32 unless you can slow the clock down  - you'll need an FPGA.

If you can live with skipping pixels ( e.g. every other pixel) there are some tricks you can do with counters and DMA to get fairly high rates. If doing this you will need to be clocking the sensor and PIC from the same source.
I did a video about doing this with alow-res sensor on a lower - end PIC, the ideas should be scalable to the PIC32MZ to some extent
https://youtu.be/rQYByorpoFk

as mike says you should use an fpga, you can deserialize it and get it out parallel, or you could get instead a NOIP2SN1300A−QDI which has a cmos parallel output working at a 75mbps rate which you could catch with a parallel interface
 
The following users thanked this post: s1m0n3t

Online nctnico

  • Super Contributor
  • ***
  • Posts: 26755
  • Country: nl
    • NCT Developments
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #3 on: August 04, 2019, 01:07:10 am »
I'd use the sensor with a parallel output and somehow use DMA to get the data into the chip. You can probably program the framerate of the sensor to a value which is low enough so the PIC32 can keep up. One thing to watch out for at low framerate is rolling shutter versus taking a snapshot (global shutter). With a rolling shutter the image is captured as it is scanned. This may cause weird distortions at low framerates when the object you are capturing is moving.
« Last Edit: August 04, 2019, 01:09:03 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 
The following users thanked this post: s1m0n3t

Offline s1m0n3tTopic starter

  • Newbie
  • Posts: 8
  • Country: it
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #4 on: August 04, 2019, 10:14:50 pm »
If you can live with skipping pixels ( e.g. every other pixel) there are some tricks you can do with counters and DMA to get fairly high rates. If doing this you will need to be clocking the sensor and PIC from the same source.
I did a video about doing this with alow-res sensor on a lower - end PIC, the ideas should be scalable to the PIC32MZ to some extent
https://youtu.be/rQYByorpoFk

Very interesting solution, Mike, thank you! Probably it would be better a lower framerate than a sub-resolution.

The application of this sensor will be a small star tracker for cubesats, for an EE degree thesis. The sensor should be able to track very slow rotational movements (change of position of few stars in the frame), so also a single frame per second will be good. But I prefer to keep the full resolution, to keep a good sensibility of the instrument.

as mike says you should use an fpga, you can deserialize it and get it out parallel, or you could get instead a NOIP2SN1300A−QDI which has a cmos parallel output working at a 75mbps rate which you could catch with a parallel interface

In fact, using the parallel interface version (P2-SN) of the sensor could be a better solution for my application. You would know how to determine the maximum speed at which a PIC can read a parallel port (8 or 10 bit), in DMA mode?

I'd use the sensor with a parallel output and somehow use DMA to get the data into the chip. You can probably program the framerate of the sensor to a value which is low enough so the PIC32 can keep up. One thing to watch out for at low framerate is rolling shutter versus taking a snapshot (global shutter). With a rolling shutter the image is captured as it is scanned. This may cause weird distortions at low framerates when the object you are capturing is moving.

If I understand correctly, with registers 199-203 I can adjust the framerate (page 59: register map), but is not really clear for me how to configure these register. For example:
Mult Timer [15:0] - Defines granularity (unit = 1/PLL clock) of exposure and reset_length
If PLL clock is 72MHz, the value of unit should be 1/72MHz, so what value of these 16 bit?

Moreover, assuming to drastically reduce the framerate (for example to 1fps), the output datarate will be proportionately slowed down, or will it remain identical, leaving a long pause between the transmission of a frame and the next one? The datasheet does not seem to detail these conditions much, or is it my impression?
 

Offline diegogmx

  • Contributor
  • Posts: 19
  • Country: dk
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #5 on: August 05, 2019, 02:02:18 am »
according to page 666 and 667 you can sample as fast as the peripheral clock, and that is as fast as the peripheral clock which is 255mhz, i assume the dma copies in 1 cycle to ram but i could be wrong, take a better look at the datasheet
although i find the datasheet for this particular device a bit cryptic and not particularly detailed, just shows all the control registers without much explanation
 
The following users thanked this post: s1m0n3t

Offline nigelwright7557

  • Frequent Contributor
  • **
  • Posts: 689
  • Country: gb
    • Electronic controls
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #6 on: August 10, 2019, 01:47:33 pm »
according to page 666 and 667 you can sample as fast as the peripheral clock, and that is as fast as the peripheral clock which is 255mhz, i assume the dma copies in 1 cycle to ram but i could be wrong, take a better look at the datasheet
although i find the datasheet for this particular device a bit cryptic and not particularly detailed, just shows all the control registers without much explanation

I used DMA on same PIC and all it could manage was 12.5MHz * 16 bits.
In the end I used OC1 to produce clock pulses for the device and just read in the data from an I/o port and managed about 18MHz.
 
The following users thanked this post: s1m0n3t

Offline s1m0n3tTopic starter

  • Newbie
  • Posts: 8
  • Country: it
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #7 on: August 10, 2019, 02:50:46 pm »
Thank you nigelwright7557 for your precious experience! If the ONsemi support does not respond to me (on the possibility to lower the sensor clock), I will probably fall back on a simpler, lower resolution sensor like the MT9V034 (0.3MP), with 27MHz 8-10bit parallel output.
But...you really managed to get only 12.5MHz acquisition frequency with a 252MHz clock? It would mean 20 clock cycles to acquire a port and save it to ram.
I hope you used the PIC with a lower clock, otherwise this is really bad news for me: it would mean that I wouldn't be able to manage even the 0.3MP sensor with a 27MHz clock!  |O
 

Offline asmi

  • Super Contributor
  • ***
  • Posts: 2728
  • Country: ca
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #8 on: August 12, 2019, 08:30:26 pm »
The real question is what are you going to do with all that data even if you somehow manage to get it into the chip.

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3137
  • Country: ca
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #9 on: August 13, 2019, 01:49:42 pm »
I hope you used the PIC with a lower clock, otherwise this is really bad news for me: it would mean that I wouldn't be able to manage even the 0.3MP sensor with a 27MHz clock!  |O

The CPU clock is of less concern to you, as the IO ports are driven by one of the peripheral clocks (PB4), which can go up to 200 MHz. If you run your SYSCLK at 252 MHz, then PB4 can only go up to SYSCLK/2 = 126 MHz, because SYSCLK/1 = 252 MHz is out of specs and there's no way to configure it to anything in between.  If you want PB4 at 200 MHz, you have to run SYSCLK at 200 MHz too.

People reported they can do up to 50 MHz IO with DMA, but I think this was in master mode (i.e. PIC generating the clock).

If you want to track stars, I would take a small FPGA which would scan high resolution sensor (as hiigh resolution as I could get), then cut out the small pieces of the picture around your stars and send it through for processing. You could do processing on a CPU with low-bandwidth communication channel between FPGA and CPU. Alternatively, you can process within FPGA. Many people like to build softcore CPUs from FPGA fabric - this way you get a free CPU within FPGA.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: Multiple LVDS channels acquisition with PIC32 from CMOS sensor
« Reply #10 on: August 13, 2019, 03:16:19 pm »
Yeah, an FPGA or some kind of ARM SoC.
Could even be a job for a MIPI CSI camera plugged to a Raspberry Pi.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf