-
Hello, RO! Hi everyone!
How is the production going for you VGN?
I have been pretty overworked last tree weeks. But no worry, I'm continuing.
1. I had some trobles with scaling of the printed parts. The SLA printer repeatability is quite high, but the scale factor way wary much, depending on the size and geometry of the part. The best solution was to print same type parts simultaneously and adjusting the XY scale. I could achieve +/- 0.05mm accuracy. All plastic parts were printed. I will make photos by the end of this week.
2. I have ordered additional sets of PCBs, as 10 PCBs (-2 for me), that I have now is not enough. The should arrive next tuesday.
Also quite a lot of people asked me to add a small adapter board for programmer, it was ordered too and will arrive next week.
3. SPI NOR flash ICs should arrive this week. I already have and adapter to preprogram the initial bootloader with selftest firmware.
4. Silicone buttons are also in production, hope to finish them by next week.
P.S. I don't know why, but these worms look like bacteriophages...
-
Hi,VGN
When I use 0x25 as bias, I get the this image. Is my FPA damaged?
-
Depends what you mean by “damaged”. Your microbolomter has 4 dead rows and a lot of column noise. The dead rows are hardware faults, the column and row noise is dealt with in software. 4 dead rows would be unacceptable on a lab grade camera but dead pixel correction could cope with it in less demanding roles. As a rule, dead pixels are common on microbolometers but whole dead columns or rows are not. Such would often result in a microbolometer QA test failure. That microbolometer may have been acceptable for its original application in a vehicle however.
Fraser -
Hello everyone. At the outset, I would like to say how amazed I am when I look at the work of VGN. The idea to use the system from a car to make a thermal imaging camera is already impressive, but I did not even think about such image quality and refreshing time. If I found such a product in a store, it would not have occurred to me that it was all DYI. It's nice to see people so involved in their project. Some time ago I became the unfortunate owner of FLIR One Pro which turned out to be damaged (I bought it on ebay, the money was recovered and there was no contact with the seller). I had a plan to use a LEPTON sensor with a simple adapter, but a friend told me about this project. There shouldn't be a bigger problem with the camera itself (we have a lot of car parts), but what about the rest?
-
I came across this interesting article about the ISC1406L sensor ("micron" teardown).
FLIR Boson IR Camera Core
>> PDF
-
Hi, I've started looking at a Tau2 640x512 sensor in hope of interfacing with this project. The sensor interface sounds like it might be identical to the ISC0901. I've attached an image what I think the sensor is doing. I guess the last two signals are pixel valid/enable like in the HDL? I can't see a CMD signal however.
VGN, could the 3 lines of pipeline garbage you note in ISC0901_capture.v be values that reflect/help with the row/column noise? Could they be Unexposed pixels? My camera reliably has empty, data, empty for the three lines and I feel like there's a chance it's not garbage.
I've linked a sigrok trace of a (empty) frame https://drive.google.com/file/d/1uf3K03b041-RHY4dU8iX2qOJ5yBbgxgw/view?usp=sharing in case someone wants to look. It shows the bias -> pixel delay and 3 lines of garbage. Frame period is 1/30sec (despite being a 9Hz camera)
-
Greg! Glad to see you in thread finally.
Yeah, most decent FPAs will not be 9Hz, rather are just FPGA gateware locks, not on the imager side. -
Hi,VGN
Hello!
When I use 0x25 as bias, I get the this image. Is my FPA damaged?
No, I don't think that your FPA is damaged, I saw something like that when I was bringing up my FPA cores.
As Fraser said, looks like there are 4 dead rows, but probably not, because probably these 4 rows require some special bias value to start working.
I call pixel dead only when it outputs only 0 or 1 regardless to any bias value in the whole range 0 - 63.
Note that real bias value for ISC0901 is 6 bit:
bias[6] - no effect
bias[5:0] - actual bias range 0-63
Also there are some fields in command, that should be set proreply. BTW, I found a way to extract original bias and command pattern values from the original firmware, just full dump of the original SPI flash is needed. I think this is the only solution for the beginning, to make the sensor work. Later I'm sure it is possible to find an algorithm that will brutforce the 64 bias values along with some fields in command to find get the best image output. This procedure can be done at the step of the "two point" calibration, when each pixel gain factor is calculated. I will add a SPI flash dumping feature in bootloader image, that will help to dump the original flash image using OpenIRV hardware. Details a bit later.
-
There shouldn't be a bigger problem with the camera itself (we have a lot of car parts), but what about the rest?
Hi, thanks a lot. If you are asking about OpenIRV hardware parts - yep, this a great trouble, almost no any FPGAs or some other parts avaliable in stock all over the world, due to global chip shortage. We can do nothing with it, just wait when this hell finishes or we all find a way to deal with it. The initial plan was to ship fully assembled boards, but even people from USA cannot buy part, so I decided to start shipping soon only bare PCBs for those, who was lucky to get parts and have enough skills to assemble the PCBs.I came across this interesting article about the ISC1406L sensor ("micron" teardown).
You can easily find articles about other sensors, including ISC0901: https://www.systemplus.fr/wp-content/uploads/2017/06/SP17330_Autoliv_Night_Vision_FLIR_microbolometer_Sample_System_Plus_Consulting.pdf
Unfortunatelly, some extemely interesting images are blured. For the pixel shifting part of this project I would like to see the SEM photos of the pixels, especially the pixel area size, the gaps between pixels. Hope that 18um piezo actuator displacement will be enough to make a full cycle shift within a single pixel in both X and Y directions.
-
Hi, I've started looking at a Tau2 640x512 sensor in hope of interfacing with this project. The sensor interface sounds like it might be identical to the ISC0901. I've attached an image what I think the sensor is doing. I guess the last two signals are pixel valid/enable like in the HDL? I can't see a CMD signal however.
Hi, Gleg!
I think the last two pins at your image are enable and cmd, there is no pixel valid strobe from sensor. Try to probe both residue lines again, there should be a command line. The waveforms from your dump really looks identical to ISC0901. Please, checkout attached .vcd dump. You can use gtkwave software to open the .vcd file.VGN, could the 3 lines of pipeline garbage you note in ISC0901_capture.v be values that reflect/help with the row/column noise? Could they be Unexposed pixels? My camera reliably has empty, data, empty for the three lines and I feel like there's a chance it's not garbage.
In fact I still have no idea what these 3 lines contain. I couldn't find any correlation of the content of the first 3 lines and the actual scene.
On the other hand, there could be a garbage in registers after power up, but no idea why it continues outputing this garbage for all other frames.
-
What beautiful work and a well done project. Gave me an aesthetic pleasure looking at it!!!
Too bad I saw the project late... I had already ordered the HT-301
( I already regretted a lot
-
Thank you for your reminder. I found 2 sets of 336*256*2 bytes of data in the original SPI flash file. The data is similar to 0xA0xx, 0x9Fxx. I used these data to generate this image. Are they bias data?
-
Except for the 4 "dead rows", the main problem now is that it can only be used to detect objects with relatively high temperature, and the image at about 40 degrees Celsius will be submerged in noise.
-
Greg! Glad to see you in thread finally.
Thanks for the welcome, I was hoping to show up right before xmas and get a present!I think the last two pins at your image are enable and cmd, there is no pixel valid strobe from sensor.
You're right thanks, I can see a command once per frame on Pin 10 and 11 is enable.
I'm not sure if the nv3 lens needs it, but I've made a prototype (tinfoil + tape) of a calibration chessboard. The different emissivities look like they might give good enough contrast to get a calibration from opencv (https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html). The test image shows a warm reflection in the aluminium foil but I think you could connect a CC supply and reverse the contrast when the tape was hot. I guess you could expose it onto a PCB and put a heater trace on the back too? Then we'd be able undistort images on the FPGA.
-
Thank you for your reminder. I found 2 sets of 336*256*2 bytes of data in the original SPI flash file. The data is similar to 0xA0xx, 0x9Fxx. I used these data to generate this image. Are they bias data? (Attachment Link)
Unfortunately no, these are gain values. These are in fact almost useless tables, as I already have an algorithm that can recalculate each pixel gain using two uniform images with a temperature difference about 30-40 degree, that is called two point calibration. This is quite easy to do. The quality of thermal images, that you saw in this thread are based on this calibration. Also bad pixels are detected and marked for pixel replacement pipeline.Except for the 4 "dead rows", the main problem now is that it can only be used to detect objects with relatively high temperature, and the image at about 40 degrees Celsius will be submerged in noise.
This is ok, as soon as you have proper command pattern, bias values, you will be able to recalculate the pixel gain. Finally after offset correction, that is periodically done by internal shutter, your camera image quality will meet my level.
Some updated.
Some boards arrived. These are the PCBs, that actually will be shipped. There is no routing or any other changes at all, this is the same v1.1.0, but with gold finish.
I decided to replace immersion silver finish with immersion gold, as gold has longer shelf time and there is no tarnishing issue.
Waiting for Main and Peripheral PCBs that should arrive in a 2-3 days + small debugging adapter board.
-
I'm intending to make a flex PCB to adapt the Tau2 through-hole pinout in my previous pictures to the Molex 52991-0408 to match the ISC0901B0 pinout.
VGN, do you think it would be reasonable to have a common bitstream between the two FPAs if I use an ID resistor on the adaptor? Maybe a sensor ID in general might avoid frying an FPA?
BTW, I think X1 and X2 are mixed up between the PCB and documentation. -
Hi, here's a picture of what I'm thinking of making. It's my second go at a PCB, anyone have suggestions? It's ~26mmx30mm. It'll fold down 180 and present the same interface as the ISC0901B0.
I'm assuming the OpenIRV M board is the same size as the autoliv board so I don't think it will fit in the Tau case.
-
I'm intending to make a flex PCB to adapt the Tau2 through-hole pinout in my previous pictures to the Molex 52991-0408 to match the ISC0901B0 pinout.
VGN, do you think it would be reasonable to have a common bitstream between the two FPAs if I use an ID resistor on the adaptor? Maybe a sensor ID in general might avoid frying an FPA?
I haven't decided yet if it is reasonable to keep a common bitstream. Probably not, as this going to be a waste of logic. Also some FPAs will require another level of bias, core or vccio voltage, that can be tuned by feedback resistor only. If some FPAs are very common, I think that common bitstream could be used.BTW, I think X1 and X2 are mixed up between the PCB and documentation.
This is because I haven't updated schematics yet. In the repo now is v1.0.0, while new boards are v1.1.0. I will upload new schematics at this week.
BTW, main and peripheral boards finally arrived.
-
Hi, here's a picture of what I'm thinking of making. It's my second go at a PCB, anyone have suggestions? It's ~26mmx30mm. It'll fold down 180 and present the same interface as the ISC0901B0.
Good job!
1. I think you could make connector's (2.54 pitch) via annular ring size much smaller to increase clearance, that will decrease the PCB cost.
2. Again, still not sure about id resistors. This is a matter of discussion. How about i2c or even better, 1-wire smallest and cheapest eeprom to keep the ID?I'm assuming the OpenIRV M board is the same size as the autoliv board so I don't think it will fit in the Tau case.
Right, I think a 3d printed adapter can solve this problem. -
Right, attempt two! this one should allow an I2C or UNI/O EEPROM. I left the I2C option in because they go up to 2MBit. I think with some compression you could store something interesting like the bias values with the sensor. With UNI/O it will only use one IO.
I think we can only rely on VCC_SYS being present in the bootloader and I've assumed the IO voltage will start off at 2.5v. Hopefully SENSOR_IO_0 could sense the pull-up to stay compatible with an FPA that needs every pin(?). VGN, would you be interested in using this in the bootloader to ID other sensors?
Switching IO voltage after the bootloader could be a problem though. What if you passed VCCIO_FPGA through? These EEPROMs can do 1.8v to 5.5v, I could get rid of the regulator and they'd follow the IO voltage.
-
Hello, I have finally uploaded schematics v1.1.0 to the repo: https://github.com/OVGN/OpenIRV/tree/master/docs/schematics
Right, attempt two! this one should allow an I2C or UNI/O EEPROM. I left the I2C option in because they go up to 2MBit. I think with some compression you could store something interesting like the bias values with the sensor. With UNI/O it will only use one IO.
ISC0901B0 requires 84KB for a single bias frame. As I know, original NV3 has 4 table to cover wide camera temperature working range, i.e. we should store 84x4 = 336KB of bias values, i.e. we need 4mbit eeprom. But as I know, you can hardly find 1-Wire, UNI/O or even I2C eeprom of such capacity. For higher resolution sensors, we probably will require even more. Also loading bias table from I2C eeprom will be slower than from QSPIx4@65MHZ, that impacts on statup time.
On the other hand, I think that some kind of attached board identification is needed. That's why I want to suggest having 1-wire smallest eeprom, that will just keep some identification data. In fact, we already have a pull-up resistor R6 at the pin 19 of the X2 connector of M-board. This resistor is planned to be used to identify that original ISC0901B0 sensor board was attached (as there is GND on this pin). We also can use this pin 19 as a communication line for 1-Wire eeprom of the addon board.I think we can only rely on VCC_SYS being present in the bootloader and I've assumed the IO voltage will start off at 2.5v. Hopefully SENSOR_IO_0 could sense the pull-up to stay compatible with an FPA that needs every pin(?). VGN, would you be interested in using this in the bootloader to ID other sensors?
We can also rely on VCCIO_FPGA and VCCIO_FPGA+SENSOR (controlled over Q3 mosfet). Bootloader can switch VCCIO_FPGA+SENSOR power on, that will power a 1-wire ID eeprom, which will be able to communicate with FPGA over pin 19 of X2 connector, that already have a pull-up resistor R6. Though I think that this resistor should be a bit stronger, i.e. probably 4.7K will be fine for 1-wire. How about AT21CS or 11LCxxx EEPROMs ?Switching IO voltage after the bootloader could be a problem though. What if you passed VCCIO_FPGA through? These EEPROMs can do 1.8v to 5.5v, I could get rid of the regulator and they'd follow the IO voltage.
This is already done) Just look though the schematics again, BTW I have uploaded lastest v1.1.0 schematics. VCCIO_FPGA+SENSOR is the same VCCIO_FPGA, controlled over Q3 mosfet.
-
Quote
ISC0901B0 requires 84KB for a single bias frame. As I know, original NV3 has 4 table to cover wide camera temperature working range, i.e. we should store 84x4 = 336KB of bias values, i.e. we need 4mbit eeprom. But as I know, you can hardly find 1-Wire, UNI/O or even I2C eeprom of such capacity. For higher resolution sensors, we probably will require even more. Also loading bias table from I2C eeprom will be slower than from QSPIx4@65MHZ, that impacts on statup time.
I was thinking that because the bias is 6bit it's only really 64KB per bias array. A lot of the data should be similar(?) and compress well. But I didn't realise there were at least 4! And slow too, I take your points. I wonder how compressible they are though.QuoteWe can also rely on VCCIO_FPGA and VCCIO_FPGA+SENSOR (controlled over Q3 mosfet). Bootloader can switch VCCIO_FPGA+SENSOR power on, that will power a 1-wire ID eeprom, which will be able to communicate with FPGA over pin 19 of X2 connector, that already have a pull-up resistor R6. Though I think that this resistor should be a bit stronger, i.e. probably 4.7K will be fine for 1-wire. How about AT21CS or 11LCxxx EEPROMs ?
I didn't read the schematic very carefully sorry. I assumed the regulators were being digitally adjusted with PWM or something. So I was thinking we had to decide which voltages to program and make sure to not enable anything in the bootloader...
I'll add a 1-Wire EEPROM like you've suggested. Would it be fine for the bootloader to detect presence by trying to communicate the whole time pin 19 isn't pulled down? -
There are 4 groups of 336*256*9byte data in FLASH, are they encrypted bias data?
-
Hi Vaagn,
do you think it is possible to order the stencils for most of the PCB‘s ?
I think otherwise it is a pain to assemble them. -
Ignore that comment, please - it was ill-conceived