OpenIRV. ISC0901B0 (Autoliv NV3, FLIR E4/5/6/8) based opensource thermal camera

[VGN]

That sounds great for this little test. What I meant is using a ring magnet with the right dimensions. They can be bought via ebay. Have a look at the attached picture. If there is a fitting magnet no additional plastic frame for the magnets is needed. Just glue the ring magnet to the housing.

Yes, a ring magnet would be the best solution. On the other hand, small magnets are very cheap and easy to find, while ring magnets are quite exotic. Anyway I measured mounting seat dimentions, you can find them in attachments.

Had a look at the schematic. Debouncing of the buttons will be done in VHDL via a filter in the FPGA? In the microcontroller unit it is an annoying task to debounce something.

Buttons pull-ups and debouncing capacitors are at the P-board (peripheral). Of course, there is additional debouncing logic in the HDL design. Agree, filtering buttons by internal mcu is a poor idea.

In the original design they seem to have gone to great lengths to keep the lens assembly from rotating when focusing.  There are 4 sliding teflon surfaces between the focusing ring and the lens.  Was that to keep wear particles from the threads off the sensor?

This unit looks quite over engineered for me. Maybe I don't understand something, but really, no idea, why using so many parts just for a fixed focus...

Looks like the foam gasket that seals the bottom of the lens to the housing is not thick enough to seal for close focusing if it requires more than about .5mm expansion from the stock setting. 

Right, but I don't think that we need this foam gasket, it prevents lens rotation a bit. Also this camera is not supposed to be used in any aggressive environment, no water/dust protection. BTW, even Flir Vue Pro is not water/dust resistant.

A popular way to get around lead screw slop on 3D printers is plastic nuts. They apparently last long enough for that extreme task.  Why not replace the threaded focusing ring with a combined focusing ring + ring gear version.  The SLA printer may be up to the task.  The threads don't have to be perfect, the errors control the slop.  And you get rid of a glue joint !  The threaded portion could be backed with a circlip if required. edit, not enough room

I'm afraid, that custom plastic nut probably will rotate too hard, and this tiny focus motor won't be able to make it move. All the more so I'm going to change the focus motor with higher RPM one. We should somehow keep lens rotating very easily and without backlash, otherwise, there is no chance for fast focus adjustment.

[bicycleguy]

Here's what I was thinking of:

There is enough room for a steel ring spring that could be tensioned for a nice fit.  I haven't tried on my filament printer.  Wonder how it would do on SLA.

[VGN]

Here's what I was thinking of:
There is enough room for a steel ring spring that could be tensioned for a nice fit.  I haven't tried on my filament printer.  Wonder how it would do on SLA.

WOW! I take my words back. These cutouts are very interesting! 
That definitly should be tested on SLA, pity that I don't have any 3D printer at all. I could purchase it for test, but only later with other parts, which I'm going to fix a little too.

1. How are you going to make the steal ring?
2. What about a special internal border that will help to center and fit the lens? Check out attachments. But I'm not completely sure how to make it properly, this is just a suggestion.
3. So the holding ring thread pitch is 0.508 (20mil)?! I thought it is metric 0.5mm pitch, though I didn't measure it, not even sure that I could...

[bicycleguy]

1. How are you going to make the steal ring?

Just bend some music wire around a pipe of a guessed dia.

2. What about a special internal border that will help to center and fit the lens? Check out attachments. But I'm not completely sure how to make it properly, this is just a suggestion.

I think the attachment might have messed up, all I see is what I previously posted.    edit  Whoops I didn't look close enough.  Have to think about this a little more.  I was still planning on using a flattened wave spring, but that doesn't make any sense.

3. So the holding ring thread pitch is 0.508 (20mil)?! I thought it is metric 0.5mm pitch, though I didn't measure it, not even sure that I could...

I don't know the pitch either, but the dia. is .998 inch so I assume 1 inch with 50thrds/inch.  Could be something else.  So short not sure it matters, especially with plastic threads.

I thought you had an SLA printer for the parts you posted?

[VGN]

I thought you had an SLA printer for the parts you posted?

No, I don't have it, I just order printing.


P-board schematics are avaliable: https://github.com/OVGN/OpenIRV/tree/master/docs/schematics
Most of the work is done, I just have to finish rounting and check everything very carefully before ordering new PCBs.
Display and Focus boards schematics will be published in a few days.

M-board is also modified a little. Added ESD protection for JTAG. Also increased sensor IOs number from 18 to 20.
I hope that 20 IO + I2C IO-expander is enough to interface most sensors and even high megapix CMOS digital image sensors like MT9F002 and others, for example.

Finally solved the problem with board stacking. Two special M1.6 SMT board spacers allow properly connect main and peripheral boards without original aluminium housing.

[bicycleguy]

@VGN  Thinking about the focus.  Your original design just needs a weaker spring.  The spring deflection is proportional to thickness^3.  Maybe the existing spring can be etched in something like pcb copper etch solution to the proper thickness.  Not sure if it has a coating on it.

[VGN]

@VGN  Thinking about the focus.  Your original design just needs a weaker spring.  The spring deflection is proportional to thickness^3.  Maybe the existing spring can be etched in something like pcb copper etch solution to the proper thickness.  Not sure if it has a coating on it.

I also tend to this. I'm going to try both the stencil spring and the magnet based one.

The spring deflection is proportional to thickness^3.  Maybe the existing spring can be etched in something like pcb copper etch solution to the proper thickness.  Not sure if it has a coating on it.

Not sure if that's going to work. Also far not everyone will be able to repeat this.

[horstmannhid]

Yes, a ring magnet would be the best solution. On the other hand, small magnets are very cheap and easy to find, while ring magnets are quite exotic. Anyway I measured mounting seat dimentions, you can find them in attachments.

You are right. I also didn't find a fitting ring magnet. I think it would be a lot better to use normal magnets.


Had a look at your new schematics. Great work!

For debouncing of the buttons I usually use the following circuit. So the bouncing is filtered even when the button is pushed.



Do you know the standby current of the whole system? Button 3 seems to turn on the whole system.

[VGN]

Had a look at your new schematics. Great work!

Thank you! Feel free to ask me any questions! Any suggestions are welcome for discussing.

For debouncing of the buttons I usually use the following circuit. So the bouncing is filtered even when the button is pushed.

Oh...right, pull-up + RC filter looks much better, going to fix this. Thanks!

Do you know the standby current of the whole system?

Of course, 8uA (micro amps).

Button 3 seems to turn on the whole system.

Right, button 3 and also EXT_GPI/PWR_ENA pin at the X9 connector is used to enable the whole system.

[VGN]

Focusing mechanics with magnet spring. Works pretty well, though a bit noisy, as focus board amplifies the motor vibrations, this is going to be fixed.
I'm also planning to replace the motor with higher RPM one, probably we could achieve faster lens movement and less noise.

https://youtu.be/jcN1_7Xi5mU

[Pantheron]

Dear VGN,

amazing project!
It clearly shows your dedication in every aspect of the complete project.
The lovely designed Focus mechanism.
The Housing.
The Software and even the PCBs are designed with love!

I really appreciate this.


But i also got some questions - sensor related.

You are reading the sensor out with 60fps?
How low can the clock for reading go, so it i spossible to readout at 10fps? or ist just internally fixed to 60fps?

For what is the CMD Pin on the sensor? 1 Wire? Any information on that?




Thank you.

[VGN]

amazing project!
It clearly shows your dedication in every aspect of the complete project.
The lovely designed Focus mechanism.
The Housing.
The Software and even the PCBs are designed with love!

I really appreciate this.

Hi! Many thanks!

You are reading the sensor out with 60fps?

Yes, the raw 14-bit per pixel data is comming out at 60fps. I left in my previos posts a link to my google drive with a video, captured right from ther sensor.
Here is the link: https://drive.google.com/file/d/1QfK6TBKxTnJjb9-Vjo1R4MsX56F_mhFq/view?usp=sharing
I advise you to download (~180MB) this video before viewing, as not smart youtube downgrades video quality too much.

How low can the clock for reading go, so it i spossible to readout at 10fps? or ist just internally fixed to 60fps?

We don't have and I believe will never have access to datasheets of this core, so I actually don't know that. Original board is forwarding clock at 73.636MHz, I'm using the same value. You will be able to experiment with clock rate, as soon as the HDL-design will be published.

On the other hand, I don't think that you really need it. If you would like to integrate this camera with any other hardware, you can choose USB, HDMI, AV or a low speed interface access over GPIO lines at X9 connector of the P-board. I have plans to implement a SPI interface over GPIO line, so that you can control the camera and readout the video data at any FPS you would like.

For what is the CMD Pin on the sensor? 1 Wire? Any information on that?

Forget about any standard interface with this FPA, everything is proprietary, the CMD interface as well. There are a lot of thing to tell about, I'm preparing a special manual for this. Well, the short answer, this is a 1-bit command line, synchronous to the main clock (73.636MHz). It is used to control the sensor. According to my investigations, a special 22 byte command is sent to the sensor to get each new frame. First two bytes of the command are 0x3F and 0xC0. I'm 99.9% sure, that this is a preamble that helps the core to detect a new command. Why? There is no any other strobe to validate the incomming command, 0xC0 = ~0x3F, very likely to be a preamble. Also, the core does not stream anything if I flip any bit in these two bytes. Other 20 bytes keep some special data to control the sensor. Unfortunatelly, the command fields are not byte aligned, that makes investigations a bit harder. There are a lot of bits that do not affect anything. Also a command for one sensor may not fit another, as themal array parameters are very different from core to core. Though I don't know what all the command fields are doing, I could find some common command pattern that should fit all sensors and a single special field that you can quite easily tune to get the image. I have tested this algorithm on my three FPAs and could make all of them working well. I'm sure that the more people will play with this sensor, the faster we will find out the function of each command field. Another way is to reverse engineer the firmware and get this data from original EEPROM. But, I'm not sure that this is a good idea, as this is an intellectual property of Avtoliv/FLIR.

[rockwell]

Do you think it's possible to have the sensor connected via a shielded cable from about 50cms long to the controller electronics?

[VGN]

Do you think it's possible to have the sensor connected via a shielded cable from about 50cms long to the controller electronics?

Do you mean the ISC0901B0 sensor?

[rockwell]

Yes, only the sensor in a compact as possible housing.

[VGN]

Yes, only the sensor in a compact as possible housing.

Sensor lines IO standard is LVCMOS 2V5. Data stream is running at @73.636MHz SDR. 50cm is a quite long distance, I'd not say that this is impossible, but will be a bit challenging. You should take in account proper termination of the I/O lines to exclude signal "ringing". I also think that using buffers/repeatters for the five high speed lines, i.e. clock, bias, data_odd, data_even, cmd is also a good idea.

After that keep in mind that the interface between FPGA and sensor is source synchronous, i.e. FPGA is generating the clock and sends it to the sensor. Bias and cmd line are also synchronous to the rising edge of the outgoing clock. At the same time sensor outputs data_odd and data_even signals synchronous to the falling edge of the incomming clock. Signals over PCB usually travels at a speed of 165ps/inch. So, 50cm gives about 3.2ns of delay. At 73.636MHz clock period is about 13.6ns. In ideal conditions (without your expander) we have 13.6ns/2 = 6,78ns of margin before and after the active edge of the clock signal to properly read incomming data at the FPGA side. But in case of additional delay of 3.2ns, the margin after the active clock edge will be lower and probably we will have to compensate this delay for proper operation. Current FPGA is quite powerfull to make such kind of tricks, so yes, this is possbile in certain conditions.

I'm not sure, but making such a long sensor power lines can also be a problem. The noise level of bias and core power rails should be as low as it is possible. This is a matter of experiments.
If it is not a secret, why do you need such a long cable?

[rockwell]

Many thanks for the comprehensive answer.
And no it's no secret at all, I would like to mount the sensor on a very small pan/tilt head on a Trimble UX5 drone.

[VGN]

Many thanks for the comprehensive answer.
And no it's no secret at all, I would like to mount the sensor on a very small pan/tilt head on a Trimble UX5 drone.

It was worth starting with this)) In this case I don't really think that this expander will be reliable enough. At the same time dimentions and weight of the camera is quite critical for small UAVs, that cannot carry this big body camera. But I think I have a solution.

The M-board, as the most expensive part, mostly because of the FPGA, was designed in a way to be fully independent from all other boards, including P-board. That's why for small UAV applications a new special board should be designed. Check out attachments, I just have copypasted some parts from the other boards.

We don't need focusing board, only shutter. So, right now the camera module that you see in the attachments weights 78g. The new P-board will add another 10g, resulting to 90g.

I have small experience with UAVs, and absolutely no any experience with FPV, FPV-cameras, interfaces, etc... Are you still flying on NTSC or PAL? 
That's why I have some question to you and community:

1. What do you think about this addon? Is it worth doing?
2. What about total weight and dimentions?
3. What kind of interfaces FPV pilots use nowadays? Is AV enough? What is the output format (NTSC/PAL/or...)?
4. What else is reasonable to add to the board?

[rockwell]

It's amazing how you can design and develop in such a short time projects like this !

But in this case, weight is not my problem, my UAV is equipt with a fixed 36 MP mirrorless full frame camera with a 15mm lens and is normally used in mapping and surveying for making orthomosaics in combination with a very capable GPS system.

My problem only is the limited space on the pan-tilt head that I would like to mount on this UAV, after that I removed the above mentioned camera.
Transmission could be done with HDMI or AV, 2.4GHZ or 5.0GHZ FHSS.
I could opt for a bigger pan-tilt had but i'm afraid this causes to much drag for this fixed wing UAV.

[VGN]

Transmission could be done with HDMI or AV, 2.4GHZ or 5.0GHZ FHSS.

Thanks, streaming HDMI is not a problem, though I can't decide what kind of connector is better to use. I saw different FPV HDMI transmitting hardware, everything looks to be proprietary.

I could opt for a bigger pan-tilt had but i'm afraid this causes to much drag for this fixed wing UAV.

Looks like this drone is not designed to carry any camera outside of the body. But there is some free space at the nose, probabry you could make some mount for a forward looking camera. In any case, the best FPV mode of this themal camera that I can suggest you is 90g within dimention of ~50x50x50mm.

[rockwell]

You are right, normally this drone carries a fixed camera looking straight down.
As this drone is launched from a catapult i can't atach a gimbal to the belly, but i can modify the nose section so it can hold a small pan-tilt head.
Again, weight is not the problem, it's size that matters.
But don't focus to much on my wishes, I can barely wait to get this camera working.

[VGN]

Finally released the first version of HyperBus memory controller. The migration from Spartan-6 was not so easy, as I expected. I had to rework quite a lot of things, and spent much more time, than I planned, but this controller is really one of the most important and critical parts of this project.

I created a special thread about this memory controller. If you have any questions or suggestions, consider using this thread:  https://www.eevblog.com/forum/fpga/openhbmc-open-source-axi4-based-hyperbus-memory-controller/

Link to the repo: https://github.com/OVGN/OpenHBMC

[zxase258]

amazing project!
It clearly shows your dedication in every aspect of the complete project.
The lovely designed Focus mechanism.
The Housing.
The Software and even the PCBs are designed with love!

I really appreciate this.

Hi! Many thanks!

You are reading the sensor out with 60fps?

Yes, the raw 14-bit per pixel data is comming out at 60fps. I left in my previos posts a link to my google drive with a video, captured right from ther sensor.
Here is the link: https://drive.google.com/file/d/1QfK6TBKxTnJjb9-Vjo1R4MsX56F_mhFq/view?usp=sharing
I advise you to download (~180MB) this video before viewing, as not smart youtube downgrades video quality too much.

How low can the clock for reading go, so it i spossible to readout at 10fps? or ist just internally fixed to 60fps?

We don't have and I believe will never have access to datasheets of this core, so I actually don't know that. Original board is forwarding clock at 73.636MHz, I'm using the same value. You will be able to experiment with clock rate, as soon as the HDL-design will be published.

On the other hand, I don't think that you really need it. If you would like to integrate this camera with any other hardware, you can choose USB, HDMI, AV or a low speed interface access over GPIO lines at X9 connector of the P-board. I have plans to implement a SPI interface over GPIO line, so that you can control the camera and readout the video data at any FPS you would like.

For what is the CMD Pin on the sensor? 1 Wire? Any information on that?

Forget about any standard interface with this FPA, everything is proprietary, the CMD interface as well. There are a lot of thing to tell about, I'm preparing a special manual for this. Well, the short answer, this is a 1-bit command line, synchronous to the main clock (73.636MHz). It is used to control the sensor. According to my investigations, a special 22 byte command is sent to the sensor to get each new frame. First two bytes of the command are 0x3F and 0xC0. I'm 99.9% sure, that this is a preamble that helps the core to detect a new command. Why? There is no any other strobe to validate the incomming command, 0xC0 = ~0x3F, very likely to be a preamble. Also, the core does not stream anything if I flip any bit in these two bytes. Other 20 bytes keep some special data to control the sensor. Unfortunatelly, the command fields are not byte aligned, that makes investigations a bit harder. There are a lot of bits that do not affect anything. Also a command for one sensor may not fit another, as themal array parameters are very different from core to core. Though I don't know what all the command fields are doing, I could find some common command pattern that should fit all sensors and a single special field that you can quite easily tune to get the image. I have tested this algorithm on my three FPAs and could make all of them working well. I'm sure that the more people will play with this sensor, the faster we will find out the function of each command field. Another way is to reverse engineer the firmware and get this data from original EEPROM. But, I'm not sure that this is a good idea, as this is an intellectual property of Avtoliv/FLIR.

Hello VGN, in the FPA timing I measured, there is another signal I don't understand its function. This signal is temporarily called "PIN6", and it appears together with the FPA data output PIN2 and PIN3. I don’t know what it does. Is it “exposure/integration” control?
In addition, from the rising edge of FSYNC (DATA IN) to the rising edge of the first data output, there are a total of 200 clocks. I think FPA samples on the falling edge of the clock.
I got a FLIR TAU324, it uses ISC0601B detector, I will measure the timing of ISC0601B. I think I also need an "FPA protocol analyzer". Specifically, I need an FPGA to monitor the commands sent by the camera to the FPA, and then send these commands to the computer to analyze how the camera configures the FPA. However, I am still new to FPGA, maybe I should ask my friend to help me do this.

73 BI4LBK.

[VGN]

Hello VGN, in the FPA timing I measured, there is another signal I don't understand its function. This signal is temporarily called "PIN6", and it appears together with the FPA data output PIN2 and PIN3. I don’t know what it does. Is it “exposure/integration” control?

If you are asking about ISC0901B0, your "PIN6" looks like to be a pixel BIAS value, check out schematic of the M-board in my repo. Controller sends 7-bit (in fact only 6-bit are significiant) LSB-first BIAS values for each pixel, each row. Approximate value is about ~0x25.

In addition, from the rising edge of FSYNC (DATA IN) to the rising edge of the first data output, there are a total of 200 clocks. I think FPA samples on the falling edge of the clock.

Is FSYNC (DATA IN)  a CMD lines in my terms? Anyway don't focus at the number of the clocks of the data output relative to the command. There is a parameter in the command that determines this value.

Concerning to the sampling edge, you are right, sensor works at the falling edge. Check out attachments. cmd_to_clk - is command or bias comming out of the controller. data_to_clk is one of the data lines comming out of the sensor.

That would be very cool if you get ISC0601B oscilloscore timing. I think this ISC0601B and ISC0901B0 are very common.

[tisher]

Car/vehicle parts

NV0  (flir pathfindir)

(12 pin connector with composite Video-out)
334-0008-00P
334-0005-00S
334-0001-00

NV1 (BMW):

(12 pin connector, no composite Video-out)
66546988779
66549134267
66549141606
66549132753
66546988774
66549125733

NV2:

(6 pin connector)
4H0980552
9205899 (66549205899)
9175668 (66549205899)

NV3:

(4 pin connector)
A2229057307
A2229052805
A2228201597
4G0980552A
9322653 (66549322653)

NV4:


btw, wat's lens dimensions in NV3?