Higher frame rate for Lepton?

[mikeselectricstuff]

Why not just buy three 9Hz cameras, offset each the master clocks by multiples of 120 degrees and combine the images digitally.

Integration time - chances are you'd get a lot of overlap between images

[Fraser]

Blueskull,

Russia has signed up to the Wassennar Arrangement.
China tends to use ULIS Microbolometers that are made in France. That may effect who the Chinese OEM's will sell to. Neither China nor Russia are actually considered enemies of the West. We have our disagreements at the Government level and Russia invading Ukraine has understandably caused issues. They are still not our enemy however.

Fraser

[-jeffB]

Even if it can be modified, because the manufacturer didn't put in place the correct safeguards, all that means is that the manufacturer broke the law. If I then modify it, I'm not breaking the law, because I'm a US citizen living in the US. However, after such a modification, it becomes my legal obligation to NOT export it to anybody outside the US (and also may be illegal to sell it to anybody in the US who's not a US citizen). There is no legal obligation however, at all, for me to avoid discussing the topic on the internet. If, for example, I had knowledge (which I don't actually) of how to modify it to get full 30Hz frame rate, there is no law that forbids me from posting such information on the internet. Such a law would be a violation of my first amendment right to free speech. The only crime would be if I posted it with the intent to help enemies of the US, which would be an act of treason. Spreading info to cause harm is illegal, while spreading info for the sake of simply educating others on a topic is not illegal. Intent is the key.

So, suppose you're right, and there's no law preventing you from developing and disseminating such a patch. You do so, secure in your Constitutional rights, or some such. What happens next?

If the powers that be go after FLIR for failing to lock down their modules, the very first thing they'll have to do is stop selling them. That might not sink the company entirely, but it would mean no more FLIR modules at any frame rate for you or anyone else until they come up with a fix.

If they do come up with a fix, they're likely to lock down any other "hackable" settings as well, one of the things we've feared about the Ex series.

Oh, and by the way, I'm pretty sure that the FLIR user agreement expressly forbids any attempt to reverse-engineer or modify the internal workings of the device. You may or may not agree with the legality of such agreements, but they give FLIR a legal opening to come after you, directly, and maybe to come after EEVblog as an accessory. So far, FLIR hasn't been aggressive about pursuing that avenue. If someone came up with a mod that brought the US government down on FLIR's head, do you think they'd be so forbearing in the future?

It's a bad idea. It's such a bad idea that the entire community of independent, free-thinking, extremely smart, and not-terribly-impressed-with-ITAR engineers and enthusiasts here have all refrained from going there.

You might be right, and everybody else wrong. You might have thought this all the way through, where nobody else has. I don't think that's the way the smart money would bet, though.

[mikeselectricstuff]

The mere act of discussing how to get around certain safeguards in equipment isn't a crime in itself

I'm not sure that's the case - ITAR covers documentation as well as products, though I don't know if this only covers disclosure by the manufacturers, or by anyone.
 

(at least not in the US, because of freedom of speech).

If the DMCA can override this, I'd be surprised if export restrictions weren't also exempt from FOS
 

[Ben321]

The mere act of discussing how to get around certain safeguards in equipment isn't a crime in itself

I'm not sure that's the case - ITAR covers documentation as well as products, though I don't know if this only covers disclosure by the manufacturers, or by anyone.

Documentation is officially released descriptions of a device's capabilities. I'm pretty sure that whatever ITAR-covered docs that a manufacturer has, can't legally be disseminated by either them or the end user. But discussing a bug (which probably isn't even documented or known to FLIR) isn't disseminating documentation. If it's a bug discovered by the end user, there's no law the prevents online discussion of that bug, because the online discussion of that bug isn't "documentation". A bug is not a spec or capability of a device, so discussion of it isn't formal documentation. Notice that this forum doesn't have any layout of tables or charts or other organized grouping of facts about the device. It's just a set of paragraphs of text written by various users, which is the textual equivalent of having a conversation with somebody. The US government doesn't (and can't legally, because of the constitution) prevent 2 or more people from holding a conversation on any topic.

[Ben321]

Want to get some high FPS cameras? Look for something from a non-western friendly country which doesn't give a shit to US ITAR, such as China and Russia.

https://item.taobao.com/item.htm?id=546722588126
http://rusoptics.com/?module=catalog&action=viewproduct&id=355

The first one is a 25Hz 384*288 design, the second one is of same resolution and refreshed at 30Hz.

And stepping up the price, here is a Chinese 33Hz, 640*512 IR camera. Not cheap though.
https://item.taobao.com/item.htm?id=527428300256

Thanks for the idea. A US company (company who's headquarters is in the US) is held to US laws. A company that exists outside the US is not held to US laws. A Chinese company has no obligation to fill out ITAR paperwork prior to selling a thermal imager to another country, while a US company does have to fill out ITAR paperwork prior to making that sale. Of course the US government also has the right to refuse to do business with that company (not buy any equipment from them), and theoretically even put trade sanctions on any country (not allow any imports or exports from that country) that allows their companies to operate in a way that the US government doesn't like, but the US government doesn't have the authority to put trade sanctions on a single company (only on the entire country that that company operates in, which would effect ALL companies in that country).

So your idea of buying from China or Russia, who's companies are under no legal obligation to avoid selling thermal imagers to countries that the US doesn't want them sold to (because US law jurisdiction does not extend to other sovereign nations), is a GREAT IDEA! Problem is navigating their sites, and making the purchase could be REALLY TOUGH, if they don't have English language websites.

[Ben321]

And let's say you've managed to hack it, US DOC can easily ban the export or even manufacturing of Flir Lepton. For the sake of every thermal imaging fan, if you find a good way of hacking it, stay quiet and stay low.

All they'd have to do would be to hardwire the frame rate (and that may already be the case). Frame rate configuration could be etched directly into silicon on the controller chip. After the US govt notified FLIR, it probably could have a fixed version out in about a month. Also I think the US has only ever put restrictions on cameras that were designed to have the frame rate changed, and don't ban manufacture of devices that take considerable effort to modify, because technically ANYTHING can be hacked with enough effort. And keep in mind, that high frame rate devices are still perfectly legal to sell in the US, so even if the US govt did crack down, the most that they would demand from FLIR is that they stop shipping the Lepton to countries outside the US. Most likely in such a situation though, as with other 30Hz thermal cores/cameras, if the purchaser does live outside the US, the purchaser is required to send in ITAR documents to FLIR indicating ITAR compliance by the end user, and then you can still buy it even if you live outside the US (with the exception of a few enemy nations like Iran and North Korea). Buying ITAR restricted devices only means this extra bit of a couple documents to fill out which isn't too much of a burden (they can usually be filled out on your computer and emailed to FLIR or whoever the manufacturer is for the camera you are trying to buy).

Has there ever been a documented case of a thermal imager that was intended to be 9Hz only, but was then modified by an end user for higher frame rate, and then in response the US government ordered the manufacturer to stop selling the device, or even put in place any kind of restrictions on its sale?

[Ben321]

The Russian site is in English. Forget about the Chinese site. I just tried to access it from US. While not under ITAR control by US, it's under Chinese similar export control.

That being said, there are always buyer agents and reshippers in case you want one, and if you don't tell them, no one knows it's export controlled. You do need a VPN to bypass Taobao's IP blocking otherwise those controlled goods' web pages will be just 404.

I'm able to access https://world.taobao.com/item/527428300256.htm?fromSite=main without a VPN. However this one https://world.taobao.com/item/546722588126.htm?fromSite=main gives this error

??

(It seems that the forum dosn't support Chinese characters, as they all got turned into smileys, and in the editor here they are now question marks)

Which Google translate says means

According to the laws and regulations or platform policy of the host country, this item is subject to purchase.

So basically I would need to fill out paperwork that are China's equivalent to the US ITAR compliance paperwork, in order to buy this item.

As for the Russian one http://rusoptics.com/?module=catalog&action=viewproduct&id=355 it costs $3000 which is only slightly cheaper than the FLIR Vue 640x480 30Hz camera https://www.adorama.com/fliv6401330.html at $3199.
At that price, I might as well buy the FLIR one, as it is a known company that makes very good quality products. FLIR is basically synonymous with thermal/LWIR imagers.

[Ben321]

It says due to laws and regulations of host country, this item is restricted to buy. In China, restricted is just a tactful way to say forbidden.

Does that mean you can't even fill out paperwork and buy it, like China is refusing to sell it to the US completely?

And stepping up the price, here is a Chinese 33Hz, 640*512 IR camera. Not cheap though.
https://item.taobao.com/item.htm?id=527428300256

Not expensive at all. I recognize that symbol on the price "¥" and it is the JPY (Japanese yen). So that means the price is 49999 JPY, not $49999. In fact, as the JPY is approximately equal to 1/100th of the USD, that would make the price approximately $500. That's actually VERY CHEAP for a thermal imager in the 640x480 category.

[hendorog]

Why not just buy three 9Hz cameras, offset each the master clocks by multiples of 120 degrees and combine the images digitally.

Integration time - chances are you'd get a lot of overlap between images

Trying to get my head around this, to me they seem to be saying the time constant is about same between 9Hz and 30Hz cameras:
http://flir.custhelp.com/app/answers/detail/a_id/236/related/1

And here they are saying that the time constant covers integration time, shutter speed and exposure:
http://www.flir.com/cvs/cores/knowledgebase/index.cfm?CFTREEITEMKEY=1136&view=35726

To my untrained (and FLIR lacking) eyes it looks like it would actually work.

Standing by for a beating with a clue stick...   

[Bill W]

This 12ms is the pixel time constant, the time to reach 1/e of the new steady state temperature from a change.  (VOx is slower than ASi, but unlikely to notice it visually).

The integration time ('exposure') is more to do with sampling the pixel resistance into the readout circuit.  Would have to be be in the microseconds on a QVGA/VGA resolution sensor being read out line by line, but on a smaller sensor could be quite long or even oversampled to reduce noise.

Obviously all you need to do is to finish reading the whole sensor before you you need to start again, so a 60 line sensor at 9Hz could be nearly 2ms integration time. 

In a Wassenaar compliant reduced frame rate camera there is nothing to say HOW you reduce frame rate, just that you must not present more than 9 images a second to the user.  You could scan slowly or scan fast and just sit doing nothing in between.

Not all 'low rate' cameras are run at 9Hz by the way, many video based systems are 8.3Hz or 7.5Hz, but that is another story.

Bill

[Bill W]

The mere act of discussing how to get around certain safeguards in equipment isn't a crime in itself

I'm not sure that's the case - ITAR covers documentation as well as products, though I don't know if this only covers disclosure by the manufacturers, or by anyone.

not ITAR, Wassenaar covers this one:

http://www.wassenaar.org/wp-content/uploads/2016/12/List-of-Dual-Use-Goods-and-Technologies-and-Munitions-List-Corr.pdf
Relevant clause is 6 D 3 c on page 129
- 'software' to enable circumvention of a 9Hz limit on a camera is itself export controlled.  So putting any hack information on the internet is likely to be exporting from your home country as you have no control over the servers involved, and so would apply to anyone in a Wassenaar signatory country.

Bill

[CatalinaWOW]

Whatever you personally think of the regulators (Wassenaar, ITAR) and whether the regulations they promulgate are good are bad, they are quite serious about it.  I have been personally involved in a product that was exported that some felt crossed the appropriate lines.  Being called for questioning into closed rooms in the bowels of large non-descript buildings with threats of real unpleasantness if things went the wrong way was terribly unpleasant.  Something any sane person would go to some lengths to avoid.

Think very carefully before you decide it is important enough to you to go through your personal version of this experience.  Which would undoubtedly be more unpleasant since we had documentation showing that we were trying to be in compliance and in fact did comply with the letter of the law, though some did not agree with that interpretation.  You would go into such discussions having gone on public record that you were looking for ways around the laws.

[-jeffB]

All they'd have to do would be to hardwire the frame rate (and that may already be the case). Frame rate configuration could be etched directly into silicon on the controller chip. After the US govt notified FLIR, it probably could have a fixed version out in about a month.

Spoken like someone who's spent years in the semiconductor manufacturing industry. 

Do you really think that, in a month, FLIR could:

• design the modifications to hardwire the refresh rate instead of loading it as a config param
• verify the firmware changes to support this
• verify and tape out the modified layout
• fabricate the new silicon
• assemble new test units using the new silicon
• run a complete test suite on the new units
• get the new parts up to full production
• assemble, package and distribute the new units

...oh, yeah, and reclaim all the existing units from their various sales and distribution channels, and destroy them in an accountable way that will satisfy the authorities. Don't forget that.

[mikeselectricstuff]

All they'd have to do would be to hardwire the frame rate (and that may already be the case). Frame rate configuration could be etched directly into silicon on the controller chip. After the US govt notified FLIR, it probably could have a fixed version out in about a month.

Spoken like someone who's spent years in the semiconductor manufacturing industry. 

Do you really think that, in a month, FLIR could:

• design the modifications to hardwire the refresh rate instead of loading it as a config param
• verify the firmware changes to support this
• verify and tape out the modified layout
• fabricate the new silicon
• assemble new test units using the new silicon
• run a complete test suite on the new units
• get the new parts up to full production
• assemble, package and distribute the new units

...oh, yeah, and reclaim all the existing units from their various sales and distribution channels, and destroy them in an accountable way that will satisfy the authorities. Don't forget that.

The info available suggests that the firmware is in OTP, so assuming it was real OTP, and not flash with a write-once feature that could be bypassed, any hack would probably exploit some weakeness, which could most likely be fixed with updated firmware.

[Bud]

Some potentially useful information can be found in this discussion:

https://groups.google.com/forum/#!topic/flir-lepton/jelgkPBxcmo

it has a link to a Lepton 2 SDK at the bottom. I am not qualified though to say how useful it may be, to a knowledgeable programmer perhaps at least it may help understand the device capabilities, including possibility of changing the frame rate.

[Ben321]

The mere act of discussing how to get around certain safeguards in equipment isn't a crime in itself

I'm not sure that's the case - ITAR covers documentation as well as products, though I don't know if this only covers disclosure by the manufacturers, or by anyone.

not ITAR, Wassenaar covers this one:

http://www.wassenaar.org/wp-content/uploads/2016/12/List-of-Dual-Use-Goods-and-Technologies-and-Munitions-List-Corr.pdf
Relevant clause is 6 D 3 c on page 129
- 'software' to enable circumvention of a 9Hz limit on a camera is itself export controlled.  So putting any hack information on the internet is likely to be exporting from your home country as you have no control over the servers involved, and so would apply to anyone in a Wassenaar signatory country.

Bill

Actually this doesn't cover descriptions of such software, only the software itself. Therefore, the loophole in the law here is this: If you know how to circumvent the 9Hz limit on a given camera, do NOT distribute your hacked firmware. Instead provide info describing in detail how your hack works. Provide enough detailed info that the hacked firmware can be created independently by anybody who reads your description, but don't distribute your copy of the hack, only the information on how to create the hack. Wassenaar does NOT prohibit distributing such information. Nor does ITAR. Until the US writes up a new law that would criminalize the sharing of such information, discussing such a hack is 100% legal.

[amyk]

That reminds me of this: https://en.wikipedia.org/wiki/Pretty_Good_Privacy#Criminal_investigation

[Fraser]

I have my doubts about your theory Ben321.

If I write a detailed book on how to build an improvised explosive device, I will likely get a visit from the police.

If I provide TECHNICAL detail in public of how to circumvent the frame rate limitation I am in contravention of the DUT regulations that control all technical aspects of the technology including publication of technical information. I would expect a document detailing the theory of how to circumvent the frame rate limit to be considered technical information by its very nature.

Put it this way.... do you want to be the legal test case ? I certainly do not.

To put this topic to bed from my perspective..... the LEPTON is a miniature module of high integration and is far harder to 'hack' than a conventional, larger scale, thermal camera core. Anyone wanting to hack such a highly integrated module is, IMHO, both brave and a Masochist !

Fraser

[CatalinaWOW]

Actually this doesn't cover descriptions of such software, only the software itself. Therefore, the loophole in the law here is this: If you know how to circumvent the 9Hz limit on a given camera, do NOT distribute your hacked firmware. Instead provide info describing in detail how your hack works. Provide enough detailed info that the hacked firmware can be created independently by anybody who reads your description, but don't distribute your copy of the hack, only the information on how to create the hack. Wassenaar does NOT prohibit distributing such information. Nor does ITAR. Until the US writes up a new law that would criminalize the sharing of such information, discussing such a hack is 100% legal.

As I said earlier the folks trying to control dissemination of these technologies are quite serious.  And at least as inventive in finding legal justifications for their actions as you are.  You really need to want to fight this battle and be willing to suffer the possible consequences.  There might actually not be such consequences, depending on luck, the particular toes you are stepping on and any number of other factors, but the risk is real.

I have better things to do, and better ways to prove my technical prowess, but your mileage may vary.

[hendorog]

We need a sticky to warn people that certain subjects will rapidly degenerate into a discussion of legal consequences (valid or not) - and so are just not worth the effort:

* Anything to do with flying FPV
* Anything to do with household power
* FLIR hacking
* ?

[Fraser]

Hendorog,

Your list is inaccurate.

No one has said hacking a FLIR is an issue.... many of us do that.

Hacking the frame rate of a FLIR is an issue due to the reasons for it being limited in the first place. The frame rate is just one element in the world of thermal camera hacking. The favourite hacks are still increased  resolution and additional Menu options. There is nothing legally challenging in those areas as they are not governed by international DUT restrictions

If these subjects were not raised in posts such as this, and others, people would not understand why members of this forum are not trying to circumvent the <9fps limit. I have no issue with Ben321 raising such questions. He just will not get the answers he wants from me, that is all.

At the end of the day, this is a friendly discussion that may be enlightening to some readers. As such it is good healthy activity on this technical forum.

No one got hurt and that is what matters

Fraser

[hendorog]

Your list is inaccurate.

No one has said hacking a FLIR is an issue.... many of us do that.

Hacking the frame rate of a FLIR is an issue due to the reasons for it being limited in the first place.

Yes you are correct - it should have been FLIR frame rate hacking...

[CatalinaWOW]

We need a sticky to warn people that certain subjects will rapidly degenerate into a discussion of legal consequences (valid or not) - and so are just not worth the effort:

* Anything to do with flying FPV
* Anything to do with household power
* FLIR hacking
* ?

I understand your point, and would add CAT ratings on meters to the list.

Having been bitten somewhat painfully on the ITAR subject in this general technology area I am sensitive, and really don't want someone else to go through that without understanding the course they are on.  My point isn't really a legal opinion, because the pain I went through was actually from doing something that even the pain inducers agreed was legal in the end.  Just pointing out an area where there are dogs that both bark and bite, sometimes without justification.   

I would in no way stop someone from doing something they feel strongly about.  It is just like working with dangerous voltages.  More power to you if you realize what you are getting into.

[hendorog]

We need a sticky to warn people that certain subjects will rapidly degenerate into a discussion of legal consequences (valid or not) - and so are just not worth the effort:

* Anything to do with flying FPV
* Anything to do with household power
* FLIR hacking
* ?

I understand your point, and would add CAT ratings on meters to the list.

Having been bitten somewhat painfully on the ITAR subject in this general technology area I am sensitive, and really don't want someone else to go through that without understanding the course they are on.  My point isn't really a legal opinion, because the pain I went through was actually from doing something that even the pain inducers agreed was legal in the end.  Just pointing out an area where there are dogs that both bark and bite, sometimes without justification.   

I would in no way stop someone from doing something they feel strongly about.  It is just like working with dangerous voltages.  More power to you if you realize what you are getting into.

Thanks and totally agree, CAT ratings another subject to avoid

It's just the natural way these subjects seem to go.

Also just for the record I wasn't criticising anyone - your ITAR interaction certainly made interesting reading and Fraser is basically Mr Lepton around here amongst other shadowy things he has alluded to