Author Topic: Open Source Oscilloscope, can it be done?  (Read 69921 times)

0 Members and 3 Guests are viewing this topic.

Offline axitece

  • Contributor
  • Posts: 12
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #125 on: December 12, 2017, 01:24:28 pm »
I have an idea to design new main board for Keysight 1000 x-series scope, to make it 800MHz bandwidth.  It have ADC, FPGA and processor on separated board, so I will be much easier then design all oscilloscope hardware and software for processor and FGPA. Do you have some schematics, application notes, literature and other resources, which can be helpful? I want to base on http://www.ti.com/lit/ug/tiduba4/tiduba4.pdf and on photos from Dave teardown - https://www.flickr.com/photos/eevblog/albums/72157657671196148 . I have complete schematic of Keysight main board from reverse engineering.

Would you mind sharing the schematics that you have reverse engineered? I would be very interested in seeing how the Keysight design compares to the already available schematics from Rigol and other vendors. Thanks a lot in advance!
 

Offline slicendice

  • Frequent Contributor
  • **
  • Posts: 365
  • Country: fi
Re: Open Source Oscilloscope, can it be done?
« Reply #126 on: December 14, 2017, 09:35:46 am »
Since opinions here seems to be very divided, I decided to contribute.

Looks like many here wants something like a high end oscilloscope with high bandwidth and multiple channels.

Let us take one commercial oscilloscope as a reference point:

Rigol DS6104
Bandwidth: 1GHz
Channels: 4
Price: > $10.000

Now let us compare this to OpenHarware/Software, by answering a few question:

Q: Would it be possible to develop such scope as OHW/SW?
A: Absolutely

Q: Will developing such scope cost more than $10.000?
A: Most likely

Q: Will the completed product, assembled and ready for use, cost more than $10.000?
A: Most definitely NOT!

Q: Who would benefit from such OHW/SW project?
A: Everybody who value OHW/SW.

Q: Would such scope be competitive/comparable in performance with commercial scopes?
A: Most definitely YES.
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #127 on: December 14, 2017, 03:33:34 pm »
We already have open source scopes on the market.  We just don't have the source code even when it falls under the GPL.

A number of OEMs produce instruments using embedded Linux.  Anything using the Zynq FPGA line is almost certainly using the Xilinx embedded Linux package.  The Arago/Yocto projects are also in use in T&M kit.  I know this from personal observation as do quite a few other people.

It seems to me far more effective to get the GPL code released and fix the bugs than it does to engineer a new instrument from scratch.  Even if a new Open Source design were made, someone would still need to manufacture it.  This is certainly not toaster oven reflow territory. Just tooling up to do the production QC is a major investment.

There is no evidence of Linux in the Rigol DS1102E, however, there's also unusually little text in the FW updates.  It does not take a rocket scientist to strip all the tell tale text from the source code.  But a port scanning tool designed to identify TCP/IP stacks should be able to tell quite easily for more recent models.  It seems highly unlikely that Rigol went to the expense of writing their own stack.  They could be using BSD licensed code, but if they are, the requisite copyright notice should be present in a FW update package for the DS1xxxZ.  Whatever instruments you have,  run strings(1) on a FW update package and grep(1) for a copy right notice.  Post what you find to the thread.

The best  way to proceed is to politely request the source code for any instrument you own using an embedded Linux distro. In the request make the point that whichever OEM provides good support for open source instruments will get your business.  They don't have to release the device drivers, but that's not that big a deal.  How many people worry about the nVidia drivers being closed source?  Most of the issues are UI bugs.

As a reminder, only people who own an instrument can make a valid request for the source code.  Only the copyright holder can take an enforcement action.  There will be a great deal of stalling by the Chinese OEMs. Rohde & Schwartz already supply the GPL source for their instruments on request by instrument owners.  Lots of polite requests for source sent to the marketing departments at the OEMs will make them realize that there is a marketing edge to be gained by complying voluntarily.  Unless the new "forgiveness" clause in GPL 2 is retroactive, failing to comply would force an OEM to take the product off the market.

I've got to take a long trip today, so I'll work on a draft source code request as I'm not driving.
 

Offline ogden

  • Super Contributor
  • ***
  • Posts: 3731
  • Country: lv
Re: Open Source Oscilloscope, can it be done?
« Reply #128 on: December 14, 2017, 09:05:13 pm »
The best  way to proceed is to politely request the source code for any instrument you own using an embedded Linux distro.

"Running Linux" does not universally mean "I can get source code for it". You cannot request source code of application that does not contain (or link to) GPL'ed software or parts of it. Even drivers can be proprietary, thus "closed source", if developed correctly from licensing point of view - as you correctly mention nVidia example. Those companies can afford to invest into development that does not rely on open source components, thus have no obligation to give away their intellectual property. All you can get - some irrelevant kernel code modifications that maybe show how to I/O hardware of instrument, but basically that's it. In case of scope it is even worse - because most of "business" happens not in the Linux "user land" application, but FPGA. You definitely can cool down on your idea :)
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #129 on: December 15, 2017, 01:56:02 am »
The best  way to proceed is to politely request the source code for any instrument you own using an embedded Linux distro.

"Running Linux" does not universally mean "I can get source code for it". You cannot request source code of application that does not contain (or link to) GPL'ed software or parts of it. Even drivers can be proprietary, thus "closed source", if developed correctly from licensing point of view - as you correctly mention nVidia example. Those companies can afford to invest into development that does not rely on open source components, thus have no obligation to give away their intellectual property. All you can get - some irrelevant kernel code modifications that maybe show how to I/O hardware of instrument, but basically that's it. In case of scope it is even worse - because most of "business" happens not in the Linux "user land" application, but FPGA. You definitely can cool down on your idea :)


Duh.  Yeah, entirely true.  Aside from a lack of attention to grammar, what is your point? The serious problems are in the UI, not the FPGA code.  What the FPGA does is so tightly controlled by mathematical physics that it's pretty trivial to recreate if there is a problem.  Apparently you think it's a big deal.  I don't know VHDL, but I know the math.  After a dozen or so programming languages, I don't see a new one as much of an obstacle.  Aside from that, there are plenty of people who already  know VHDL.  I'm sure some of them are interested.

I've had to maintain over 2 million lines of other people's code.  At *least* 500,000 lines of it had no comments other than the author's name. And the range of calculations being done in that stuff makes an oscilloscope look trivial.  I could do it because I knew "if it's Tuesday we must be in Rome".

My point is simply this.

If we can read the data from the input channels and write data to the display, the rest is easy to do.  It does *not* require doing something difficult like designing a broadband front end or manufacturing it. Writing a new UI is not a big deal.  And using what exists as a base is a waste of time it's so bad.

If you think the GPL code is irrelevant, you have no clue about the subject.  While it is not necessary, it is highly useful to have the source code for the GPL parts.  This is why Stallman created the Gnu license.  It doesn't solve all the problems, but it makes things a lot easier.



 

Offline ogden

  • Super Contributor
  • ***
  • Posts: 3731
  • Country: lv
Re: Open Source Oscilloscope, can it be done?
« Reply #130 on: December 15, 2017, 09:15:35 am »
Duh.  Yeah, entirely true.  Aside from a lack of attention to grammar, what is your point?

I find your public comment about my grammar as rude. If you want to improve my English which is no my native language - send PM instead.

My point is: "You cannot request source code of application that does not contain (or link to) GPL'ed software or parts of it.".

Quote
The serious problems are in the UI, not the FPGA code.

So what? How does it help if you cannot get source code of it? For example try to get source code of those products to improve them yourself:

https://en.wikipedia.org/wiki/List_of_proprietary_software_for_Linux

Quote
If you think the GPL code is irrelevant, you have no clue about the subject.

Don't put words in my mouth. Where did you read "irrelevant"?

Quote
I've had to maintain over 2 million lines of other people's code.

Well.. Seems, your experience is not enough to comprehend simple thing - Linux-based device can be designed such a way that you gain absolutely nothing by getting GPL-relevant source code of it. Most reputable manufacturers are careful enough so their intellectual property is safe from individuals like you and most importantly - from competitors.

For example check this thread where guys tear down one Linux instrument, see how this product is made from software point of view:

https://www.eevblog.com/forum/testgear/agilent-e7495-linux-root-account/


Quote
If we can read the data from the input channels and write data to the display, the rest is easy to do.  It does *not* require doing something difficult like designing a broadband front end or manufacturing it. Writing a new UI is not a big deal.

Sorry, but seems you have no idea how scope actually works and what is actual complexity of firmware parts that are not UI. FPGA firmware is much more complex part of whole thing. As frontend you simply can use reference design mentioned in this thread.

Do you really suggest that anybody would want to buy 10k$ scope, throw away it's original software including warranty and use "open source firmware" that is developed by unknown guys who most likely will not support "product" in time when it is needed?
« Last Edit: December 15, 2017, 12:19:15 pm by ogden »
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #131 on: December 15, 2017, 02:48:27 pm »
I was deliberately rude because your sole interest is trumpeting that you are right and everyone else is wrong.   That may make you feel good, but it does not accomplish anything useful. You're also sloppy in reviewing what you write for errors.  I suggest you read your own post.  I did *not* put words in your mouth.

You have no way of knowing what I do or do not know.  Fact is I have repaired several analog scopes including a Tek 465.  I also have over 30 years of DSP experience.  There are lots of things I don't know.  But I do know how to learn.  If you actually knew what a DSO is doing, you would not consider it very complex.  It's not.  There are some significant timing challenges, but that and broadband circuit design are the only hard parts.  The timing constraints pretty much dictate everything about the VHDL code for the FPGA.  There's just not much wiggle room in how the logic is laid out on the device.

As regards the Agilent thread,  I'd seen it, but never bothered to read it.  As I suspected, it's almost entirely about enabling license options.  I would not,  and do not advocate hacking a $10k instrument.

The primary purpose of having people ask for the source is to make clear to the the low end Chinese T & M OEMs  that there is a marketing advantage to making their products "open source" and allowing the user community to write apps for the instruments.

Substantial parts of the UIs are written in a scripting language. Once one has the ability to run a script that reads button presses and sends commands to the FPGA and the ARM host, pretty much all the functionality of the unit becomes accessible.  That's not a very high bar.  The T & M OEMs are using the dev tools provided by the chip makers.  They are not developing all this stuff from scratch the way Keysight does.

If you're not interested in having an open source instrument, that's fine with me. I and other people are.  Unfortunately, you seem to have driven them away.

Your assertions amount to saying that the Chinese are able to clone competitor's instruments but no one else can even grasp how they work.  I don't recall that the Chinese invented the stuff they're making.  They just copied American products using American parts.
 

Offline ogden

  • Super Contributor
  • ***
  • Posts: 3731
  • Country: lv
Re: Open Source Oscilloscope, can it be done?
« Reply #132 on: December 15, 2017, 07:21:16 pm »
I was deliberately rude because your sole interest is trumpeting that you are right and everyone else is wrong.

Everyone else? I did not notice that somebody else is "trumpeting" getting GPL source code of instrument, just you.

What did you expect? - That everybody here will applaud "yes you are so right!" and that's it? Whole purpose of discussion is to discuss :)

Obviously I do not share naive optimism of yours: "It seems to me far more effective to get the GPL code released and fix the bugs", so I tell you that source code you can get such way (if any) will have just some irrelevant (to instrument function) code.

Quote
I suggest you read your own post.  I did *not* put words in your mouth.

I said "some irrelevant kernel code modifications", you read "GPL code is irrelevant".  What a stretch.  :palm:

Quote
You have no way of knowing what I do or do not know.

Obviously I can comment only about what you said here. And you were mistaken about many things.

Quote
Fact is I have repaired several analog scopes including a Tek 465.  I also have over 30 years of DSP experience.

What can I say - respect!

Regarding mine experience: while ago I was involved in design decision making of Linux-based device (not test instrument obviously) and can assure that Linksys router "incident" was isolated one. Other companies do not make such a stupid mistakes and oscilloscopes are not routers as well.

Quote
If you actually knew what a DSO is doing, you would not consider it very complex.  It's not.

Right.  :-DD

Quote
There are some significant timing challenges, but that and broadband circuit design are the only hard parts.  The timing constraints pretty much dictate everything about the VHDL code for the FPGA.  There's just not much wiggle room in how the logic is laid out on the device.

So you just explained what DSO is doing, right?

Quote
As regards the Agilent thread,  I'd seen it, but never bothered to read it.  As I suspected, it's almost entirely about enabling license options.

Sure it talks about enabling licenses, but this is not why I mentioned that thread. You missed that all the application code of particular instrument is binary Java code which do not fall under GPL. By requesting source code you would get tiny part, close to nothing of what you need to fix bugs.

Quote
The primary purpose of having people ask for the source is to make clear to the the low end Chinese T & M OEMs  that there is a marketing advantage to making their products "open source" and allowing the user community to write apps for the instruments.

Initially you said something completely different: "It seems to me far more effective to get the GPL code released and fix the bugs than it does to engineer a new instrument from scratch."

Quote
Substantial parts of the UIs are written in a scripting language. Once one has the ability to run a script that reads button presses and sends commands to the FPGA and the ARM host, pretty much all the functionality of the unit becomes accessible. That's not a very high bar.

What's the point of changing buttons if you cannot add or modify functions of instrument?

Quote
The T & M OEMs are using the dev tools provided by the chip makers.

Indeed they use dev tools provided by chip makers! If they use Xilinx FPGA - they for sure use Vivado suite. So what?

Quote
If you're not interested in having an open source instrument, that's fine with me. I and other people are.

This is good one  :popcorn:

I just say that asking source code of Linux based devices is dead end. You shall re-read thread to see that I am not against open source instrument/scope, also I have my own idea what open source scope shall look like.

Quote
Unfortunately, you seem to have driven them away.

Wow this is something. You blame me that others do not stand on your side. How convenient.

Quote
Your assertions amount to saying that the Chinese are able to clone competitor's instruments but no one else can even grasp how they work.

It's just you who do not know which functions of DSO is handled by FPGA and which - by CPU or scripting language :)

Quote
I don't recall that the Chinese invented the stuff they're making.  They just copied American products using American parts.

Seriously?  :-DD

With the same success we can say that Americans also did not invent stuff they are making. Stuff was invented by German inventor.
 

Offline zeqing

  • Regular Contributor
  • *
  • !
  • Posts: 84
  • Country: de
Re: Open Source Oscilloscope, can it be done?
« Reply #133 on: December 16, 2017, 02:16:43 am »
i have used the seeed DSO nan0https://www.seeedstudio.com/DSO-Nano-v3-p-1358.html for 4 years, and studied it's hardware and firmware(seems not designed by the seeed official but some pressional firmware engineers), almost meet my daily requests , with only about 60$(purchased in their special sales.)
 

Offline ez24

  • Super Contributor
  • ***
  • Posts: 3082
  • Country: us
  • L.D.A.
Re: Open Source Oscilloscope, can it be done?
« Reply #134 on: December 16, 2017, 03:05:27 am »
i have used the seeed DSO nan0https://www.seeedstudio.com/DSO-Nano-v3-p-1358.html for 4 years, and studied it's hardware and firmware(seems not designed by the seeed official but some pressional firmware engineers), almost meet my daily requests , with only about 60$(purchased in their special sales.)

Funny a lot cheaper on Amazon

https://www.amazon.com/seeed-studio-TOL01241P-Nano-Oscilloscope/dp/B00BB4ETJW/ref=sr_1_1?ie=UTF8&qid=1513393398&sr=8-1&keywords=dso+nano+v3

YouTube and Website Electronic Resources ------>  https://www.eevblog.com/forum/other-blog-specific/a/msg1341166/#msg1341166
 

Offline ogden

  • Super Contributor
  • ***
  • Posts: 3731
  • Country: lv
Re: Open Source Oscilloscope, can it be done?
« Reply #135 on: December 17, 2017, 02:00:19 pm »
Funny a lot cheaper on Amazon

https://www.amazon.com/seeed-studio-TOL01241P-Nano-Oscilloscope/dp/B00BB4ETJW/ref=sr_1_1?ie=UTF8&qid=1513393398&sr=8-1&keywords=dso+nano+v3

Funny that for same money you can get decent 2-channel 100MSPS/25MHz USB scope:

https://www.amazon.com/Owon-VDS1022-USB-Oscilloscope-100MS/dp/B00HC4KY2G

By adding 20$ you get isolated VDS1022I model.

Actually this scope is good candidate for hardware platform of "open source scope" we are talking about here in this thread. Thou I still believe that hardware shall be open source as well because nobody can tell how long any particular scope will be available.
 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 16510
  • Country: us
  • DavidH
Re: Open Source Oscilloscope, can it be done?
« Reply #136 on: December 17, 2017, 08:43:46 pm »
The problem with the proposals I have read here is that they duplicate functionality which is already available in low cost budget DSOs.  What about designing something which does what they cannot?

1. Include a simple function generator so the relatively low performance DSO may be used as a low frequency network analyser.  Keysight has taken a minor step in this direction but the performance of their solution is terrible; is this to prevent competition with their low frequency network analyzers?  There are a couple of USB DSOs which support this but not very well.

2. Include the noise marker function in the FFT capability.  I am tired of DSOs which cannot do this simple thing making them useless without a PC to do it for them and in that case, why is the DSO needed at all?

3. Design a real digital *sampling* oscilloscope for high bandwidth.  Yea, a simple design will *only* be 100 MSamples/second and perhaps slower but greater than 4 GHz of bandwidth with 10ps (100 GSamples/second) timing resolution is feasible.  Picotech makes some expensive USB oscilloscopes like this.
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #137 on: December 17, 2017, 10:33:44 pm »
I had mentioned the idea of implementing a VNA on an Instek MSO-2204EA when I was doing pre-purchase research.  I *think* they are pursuing the idea.  The marketing guy seemed quite interested.  It's easily doable for HF by dumping the data to a CSV file and using Octave.  When I saw that their PC scope software used Python I thought it would be easily scriptable.  Then I tried the software :-(  All it does is retrieve a screen dump.

However, if the SCPI facility works it's still scriptable, just more work.

My view is that what's needed most is the ability to fix the shortcomings of existing products.  I don't care if the HW is "open source" or not so long as it is documented and usable.  I want the ability to fix bugs and implement new features at will.  There's a nice SA facility in the Instek MDO line.  It works on all the GDS-2000E line, but is crippled by design so it only works on the MDO line.  However, it *does* work on the MSO2204EA with some clumsiness.

Take a look at the article in QEX #305 (Nov/Dec 2017) by DC9ST implementing an SDR using COTS eval boards.  At 250 MSa/s and 16 bits, the combination of a Zedboard and an AD9467-250 eval board is pretty potent. It  is most of the hardware for the sampling scope you mentioned for about $1k. For a 1920 point screen that would be 4000 wfms/s for a 4 GHz signal.

I plan to get a miniZed to play with, but I'm more interested in hacking a commercial DSO for the simple reason the hardware is cheaper and it's already in a convenient package.
 

Offline 2N3055

  • Super Contributor
  • ***
  • Posts: 6407
  • Country: hr
Re: Open Source Oscilloscope, can it be done?
« Reply #138 on: December 17, 2017, 11:28:21 pm »
Sampling scope is already done....
http://www.fastsampling.com/PHP/products.php

4 GHz for 300+ USD, 11 GHZ for a 1000 USD... Members of the forum here tested it and said it worked quite well..

I will probably spring for a 4GHz one in months to come.. I have  some work for it...


Also  @rhb , you should check Picoscopes... Full API and lots of examples...

Regards,

Sinisa



 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #139 on: December 17, 2017, 11:59:13 pm »
@2N3055

Quite a deal.  I'd never heard of them.  I don't happen to need one at the moment though.

I looked at the PicoTech scopes quite closely, but decided on the Instek MSO-2204EA instead.  Crappy FW, but the HW is quite good from what I can tell without a cal lab.  It *should* do everything I want via SCPI commands, but I've not gotten started on that yet.  I'm hoping I can find a way to modify the FW so I can write user apps for it.  Lots of stuff to learn about the Zynq platform.  But I'm an old hand at Unix and already know a fair bit about how the Instek works.

I've got some serious work setting up the development tools for the project.  I've got  disks on the way to run a RAID 0 mirror.  But I may bump up to a 5 disk RAIDZ2 (double parity ZFS) on OI Hipster.  I need to have a couple of VirtualBox  VMs running to access software.
 

Offline ogden

  • Super Contributor
  • ***
  • Posts: 3731
  • Country: lv
Re: Open Source Oscilloscope, can it be done?
« Reply #140 on: December 18, 2017, 10:53:59 am »
1. Include a simple function generator so the relatively low performance DSO may be used as a low frequency network analyser.  Keysight has taken a minor step in this direction but the performance of their solution is terrible; is this to prevent competition with their low frequency network analyzers?  There are a couple of USB DSOs which support this but not very well.

Rigol 1000Z series and Instek MSO-2000EA scopes are doing what you ask for, they both have 2-channel 25MHz AWG. Just add PC software and directional coupler(s).

Quote
2. Include the noise marker function in the FFT capability.  I am tired of DSOs which cannot do this simple thing making them useless without a PC to do it for them and in that case, why is the DSO needed at all?

Even more annoying is that FFT of all scopes (that I know) have fixed start frequency of the span: 0Hz.

All what's needed for scopes to be much more useful in frequency domain - DDC (Digital Down Converter) between ADC and FFT which is quite straightforward software function (numerical oscillator + mixer + decimating filter). Instead of having ultra slow 1 zillion points FFT handled by CPU, I would love to "zoom" into spectrum using 512 or 1024-point FFT which can be run in the FPGA having fast display update rate as spectrum analyzer shall do.
« Last Edit: December 18, 2017, 10:58:38 am by ogden »
 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 16510
  • Country: us
  • DavidH
Re: Open Source Oscilloscope, can it be done?
« Reply #141 on: December 18, 2017, 03:09:12 pm »
Sampling scope is already done....
http://www.fastsampling.com/PHP/products.php

4 GHz for 300+ USD, 11 GHZ for a 1000 USD... Members of the forum here tested it and said it worked quite well..

I will probably spring for a 4GHz one in months to come.. I have  some work for it...

I saw some details on these and while they operate like sampling oscilloscopes, they are not.  I kept things simple in my description but what I have in mind is a little more complex and does away with all of those trigger connections and difficulties on the Fastsampling products.

1. Include a simple function generator so the relatively low performance DSO may be used as a low frequency network analyser.  Keysight has taken a minor step in this direction but the performance of their solution is terrible; is this to prevent competition with their low frequency network analyzers?  There are a couple of USB DSOs which support this but not very well.

Rigol 1000Z series and Instek MSO-2000EA scopes are doing what you ask for, they both have 2-channel 25MHz AWG. Just add PC software and directional coupler(s).

That is the maddening thing about it.  These DSOs have all of the necessary hardware but the firmware lacks these features.  In some cases like FFTs, the firmware deliberately prevents it from being used this way by for instance discarding the phase results.

If the firmware could be rewritten this could be solved but that is never going to happen.

« Last Edit: December 18, 2017, 08:18:03 pm by David Hess »
 

Offline xani

  • Frequent Contributor
  • **
  • Posts: 400
Re: Open Source Oscilloscope, can it be done?
« Reply #142 on: December 18, 2017, 05:30:42 pm »

That is the maddening thing about it.  These DSOs have all of the necessary hardware but the firmware lacks these features.  In some cases like FFTs, the firmware deliberately prevents it from being used this way but for instance discarding the phase results.

If the firmware could be rewritten this could be solved but that is never going to happen.

Isn't most of the low end scopes just an FPGA(s) + some ARM CPU to do display/buttons/communication ? Writing oscilloscope software from scratch shouldn't be much harder than writing it for "fully open sourced" scope hardware (that would probably also end up being some FPGA + CPU or zynq).

I dont think rewriting it is the biggest problem, the problem is what if vendors says "fuck you" and blocks ability to flash custom firmware in new version/model and that would waste a ton of work even if hardware-independent parts could be salvaged to another project
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #143 on: December 18, 2017, 06:09:50 pm »
@ David Hess

Look at these and the following figures I posted.

https://www.eevblog.com/forum/testgear/feeltech-fy6600-60mhz-2-ch-vco-function-arbitrary-waveform-signal-generator/msg1360021/#msg1360021

@xani

The vendors are using the development tools provided by the FPGA OEMs. Yes, they *can* make life tedious, but I can't see anything they can do to stop people from loading custom FW.

A FW update consists of a pair of images, one of the filesystem for the ARM and one for the FPGA plus a few other odd bits.  These get copied to the appropriate places and the system reboots.  The programs to do the copying get copied to RAM disk for execution during the update.  The various OEMs have their own version of this, but the basic principle is the same.  It's what you *have* to do if you're using an embedded Unix based OS to run the instrument and you're going to provide FW updates.

At present the updates are not encrypted.  They *could*, but there is not a lot of CPU available.  So doing that would make updates *really* slow.  Moreover, the relative speed of a desktop machine with a GPU makes brute forcing the encryption key very simple, albeit perhaps slow.  But so what?  Let's say it takes a week to brute force the FW update key.  The OEM simply cannot use a key that is unique to each scope unless it's a hash of something like the serial number.  So once the key is cracked it does not need to be done again.

Which goes back to what I said some days back.  We *already* have open source scopes.  We just don't have the source.  But with the GPL part in hand from the instrument OEM, the rest is pretty straight forward.  A lot of the code can be used as is with or without the source.  There is no need to rewrite all the scope code before gaining control of the instrument.

If there are a large number of people who buy a scope *because* it is fully open source, the OEM is not going to should themselves in the head to prevent it.   My theory is that at least one of the OEMs will decide to exploit the marketing advantage.



 

Offline xani

  • Frequent Contributor
  • **
  • Posts: 400
Re: Open Source Oscilloscope, can it be done?
« Reply #144 on: December 19, 2017, 03:24:13 pm »

At present the updates are not encrypted.  They *could*, but there is not a lot of CPU available.  So doing that would make updates *really* slow.  Moreover, the relative speed of a desktop machine with a GPU makes brute forcing the encryption key very simple, albeit perhaps slow.  But so what?  Let's say it takes a week to brute force the FW update key.
Nowadays even tiny micros get hardware AES acceleration and you ain't bruteforcing AES128 anytime soon. So getting code out isn't that easy, if vendor really wanted to.

If you want to see some raw numbers here is some router chips vs SSL https://wiki.openwrt.org/doc/howto/benchmark.openssl but short version is that anything bigger than say Cortex M3 will probably get at least 1MB/s

Preventing getting code in is much harder (as if all things fail you can go to JTAG) so I doubt any would bother doing that as that would also make life harder for them.
Quote
The OEM simply cannot use a key that is unique to each scope unless it's a hash of something like the serial number.  So once the key is cracked it does not need to be done again.
Actually, they can generate firmware update per-device, pretty easily. What you do is.

* have a x509 certificate loaded on the scope
* manufacturer has a key for that certificate
* manufacturer makes a firmware image that has device's serial number embedded
* manufacturer signs it using his private key
* firmware updater checks for the signature and compares image's serial with device's
* if all matches, it proceeds with update.

even few hundred MHz ARM can do that verification in few seconds or less.

IIRC similiar method is used for license keys for switches (the ethernet kind)

Quote
If there are a large number of people who buy a scope *because* it is fully open source, the OEM is not going to should themselves in the head to prevent it.   My theory is that at least one of the OEMs will decide to exploit the marketing advantage.

That's probably the closest we're gonna get, the problem is that there would have to be "the first" fully hacked scope with fully open-source software that also is significantly better than base software. Which is a LOT of work. Also that cuts into their profits of selling software-unlocked features for way too much...

But having something like Digilent's Waveforms 2015 equivalent but on beefy hardware would be amazing
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #145 on: December 19, 2017, 04:08:27 pm »
It's not really necessary for it to be "fully open".  It just needs to be sufficiently open that it can be modified to add features and fix bugs.  Doing that *is* a good bit of work, but not as difficult as it might seem. If the FPGA bitstream is working properly it doesn't matter if the source is not available at the moment.

Take a look at FW update files for a few instruments. They're pretty simple and do the obvious thing.  Better yet, get a Zedboard and implement a FW updater.  The minZed is about $100.

I agree with your comments about encryption, but I stand by my original comment.  I am not an expert, but I learned the rudiments of cryptanalysis 50 years ago.  The principles have not changed.  But the nVidia and Radeon GPUs have radically changed how large a problem you can solve by brute force.  Bitcoin miners are doing this at large scale.

Rigol has sold a lot of DS1054Zs by allowing the license keys to be hacked.  Another OEM would get a lot of that market share if their scope were open.  I don't think they would be so stupid as to fight it.  Linksys restarted production of the WRT54G in response to the popularity of that device.

As I have commented previously, an important first step is to get the GPL'd  source code so we can build the infrastructure from source.  The OEMs are using one of several embedded Linux distros.  I *very* much doubt that any of the Zynq based scopes use anything other than the Xilinx tool chain.  Try selling "We're going to use a Zynq, but we don't want to use the Xilinx supported Linux distro for the Zynq." to your boss.
 

Offline xani

  • Frequent Contributor
  • **
  • Posts: 400
Re: Open Source Oscilloscope, can it be done?
« Reply #146 on: December 19, 2017, 05:47:18 pm »
It's not really necessary for it to be "fully open".  It just needs to be sufficiently open that it can be modified to add features and fix bugs.  Doing that *is* a good bit of work, but not as difficult as it might seem. If the FPGA bitstream is working properly it doesn't matter if the source is not available at the moment.

Take a look at FW update files for a few instruments. They're pretty simple and do the obvious thing.  Better yet, get a Zedboard and implement a FW updater.  The minZed is about $100.

Well unless you need FPGA to implement something extra (and you probably would want that pretty soon, like extra trigger modes or hardware-accelerated FFT). But yes, even having a fpga blob with documented communication would be a big step.

I was wondering about getting one of them, but while I have decent amount of Linux experience (I've built few Linux images from scratch incl. compiling all apps in system), I have absolutely no FPGA experience.

I agree with your comments about encryption, but I stand by my original comment.  I am not an expert, but I learned the rudiments of cryptanalysis 50 years ago.  The principles have not changed.  But the nVidia and Radeon GPUs have radically changed how large a problem you can solve by brute force.  Bitcoin miners are doing this at large scale.
That's not the same level. Unless there is a bug in AES, it would take a lot of time. Here is someone's that did actual math on it: https://www.reddit.com/r/theydidthemath/comments/1x50xl/time_and_energy_required_to_bruteforce_a_aes256/

TL;DR: if it's encrypted "correctly not happening. Even AES128 would take bilion years, assuming crypto itself wouldn't be broken

Quote
Rigol has sold a lot of DS1054Zs by allowing the license keys to be hacked.  Another OEM would get a lot of that market share if their scope were open.  I don't think they would be so stupid as to fight it.  Linksys restarted production of the WRT54G in response to the popularity of that device.

Yes, if we had OS software at that level, I wouldn't be even suprised if vendors customized and installed it on their own scopes, as that would basically be free R&D for them and advertisement.

As for Rigol I do not think they "allowed" it to be cracked, rather saw  it and said"oh, they are cracking them! But hey, actually that's not bad, people buy us over competition now".

I have noticed that in many fields there is a lot of fear for open-sourcing things(latest example: Dave's new multimeter ), companies being afraid that it would "aid their competition" or some other bogus reason, even tho 95% of the code is just repeating what someone's else invented decades ago but with different bugs.

Quote
As I have commented previously, an important first step is to get the GPL'd  source code so we can build the infrastructure from source.  The OEMs are using one of several embedded Linux distros.  I *very* much doubt that any of the Zynq based scopes use anything other than the Xilinx tool chain.  Try selling "We're going to use a Zynq, but we don't want to use the Xilinx supported Linux distro for the Zynq." to your boss.

The problem is that selling "hey, you know that code we put 10 man-years into ? Let's give it away for free" to your boss is even harder. Even if long term option would be "customers buy our scopes instead of 5x as expensive LeCroys because our software capabilities are same".

Because now mid-high end scopes seem to be more about software than actual hardware. Sure you still need to get that part right, but no technology gonna save scope with bad firmware.
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #147 on: December 19, 2017, 08:42:22 pm »
I've got lots of Unix experience, though relatively little of it with Linux.  And *no* FPGA experience.  But being an old R&D rat, that's not much concern.  In fact it's an attraction.  I haven't done that.  The things I dislike are the ones I've done 3-4 times already.  These days I only do those for money or necessity.

There's brute force and there's brute force.  Don't underestimate the impact of known plaintext.   More interestingly, David Donoho proved that L1 == L0 under certain rather general and highly probable conditions.  That's *extremely* important.  It's a proof that linear programming can solve an NP-Hard problem in tractable time.  I don't know how to apply it to a cryptanalysis problem at the moment, but one might be able to. The only thing that makes *any* modern crypto system secure is that the problem of finding the key is NP-Hard.  I'm rather amazed that this was done in 2004 and I did not learn of it until 2013. And I've still never seen mention of it in a CS & crypto context.

Vendors like R&S raised the bar to unauthorized licenses.  Rigol and apparently also Siglent have elected not to.  They could *very* easily change the algorithm for the license keys.

I'm *not* and never have suggested the OEMs should give away their proprietary code.  I am simply saying that the first step to an open source instrument is to get the GPL'd code from the scopes on the market already.  To paraphrase Lao Tsu, the journey from Shenzen to Santa Clara begins with a single step.

I'm also saying that analyzing FW updates will provide a lot of information needed to build an open source alternative.  I am working on collecting some of that information.  It will go faster if other people also work on the problem.  Arguing about whether it is possible is useless.  It's obviously possible.  It just takes sufficient resources.
 

Offline Marco

  • Super Contributor
  • ***
  • Posts: 6686
  • Country: nl
Re: Open Source Oscilloscope, can it be done?
« Reply #148 on: December 22, 2017, 01:08:16 pm »
Reverse engineering the software on an oscilloscope to improve it is obviously possible, it's been done. But it takes someone very far out on the spectrum donating 10s to 100s of thousands worth of his time. Stuff like this is never a team effort at the start, it's one lone mad man getting something to a workable level.

The source code of the OS doesn't really help, because with GPL V2 they don't have to provide you the tools and means to get it running. You can generally hack shell access, which is infinitely more useful than the source code of the OS ... the OS doesn't need to be improved after all, you can leave it entirely alone. All the important functionality is in the drivers and applications.
 

Offline rhb

  • Super Contributor
  • ***
  • Posts: 3476
  • Country: us
Re: Open Source Oscilloscope, can it be done?
« Reply #149 on: December 22, 2017, 03:03:19 pm »
Actually writing code is probably largely a solo effort at the start.  However, there is the ARM code and the FPGA code, so two people is probably ideal. But there is a lot of information to be gathered.  For example:

Block diagrams and major components
Existing menu structure and modules
Datasheets for key components
Chip vendor code examples and libraries

All of that takes time.  Right now all of my time is being consumed by trying to get guest OSes to load on VirtualBox which is throwing a hissy fit for no obvious reason.  Once that is done I can get back to the task of peeling apart update images to see how the updates are handled and identify the components of the scope application.  For me the first task is to reenable ssh.

Your estimate of the cost in labor is pretty accurate, but there is no work for old oil & gas R&D types, and I'm *really* annoyed by the UIs.

There is also the not small matter of conveying to the OEMs that an open source instrument is desired by many people.

I started a job once and discovered I had 1.25 million lines of code to support, 500,000 lines of which the only comment was the author's name.  It took 6 weeks to sort out the system so it would compile.  The authors still worked there, but they had no real idea how to compile the mess.  Before I could fix *any* bugs I had to be able to compile and link everything.

The first major step is to be able to build a working image from the available source.  That will not be everything.  Some pieces will just be binary blobs that have to be put in the right places.  But that *is* a major milestone for which the GPLd parts are essential.  One might or might not be able to do that using the chip OEM distro.  A few changes by the instrument OEM can translate into many hours of work.  Once one has a buildable code base, then one can start the task of replacing the binary only parts.  But until you have the GPL'd code base in buildable form you cannot do *any* work on the FPGA and UI code.

How do you swallow an elephant?  One mouthful at a time.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf