Hacking the Rigol DHO800/900 Scope

[norbert.kiszka]

It's ready! Time to run some tests and prepare for the release.
(Attachment Link)

This looks similar to mine oldest release v0.0.1, beside of the added triggers from the DHO4000, linear interpolation and no UPA at that time.

Humorously in kindergarten, some teachers would say "Sharing is caring". It's nice to see many people are sharing how they made it, not just showing off. "mrisco" not only shared his process but also shares the methodology of doing it which is very valuable.

This is one of many reasons why opensource is usually the best choice. Even with the bad code, somebody can modify it and publish it further. Like a DSRemote which I ported for the DHO800/900 and somebody on this forum modified it to make it work better. However I didn't tested his modification yet.

Until now, no one joined the public modification of the firmware somewhere such as GitHub or so

I have published some work on my GitHub and I have plans to do more.

With my plans to rewrite, it primarily will work on a Orange Rigol system, which is fully open source (modified Debian which uses original Debian repositories) and it's better than Android (which probably will be supported much later). I need to port more drivers for the Rigol things, because now we have about the half of it - AFE, FPGA, touchscreen and right now I don't even remember what else I did.

Nobody will have to be familiar with the Linux, because it's intuitive GUI called Mate. There a lot of myth around Linux being hard etc. How hard is to flash SD card with a image from a single file? Turn it on and You have similar GUI scope app in front of Your eyes. Fighting with those myths is very hard.

As we know the original source code for the App was not published. The reverse engineering takes time. norbert.kiszka said he might can do a new app. I think this confidence is from his findings of the API libs he found from the firmware. But it would take a lot of time for 1 person. If more people can participate, the process could be faster. Some people can do the API reverse engineering and publish API libs for Linux or Android, some can do the UI modification, some can do the testing and someone can even make a small pull-request for a small modification. The hardware is not too slow. It's just the firmware is just too buggy and inefficient. The extra hardware resources SHOULD BE used for user-defined scientific calculations.

Making everything from scratch is insanely time consuming. That's why my plan is to take .so lib and link it with the code written in C. Mostly I will have to rewrite Java part, which is UI mostly.

As I said couple times, I want to make a open source UI based on HTML+CSS+JS eventually plus any scripting language like a PHP or Python (I have many years of experience with PHP and making PHP code to have good performance). Even some settings management.

Waveform will be working as a video stream (just like Odysee or YouTube), so this will be efficient.

There is a one open source browser engine with a good enough license, which I can use to make it visible and usable like a regular app instead of web browser.

I don't have any plans to rewrite FPGA firmware (there are some problems with the stock one), because my knowledge around FGPAs is weak. Maybe I will come back with my tries to do some small hacks (in FPGA), but likely nothing more with it. Eventually somebody else with my help, because of my knowledge about the communication between the app and the FPGA.

Anyway, when I published months of my work as a open source, a lot of people was using it, but literally nobody supported it to make it going further. I went aboard for a quite long time and later I decided to do what Im doing right now.

(Fully) open source is much better, but most people can't do things for free for 60+ hours per week. A lot of code on GitHub (including my own s***t) is done after hours. Beside of a big open source projects (Linux, Debian, BSD, KDE, Mate and much much more) supported by a big corporations, which they have huge interest in it.

Coming back to my rewriting, I think to start it after releasing v0.4.2. In the meantime likely I will do about 1-3 more releases of this and I will switch completely to a new app, when it will be good enough for the normal usage.

I almost forgot to mention of possibility to do a switch 1.25 -> 2 GSa/s for the price of limited memory depth. I did it already in my mod as a hardcoded change, but to make it usable without source code is not easy. After mentioned partially rewriting, this will be much much easier to do.

If more people can participate, the process could be faster.

With the documentation for the API for the open source UI, that will be more than possible.

I think this confidence is from his findings of the API libs he found from the firmware.

A lot is going on at the app init, but at the beginning it will stay in the modified .so lib, so I will save a lot of time.

Later I can rewrite this too, to make it boot faster for a couple seconds. Five second per day can save 30 minutes per year.

API libs for Linux or Android

With the web based UI, there will be no need to do much. HTML, CSS and JS are parsed by a browser. Maybe somebody will like to do a "browser" by using existing code (browser engine), to make it work fast and on fullscreen (without typical web browser controls).

The hardware is not too slow. It's just the firmware is just too buggy and inefficient.

I would say it's way worse. With my changes on a Assembly level (done manually), some app parts are about 5 times faster. Those things can be even much faster after rewrite.

5 times faster, is not because Im good or something like that. It's because Rigol did everything to just make it work, because people will buy it anyway. They didn't said anything about performance or a increadible amount of bugs in their advertisements...

Current UI (Java and Android layouts) and Android itself are the reasons why I didn't tried to enable the dead code for the zone trigger. Implementing new things in the existing UI is a huge pain in the ...

And yet I hope the Corp. can publish more documents which can help not only itself for better reputation and sales but also buyers. Otherwise there are so many alternatives. Once people lose interests, people won't buy.

Open source firmware and hardware documentation increases sales a lot. But greedy companies knows "better".

Same thing with the piracy. Piracy increases sales, because this is a free advertisement. But most companies instead of making more or better products, they fight piracy instead (even Keysight with a copyright on their scopes documentation...), which in the end hurts legal users the most.

I also implemented license system, which is a nothing else than a copy protection (only one time thing, in the opposite of what some people think about it). I had other reasons than sales to do this. And exactly as I expected, after release with this thing implemented, sales went down very quickly. But there are other more important things than money.

Maybe make it return true only for valid cases, not always

That's the thinking I don't like. Maybe add a 10 another functions to make it 10 times slower than it is originally? Or maybe call a even slower script because it's easier?

In my mod, this function is not even called at all. I removed all calls to all Rigol license system functions for the better performance. Long time ago.

BTW. Who want's to have overclocked AFG to 250 MSa/s? This will give max frequency 80 MHz instead of 50 MHz as it is right now. Only reason why I didn't release it yet, is because of the calculations of the user input (frequency set in a multiple places), which I need to do.

[kd7eir]

I also implemented license system, which is a nothing else than a copy protection (only one time thing, in the opposite of what some people think about it). I had other reasons than sales to do this. And exactly as I expected, after release with this thing implemented, sales went down very quickly. But there are other more important things than money.

The licensing system didn't bother me at all. I did make a foolish mistake by not reading a post thoroughly and thinking I would need to repurchase a new release.

BTW. Who want's to have overclocked AFG to 250 MSa/s? This will give max frequency 80 MHz instead of 50 MHz as it is right now. Only reason why I didn't release it yet, is because of the calculations of the user input (frequency set in a multiple places), which I need to do.


I would be interested in this as I am in the process of building the AFG from here: https://www.eevblog.com/forum/testgear/awg-function-generator-for-dho800-900-series-oscilloscopes/

[kd7eir]

I also implemented license system, which is a nothing else than a copy protection (only one time thing, in the opposite of what some people think about it). I had other reasons than sales to do this. And exactly as I expected, after release with this thing implemented, sales went down very quickly. But there are other more important things than money.

The licensing system didn't bother me at all. I did make a foolish mistake by not reading a post thoroughly and thinking I would need to repurchase a new release.

BTW. Who want's to have overclocked AFG to 250 MSa/s? This will give max frequency 80 MHz instead of 50 MHz as it is right now. Only reason why I didn't release it yet, is because of the calculations of the user input (frequency set in a multiple places), which I need to do.

I would be interested in this as I am in the process of building the AFG from here: https://www.eevblog.com/forum/testgear/awg-function-generator-for-dho800-900-series-oscilloscopes/

[norbert.kiszka]

I would be interested in this as I am in the process of building the AFG from here: https://www.eevblog.com/forum/testgear/awg-function-generator-for-dho800-900-series-oscilloscopes/

Im not sure how it will behave with the 250 MHz input data and 80 MHz output. In the datasheet of DAC IC there is: "210 MSPS", so there is a chance.

My DHO924S with the original and unmodified (yet  ) AFG works properly with those parameters, beside of the lower amplitude (output opamps and filters).

[gabiz_ro]

A binary patch can only change one byte for another, so we are restricted in terms of the changes we can make. There are, of course, techniques for code injection, but they are more complicated and perhaps not worth the effort. I have shown that the necessary adjustments can be made at this entry level using only simple binary patches.

Regards.

Indeed but you can use it many times.
I don't see entire function but I think is more complex and space required is enough  to fit new code.
Maybe I don't explain it right.
For example
Function that check licence is on area that is 280bytes you write some code in assembly that if is a valid option return true otherwise return false. That result in 148 bytes size code
Then patch 148 bytes
Function is more simple instead of doing all stuff just return true if is asked for valid option

@norbert.kiszka
That discussion is about patching original firmware, not like your case when you rebuild it or make it from scratch.
I don't see how replacing a function with one much simpler result in add a 10 another functions to make it 10 times slower than it is originally

[Fungus]

In Ghidra is more easy!! you can see the offsets, the opcode the assembly and the c++ code at the same time!

Ghidra makes it easier to understand the code and find things in it but it doesn't make it easy to modify it.

Not at all.

[norbert.kiszka]

That discussion is about patching original firmware, not like your case when you rebuild it or make it from scratch.

Actually it was more discussions in single thread.

I don't see how replacing a function with one much simpler result in add a 10 another functions to make it 10 times slower than it is originally

This was a exaggeration to emphasize a fact. Most of today software engineers just doesn't care about software performance. They just add new code without thinking if that will be very fast or very slow to execute, or easy to maintain in the future. In the stock app, exporting (to a file) and importing settings takes around 5 seconds. After my ~15 minutes of work it does the same job in one second - "only" 5 times faster.

Ghidra makes it easier to understand the code and find things in it but it doesn't make it easy to modify it.

100% agree.

On top of it, there is a built in decompiler which often generates pseudocode with a completely different logic than it is in reality. So every use must be done with caution.

This is one of the reasons why I want to make partial rewrite of it. Instead of doing extremely slow modification of the compiled binary, I will be able to make one or more functions in the separate C code linked with it.

As I said, C, not this C++ "rubbish". C++ code is much slower than same thing done in C.

Try to do a simple hello world (print simple message on the screen and nothing else) in both and compare it in Ghidra or measure time needed to execute. Difference between those is huge. Also C code is easier to maintain, because there is less unnecessary but "fancy" things.

Yes, you need to use the binary file editor of your preference. In Windows, Winhex for example. But I prefer to use a Python script, for easy documentation and automatization.

In this way it's even slower than doing it in Ghidra. Many times slower.

[Fungus]

C++ code is much slower than same thing done in C.

That's fighting words...

[norbert.kiszka]

C++ code is much slower than same thing done in C.

That's fighting words...

C++ and C are not the same exact language. Compiled binaries are extremely different in many cases (sometimes are identical, but only sometimes).

[Fungus]

C++ code is much slower than same thing done in C.

That's fighting words...

C++ and C are not the same exact language. Compiled binaries are extremely different in many cases (sometimes are identical, but only sometimes).

Oh, you men hacking C vs. hacking C++.

Yeah, C++ will be messier.

[norbert.kiszka]

C++ code is much slower than same thing done in C.

That's fighting words...

C++ and C are not the same exact language. Compiled binaries are extremely different in many cases (sometimes are identical, but only sometimes).

Oh, you men hacking C vs. hacking C++.

Yeah, C++ will be messier.

Yes, C++ is messy, but what I mean: generated executable binaries are much slower at doing stuff when those are executed and not the best for maintenance (source code).

I hope You understand now what I tried to say.

[mrisco]

To add F-Droid to Actions panel:

Code: (json) [Select]

    {
      "glyph": "fdroid.png",
      "caption": "F-Droid",
      "type": "APP",
      "command": "org.fdroid.fdroid"
    },

Icon:

[gerbay]

C++ code is much slower than same thing done in C.

That's fighting words...

C++ and C are not the same exact language. Compiled binaries are extremely different in many cases (sometimes are identical, but only sometimes).

Oh, you men hacking C vs. hacking C++.

Yeah, C++ will be messier.

Yes, C++ is messy, but what I mean: generated executable binaries are much slower at doing stuff when those are executed and not the best for maintenance (source code).

I hope You understand now what I tried to say.

What I understand from what you said is that you do not know C++ well.    I don't want to argue.

[norbert.kiszka]

C++ code is much slower than same thing done in C.

That's fighting words...

C++ and C are not the same exact language. Compiled binaries are extremely different in many cases (sometimes are identical, but only sometimes).

Oh, you men hacking C vs. hacking C++.

Yeah, C++ will be messier.

Yes, C++ is messy, but what I mean: generated executable binaries are much slower at doing stuff when those are executed and not the best for maintenance (source code).

I hope You understand now what I tried to say.

What I understand from what you said is that you do not know C++ well.    I don't want to argue.

This is what I hear quite often from a people experienced in C++ but not so much in C.

Make a some code and compare performance.

*nix operating systems (BSD, Linux, MacOS and others) are written in C and some small parts in Assembly (either hardware stuff or something used very often). You can read/hear developers why they don't want to see any C++ code.

C++ developers like to say that in C You can't call functions by using pointers, which is nothing else than another myth. Everything written in C++ can be implemented in C.

But if You do a bad code, it will be slow - no matter what language You will chose.

Sometimes same thing is with the Java developers - about half of them say that Java is the fastest. Scripting (VM) languages never was the fastest, quite the opposite actually.

[Fungus]

Everything written in C++ can be implemented in C.

It just takes much, much longer to do.

My answer to C programmers is to ask them why they don't do everything in assembly language?

Whatever their answer is I then say, "and that's the reason I use C++ and not C".

[norbert.kiszka]

Coming back to hacking, I published .GEL file to revert both app and the OS changes done by my mod (any version).

About 5-10% users has to make warranty claim due to hardware issues.

[norbert.kiszka]

Everything written in C++ can be implemented in C.

It just takes much, much longer to do.

My answer to C programmers is to ask them why they don't do everything in assembly language?

Whatever their answer is I then say, "and that's the reason I use C++ and not C".

Whenever I want to do something much faster and I don't care about execution time, Im using scripting language (especially bash which allows to do a lot of stuff just by simple and short one line of code).

I think people learn C++ as the only one language and they are fixated to be the best language. While they never wrote anything in other languages, maybe beside of the simple hello world.

In C++ making one function or method into multiple functions/methods just to handle different variable types is crazy and bad - it doesn't make anything to be faster.

In the Rigol app, even simplest getters are in the separate functions (or methods if You prefer, but after compiling this is a function). Their great C++ developers never heard about possibility to enable compiler optimizations.

Right now in the stock app, in many places if the code wants to read 4 or 8 bytes into single CPU register, it calls thunk function first, which reads pointer to a function from a memory, then executes it (BR in Assembly). Instead of doing it in around 1 ns, it can be like 15 ns. Stock app does less that 30 k waveform updates per second. With my changes it goes up to 108 k. Use Ghidra if You don't believe me.

[gerbay]

You know C++ at an introductory (or maybe intermediate) level.

[norbert.kiszka]

You know C++ at an introductory (or maybe intermediate) level.

My knowledge level about C++ doesn't change the fact that is slow as hell. Rigol scope app is a great proof of this fact. This is not only my own words.

[ejeffrey]

C++ with a good implementation is probably the highest performance general purpose language in common use.  Nobody who understands programming or performance really disagrees with this.  If you believe otherwise you are simply wrong.

It's not 100% of course.  C can be slightly faster in a few situations.  As can Rust.   Generally they are all close enough the performance differences are rarely important.

That's not to say they are always the best choice, there are many other important factors.

C++ does depend a lot more on the quality of the implementation for performance than simpler languages like C, but all three major implementations are pretty good.

Generally if a program written in any systems language is too slow the problem that the program is written badly.  That is of course is possible in any language.  C++ is probably middle of the road in terms of making it easy to accidentally do something much slower than you intended.

[norbert.kiszka]

C can be slightly faster in a few situations.

Text (string) situations are not rare. In this case C++ is extremely slower than C.

Generally if a program written in any systems language is too slow the problem that the program is written badly.  That is of course is possible in any language.

Correct. Also Dunning–Kruger effect is another factor in every language.

Speaking of the bad code, there are a lot of people using too much try-catch blocks and other C++ things, which ends up as a much slower app than it should be.

[ejeffrey]

C can be slightly faster in a few situations.

Text (string) situations are not rare. In this case C++ is extremely slower than C.

It's not.  You are just poorly informed.  In many ways C++ string handling is faster because the length is stored explicitly which is for many operations much faster than null termination.

[norbert.kiszka]

C can be slightly faster in a few situations.

Text (string) situations are not rare. In this case C++ is extremely slower than C.

It's not.  You are just poorly informed.  In many ways C++ string handling is faster because the length is stored explicitly which is for many operations much faster than null termination.

1. In C You can use structs and store length if You often need this information.
2. C++ has it's own text format. That requires to do a lot of conversions between ASCII and C++. In the Rigol firmware this is done a lot at the init, which takes literally a couple seconds and later when app is running, multiple times per second, slowing it down.

[ejeffrey]

Are you talking about Unicode?  That has nothing to do with C vs C++.

[norbert.kiszka]

Are you talking about Unicode?  That has nothing to do with C vs C++.

No. C++ has its own format to store and handle text strings.