EEVblog Electronics Community Forum

Products => Test Equipment => Topic started by: NorthGuy on November 28, 2016, 04:27:20 pm

Title: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on November 28, 2016, 04:27:20 pm
I am interested in deep memory of Rigol scopes. My idea is to do a single acquisition of full memory range, download the data over the Ethernet (LXI) to my computer and then analyze the data. DS2072A has 56Mbytes, and DS4014 has 280Mbytes. It looks fantastic, but then I thought that downloading through the Ethernet may take really long time, so it might not be as useful as I imagine.

I don't know if the Ethernet transfer rates have ever been measured for these scopes. If anyone has any measurements done on the speed of transfers, would you please share the data.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on November 28, 2016, 06:36:08 pm
I have had some LAN speed issues that you might want to know, but it was because of the Windows OS, not the scope itself. There were no LAN speed issues with Linux.
https://hackaday.io/project/5807-driverless-rigol-ds1054z-screen-capture-over-lan/log/18807-microsoft-bullied-tcp-rfcs-the-result-a-painfully-slow-lan-on-windows-and-a-wasted-weekend

For DS1000Z series, it will take tens of seconds with Linux and many minutes with Windows in order to download 24 million data points. Also, when you download the raw data, the scope must be in STOP mode and you need to download multiple data batches to extract the whole waveform. If you want to download in real time, it won't work.

I didn't test the models you specified.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on November 29, 2016, 04:53:13 am
I have had some LAN speed issues that you might want to know, but it was because of the Windows OS, not the scope itself.

TCP window is not a parameter. This is the size of the remaining buffer on the receiving side - to make sure the sender doesn't send too much data. It depends on the total size of the receiving buffer as well as the sizes and speed of your reads. If you read in awkward sizes or if you don't read often enough the window will shrink. So, to get the best speed you need to read fast and in big chunks.

It is also possible to set the size of the receiving buffer with setsockopt() SO_RCVBUF to make the buffer substantially bigger (although I've never tried doing this). The buffer could be made big enough to fit all the data at once.

10-s of seconds (e.g. 30 sec) for 24Mbytes is less than 1Mbyte/sec - rather slow for 100Mbit Ethernet connection. Is this number for binary data or for ASCII?
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on November 29, 2016, 06:42:19 am
TCP Receive buffer and TCP Receive Window (AKA RWIN), is not the same thing. RWIN is the amount of data that can be received without ACK, and it is gradually increasing in a normal TCP connection. Microsoft decided to implement it's own "RWIN Auto-Tuning" algorithm for RWIN, disregarding how the rest of the world is doing the same thing. I am not the only one who got into trouble because of this, just google "RWIN Microsoft", e.g. the first Google result https://www.duckware.com/blog/how-windows-is-killing-internet-download-speeds/index.html (https://www.duckware.com/blog/how-windows-is-killing-internet-download-speeds/index.html)

My tests were made using a Telnet connection from Python 2.7, first with the native Win10 OS, and second with a Debian inside a VMware running on the same Win10 OS. A Rigol DS1054Z oscilloscope was the server. Linux transfer time was way more faster then Windows. Same results when opening a TCP socket from Python 2.7. Same results when using NetCat without Python.

To answer your question, we were talking about 24 Mbytes of binary data sent over a 100 Mbps TCP connection.
Please try it yourself and let me know your results.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: 2N3055 on November 29, 2016, 09:59:52 am
Those TCP parameters should have no influence on speed on a local network with latencies of less than 1ms..
I have transfer speeds in range of 100Mb/sec on a 1Gbit LAN between mixed hosts..

These Receive buffer and window problems are old news, on Vista and early Win7, when running on fast WAN with lots of bandwidth and relatively high latencies.. I has been patched since...

I'm not saying you're wrong, just it is a bit unusual..

I presume you are doing lots of small block transfers? Maybe buffering, maybe something in Python implementation on windows.. like shutil.copy slow on windows but working fine on linux...

I will look into it these days to see if I can find out something..
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: thm_w on November 29, 2016, 10:35:25 am
Did you see the software from marmad?
https://www.eevblog.com/forum/projects/software-tips-and-tricks-for-rigol-ds200040006000-ultravision-dsos/ (https://www.eevblog.com/forum/projects/software-tips-and-tricks-for-rigol-ds200040006000-ultravision-dsos/)

Quote
My software will connect with the scope using either USB or TCPIP VISA resources. If you're planning to save many recorded frames, it's best to use USB (USB: ~2.5s = ~6s transfer time on TCPIP). The VISA resource locator is not coded with a sniffer for finding TCPIP connections - so it's default behavior is as follows:
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Dwaine on November 29, 2016, 03:35:47 pm
Running my DS1054z by LAN Connection under Fedora 25 with DSRemote.  The screen updates every second.  I was actually quite surprised.  No slow slow downs at all.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on November 29, 2016, 08:55:05 pm
I presume you are doing lots of small block transfers? Maybe buffering, maybe something in Python implementation on windows..

The DS1054Z oscilloscope can not transfer all 24 million data points at once. The programming Guide of the scope recommend maximum 250 000 data points at once, so 24 millions need 96 distinct TCP transfers. Before each of the 96 transfer, 2 SCPI commands for setting setting the start and the stop data point are sent:
Code: [Select]
:WAV:STAR 1
:WAV:STOP 250000
:WAV:DATA?
- here we receive the first 250 000 data points (plus a few bytes of header)
:WAV:STAR 250001
:WAV:STOP 500000
:WAV:DATA?
- here we receive the next 250 000 data points (plus a few bytes of header) and so on, until all 24 000 000 bytes are transfered.
The same speed problems was seen with NetCat, AKA 'nc', a very old tool (before Python) from Unix, also ported for Linux and Windows. I didn't checked the sources for NetCat, but I guess it's written in C, not Python.

RWIN algorithm guilty was my conclusion after looking at the TCP packets with WireShark. Maybe it's not from RWIN algorithm, maybe is something else.

The problem is still present today, in the latest Win10 updated this month, so if you have the opportunity to investigate this issue with a Rigol scope, please do it.



Running my DS1054z by LAN Connection under Fedora 25 with DSRemote.  The screen updates every second.  I was actually quite surprised.  No slow slow downs at all.

To update a screen trace, only 1200 data points are necessary. The OP want to transfer the RAW data for all 24000000 data points.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: 2N3055 on November 29, 2016, 10:37:39 pm
Thanks for details, as soon as I get time I'll look into it... Have a good one.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on November 30, 2016, 01:48:11 am
@RoGeorge: The "Windows" problem is probably unrelated to TCP window. The article you referred to is talking about speeds which are at least order of magnitude faster than the speed you get with Linux.

I cannot try any of this because I don't have any of these scopes. I started this thread hoping to get speed information so I could better choose which one (if any) to buy.

If you wish, I can write a simple test program in C which reads the data in big chunks with minimum overhead, and then you can run it with yours scope.

@thm_w: This is an interesting thread. Took me a while to sift through :) The 6s number is for a screenshot and suggests that the transfer speed is 0.2 MBytes/sec, which is horrible and even much worse than the data speed quoted by RoGeorge. But they use Rigol software and VISA (don't know what it is), so it is likely that there's a lot of overbloat and speed lost. Also, the speed of sending screenshots may be drastically different from the data speed.

The data speed is uncertain. Looks like Marmad uses ASCII transfers for data. In his post #243 he quotes:

:WAV:FORM ASC

instead of

:WAV:FORM BYTE

This not only increases the number of transfer bytes several times, but also burdens the scope because everything needs to be scaled and convert it to the text.

The only real time data that I could find is for 140-byte transfers for the screen data (in post #455 and then forward). They get 8ms per 140 data points on DS2072A, but this includes sending a command and waiting for the answer, and it most likely ASCII. So, it is not much you can infer from this.

BTW: They've got much worse and uneven speeds for DS1054Z.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on November 30, 2016, 02:03:31 am
If you wish, I can write a simple test program in C which reads the data in big chunks with minimum overhead, and then you can run it with yours scope.

Sure, I will gladly test your code with my scope and publish here the transfer time for both Linux and Windows.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Karel on November 30, 2016, 03:12:13 am
With the DS1000Z series it takes 55 seconds to download 24Mpts via usbtmc. Via ethernet it is more or less the same.
The bottleneck is the scope, not the connection.
Data needs to be downloaded in chunks of max 250,000 bytes.
Between those chunks there's a long delay in the response of the scope's firmware.
Why? Only Rigol knows...

Edit: I did a new test with the latest firmware (the one Rigol removed) and now the download time is 33 seconds for 24Mpts.


Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on November 30, 2016, 06:35:00 am
Sure, I will gladly test your code with my scope and publish here the transfer time for both Linux and Windows.

Here it is. The program is called ds1064z (the Windows one has exe on the end, the Linux one will need execution permission set). Both need to be run from the terminal/command line. They'll print the amount of data received. They do this:

:STOP
:WAV:SOUR CHANnel1
:WAV:MODE RAW
:WAV:FORM BYTE
:WAV:STAR 1
:WAV:STOP 250000
:WAV:DATA?
...
:WAV:STAR 250001
:WAV:STOP 500000
:WAV:DATA?
...

etc. so the data must be acquaried on CH1 with memory depth of 24Mpts.

Of course, I don't know if they work correctly or if they're capable of getting data from the real scope (for example I didn't know whether to use \r or \r\n as the line ending (used \n).  I tested it with ds1054zs.exe (included in the zip), which pretends to be Rigol, but is really dumb - it only outputs 250000 of random data when it sees ":WAV:DATA?" and does nothing else. If you want to use it, you'll need to open port 5555 in your firewall. The transfer takes 2s either on Linux or Windows. So, there doesn't seem to be any problem with the connection speed. Rigol, certainly won't be that fast.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on November 30, 2016, 09:42:57 am
Thank you. May I have the sources and compile them myself, please? I don't want to run a binary from a forum.

If the sources are not for publishing, it's OK, but you need to wait until Friday, when I will have time to prepare a clean machine isolated from the rest of the LAN.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on November 30, 2016, 10:25:27 am
I don't want to run a binary from a forum.

That's a good idea. I think the same way.

If the sources are not for publishing, it's OK, but you need to wait until Friday, when I will have time to prepare a clean machine isolated from the rest of the LAN.

I wrote it in Pascal because I had a cross-compiler. And it uses few of my private libraries which I cannot give out. So, you won't be able to compile it anyway. I should've used C :(

I guess we'll have to wait until Friday. I predict it'll be the same speed as Karel reported 2 posts back.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 01, 2016, 03:36:08 am
Found a clean Debian machine (don't have a clean win10 machine yet) and this is the result:

Code: [Select]
rogeorge@debian80:~/Downloads/tbt$ ./ds1054z
Usage: ds1054z ip.address.of.the.scope
rogeorge@debian80:~/Downloads/tbt$ ./ds1054z 192.168.1.3
Connected
Received 7272 bytes in 0.634 sec.
rogeorge@debian80:~/Downloads/tbt$

While running this, the screen of the scope displays the text message "Invalid Input!".

Looking inside the binary with IDA, it seems like the program is sending all the SCPI commands at once, with \n between them. This can not work. The client must wait for the instrument reply and, if necessary, to interrogate the instrument for its status. There is a state machine described in the SCPI protocol.

Please read the documentation for the SCPI protocol. SCPI is now part of the IVI Foundation. IVI specifications are not free, but SCPI specs are free to download.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 01, 2016, 05:27:48 am
This can not work. The client must wait for the instrument reply and, if necessary, to interrogate the instrument for its status. There is a state machine described in the SCPI protocol.

I see. I've read the DSxxx programming scpecifications and it looked to me as request-response protocol over TCP - server fetches commands from the stream one by one and executes them consecutively (such as in FTP, SMTP, POP3 and many other Internet protocols).

Please read the documentation for the SCPI protocol. SCPI is now part of the IVI Foundation. IVI specifications are not free, but SCPI specs are free to download.

Ok. I downloaded the SCPI-99 document from here: http://www.ivifoundation.org/specifications/default.aspx (http://www.ivifoundation.org/specifications/default.aspx)

It does say that the commands can be concatenated with ";", such as ":STOP;:WAV:SOUR CHANnel1;MODE RAW". Is this what I'm supposed to use instead of "\n"? How do I know which commands can be chained with ";" and which cannot?

It doesn't say anything else about delimiters, but it has to be some way to delimit commands - the Rigol examples don't seem to contain any ";" at the end and same goes for the examples in SCPI-99. I assumed "\n", but it doesn't seem to work.

It doesn't say anything about state machines and ready states and responses to the commands. E.g. Am I supposed to receive something in the response to the ":STOP" command?

Could you point me to a document which describes this basic stuff?

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Karel on December 01, 2016, 06:05:12 am
Could you point me to a document which describes this basic stuff?

Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail. I don't know if Rigol is following the exact SCPI/TMC guidelines however...

What I noticed during my experiments some time ago is that, when you send two commands to quickly one after another,
strange things can occur. For me, the only way to get things working with Rigol stuff is by keeping a minimum delay between
consegutive commands. And for sure, don't send multiple commands in one packet...
But maybe now with the newer firmware versions they behave better, I haven't tried that.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 01, 2016, 06:58:35 am
Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail.

It explains what to send, but it doesn't say how. May be this is obvious to someone familiar with SCPI, but I'm not. So, the questions I have are these:

1. How do I terminate commands. Do I send ":STOP", ":STOP\n", ":STOP\r\n"?

2. Can I concatenate commands. Like ":STOP;:WAV:MODE:RAW"?

3. What response should I expect. E.g. I say ":STOP\n". Does it say something back?

4. How do I figure if it's ready for the next command or not?

Looks like it is not simply "apply the specs and it complies", but needs lots of tweaking and obeying scope's idiosyncrasies. Easy to figure out when you have a scope. Not that easy otherwise.

@RoGeorge: Looks like it won't be that easy to write the test program without having the scope. If I buy one, I'll make the test program for you. But I can assure you that slow transfers are not Windows fault - the time necessary to transfer 24MBytes over the LAN is 2s for either Linux or Windows (tested W7 and W8.1) - everything above this is caused by the scope and/or the system you're using to access it.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Karel on December 01, 2016, 07:41:00 am
Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail.

It explains what to send, but it doesn't say how. May be this is obvious to someone familiar with SCPI, but I'm not. So, the questions I have are these:

1. How do I terminate commands. Do I send ":STOP", ":STOP\n", ":STOP\r\n"?

:WAV:STOP\n

2. Can I concatenate commands. Like ":STOP;:WAV:MODE:RAW"?

According to the SCPI spec: yes.

In the case of Rigol: yes/no/maybe  (pick one)

3. What response should I expect. E.g. I say ":STOP\n". Does it say something back?

A command is just a command. There's no response. Only queries have a response. Queries end with a questionmark.

4. How do I figure if it's ready for the next command or not?

According to the SCPI spec you can use the *OPC? command to query the device if it has completed the operation.
In the case of Rigol... better to put a delay of 20 milli-Sec. between two consegutive commands...

Looks like it is not simply "apply the specs and it complies", but needs lots of tweaking and obeying scope's idiosyncrasies. Easy to figure out when you have a scope. Not that easy otherwise.

I completely agree.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 01, 2016, 09:32:59 am
33 sec for 24 Mb is 0.73 MB/s. If I get similar rates then:

77 sec for 56 MB in DS2072A, which is worse than I expected, but not that bad.

Almost 7 min for 280 MB in DS4014 doesn't sound good.

Of course, these may be faster or they may be slower (because they are older). But looks like nobody knows :(
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 01, 2016, 10:17:11 am
":STOP;:WAV:SOUR CHANnel1;MODE RAW". Is this what I'm supposed to use instead of "\n"? How do I know which commands can be chained with ";" and which cannot?

- I would use ":STOP;:WAV:SOUR CHANnel1;:MODE RAW\n". ':' before a commands starts interpreting the command from the root of the commands tree. Otherwise, the command starts from the last branch. '\n' must be the terminator.
- Not sure how many and which commands can be sent on the same line. I saw this technique usually for sending multiple settings before executing a command, but for other instruments, not for Rigol scope, e.g. send the start frequency, stop frequency, amplitude, etc, all on the same line, then start a frequency sweep in frequency generator instrument on the next line. For Rigol scope, I just tried ":stop;:wav:form byte;:wav:form?\n" and it returned "BYTE\n"

(SCPI-99) doesn't say anything about state machines and ready states and responses to the commands. E.g. Am I supposed to receive something in the response to the ":STOP" command?

- If there is no state machine in SCPI-99, then I might have read about it somewhere else. I did a lot of googling when I start learning about SCPI.
- For the ':STOP\n' command no. Usually, for (correct) commands you don't receive an SCPI response. Only for queries. The difference between commands and queries is described in the SCPI standard.

Could you point me to a document which describes this basic stuff?
- I am not sure if such a thing exists. At least I couldn't find it, but after a few days of googling and reading various documents you will have a good grasp about SCPI.

From memory, what seemed most important to me at that time:
- I remember some manuals from Agilent instruments had good explanations and charts with status registers, enabling registers, etc. e.g.: http://cp.literature.agilent.com/litweb/pdf/ads2001/pdf/vsaprog.pdf (http://cp.literature.agilent.com/litweb/pdf/ads2001/pdf/vsaprog.pdf)
- Sometimes, when you send an invalid command (mostly because of a wrong parameter) you might receive something like "Command error". Parameter validation is not guaranteed, so if you send something wrong, it might be just ignored.
- There are a lot of SCPI registers, some mandatory, some others are instrument dependent. You need to test the status of your instrument before sending something.
- The most important is to test if the previous operation completed (*OPC?) and if there are errors. Even the simplest commands like :STOP might take a very long time, or might fail.
- Some queries will have an answer with the length of the expected reply, i.e. the :WAV:DATA? will send a header that contains the length of the binary data and will end in \n

====================================================

Long story short, it is not trivial to write a full program for SCPI, but this is not the point for the moment. We were testing how fast can your buffering method download the data on both Linux and Windows.

I will do all the scope settings manually, then run your test program. Try to write just the part for setting the start, stop, receiving/saving data and timing the whole loop. Don't bother treating errors for the moment, just make sure you received all the data you requested. If you feel you don't need this test any more, it's OK from my side.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 01, 2016, 10:38:31 am
I placed the previous post before refreshing the page left open this morning.

... I can assure you that slow transfers are not Windows fault ...

OK.  ^-^

https://www.youtube.com/watch?v=K2zKaM9yqfg (https://www.youtube.com/watch?v=K2zKaM9yqfg)
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 02, 2016, 09:08:42 am
I will do all the scope settings manually, then run your test program. Try to write just the part for setting the start, stop, receiving/saving data and timing the whole loop. Don't bother treating errors for the moment, just make sure you received all the data you requested. If you feel you don't need this test any more, it's OK from my side.

Based at what you and Karel say, it doesn't require much of special care. And based on your test, it did do smething for 0.6s and then it sent some data. So, looks like it actually worked. I think it simply got upset when it found the next "DATA?" command while still executing the previous one. So, we can try downloading a single segment and see how long it takes. Then I can modify the program to do it for all 96 segments. Sounds simple enough.

The improved version is attached. I removed the ":STOP" command from the beginning (because it may require some lengthy processing on the Rigol side). The program now downloads only one 250000 byte segment, and then it prints first 128 bytes of what it received and quits. Let's see if any gardeners show up :)

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 03, 2016, 09:09:26 pm
@NorthGuy, you were correct, if there was a bug then it was fixed, so there is no garden any more  :)
I can not reproduce the transfer problem any more.

PS. Looking at the TCP packets with WireShark, the Windows and Linux versions of the last attachment are slightly different, meaning the 2 versions are not sending the same SCPI commands. These are the outputs:

Debian 8.0
Code: [Select]
rogeorge@debian80:~/Downloads/tbt/v2$ ./ds1054z 192.168.1.3
Connected
Received 1250073 bytes in 0.972 sec.
rogeorge@debian80:~/Downloads/tbt/v2$

Windows 10
Code: [Select]
C:\tbt from EEVblog forum\v2>ds1054z.exe 192.168.1.3
Connected
Received 250012 bytes in 0.438 sec.
23 39 30 30  30 32 35 30  30 30 30 83  83 83 83 83   |#9000250000.....|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 83   |................|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 83   |................|
83 84 83 84  83 84 83 84  83 84 83 83  83 83 83 83   |................|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 84   |................|
83 84 83 84  83 84 83 83  83 83 83 83  83 83 83 84   |................|
83 84 83 84  83 83 83 84  83 84 83 83  83 84 83 83   |................|
83 83 83 83  84 84 84 84  83 84 83 84  83 84 84 84   |................|

C:\tbt from EEVblog forum\v2>

The Linux version in the first and the second attachment are the same:
Code: [Select]
C:\>fc /b "C:\tbt from EEVblog forum\v2\ds1054z" "C
:\tbt from EEVblog forum\ds1054z"
Comparing files C:\TBT FROM EEVBLOG FORUM\V2\ds1054z and C:\TBT FROM EEVBLOG FORUM\DS1054Z
FC: no differences encountered


C:\>

Anyway, the time calculated from the WireShark timestamp of the TCP packets for the first 250000 bytes only is basically the same, and I couldn't see the RWIN problem any more:

Debian80 transfer time = 0.066444
Win10 transfer time = 0.067693

The HaD info was updated with a 'Later edit' too: https://hackaday.io/project/5807-driverless-rigol-ds1054z-screen-capture-over-lan/log/18807-solved-a-painfully-slow-lan-on-windows-and-a-wasted-weekend

Thank you for the test programs.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 04, 2016, 04:46:45 am
Sorry about the Linux. A bug in my build script  |O The correct one is attached.

For whatever reason it went further now  :-//

But look at the transfer speed. It did five segments until it died. If it transfers the first segment in 0.44 sec and the overall time is 0.97s, then it transferred 4 segments in 0.97 - 0.44 = 0.53 sec. This is roughly 2Mbytes /sec - 12s for the whole memory. Not anywhere close to 2s, but substantially less than 33s reported by Karel.

Do you want me to expand the test program to read the whole 24M to see if we can sustain the speed? I'll do basically the same for all 96 segments, except that I won't send the next ":WAV:DATA?" command until all the data from the previous command is received.

For that, we need to resolve one controversy - it received 250012 bytes, while I expected 250011. Do you know what is the extra byte? I suspect it added "\n" at the end. Can you confirm that?

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 04, 2016, 05:43:35 am
Do you want me to expand the test program to read the whole 24M to see if we can sustain the speed? I'll do basically the same for all 96 segments, except that I won't send the next ":WAV:DATA?" command until all the data from the previous command is received.

From my side, I am convinced that the bug that lowered down the transfer time 100 times is gone. If you want me to run other test programs that might help you, sure, with pleasure. I already have the setup, so it will be very easy to run further versions and publish the results. I am curious to see a full RAW transfer time too. Please keep posting both the Win and Lin builds, and let's use versions for the ZIP name, so we can easily refer to the results. If you agree, your next zip should be v4.

For that, we need to resolve one controversy - it received 250012 bytes, while I expected 250011. Do you know what is the extra byte? I suspect it added "\n" at the end. Can you confirm that?

AFAIK all the SCPI responses end up in \n. I saved all the packets from the previous test, so you can check this. The PCAPNG files can be opened with WireShark (files were saved with WireShark v1.12.4 for Windows) and you can see the timing, the handshake, the TCP payload, pretty much everything. The sniffer was set to record all the LAN traffic between the PC and the DS1054Z for the v2 of your zip.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 04, 2016, 09:19:04 am
Here it is. It doesn't parse the responses, just count bytes. Sends a command, waits for 250012 bytes, then sends another. If it doesn't get enough within 3 seconds, it bails out. Otherwise, same as v2.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 04, 2016, 10:47:29 am
Linux manual preset:
Code: [Select]
rogeorge@debian80:~$ nc 192.168.1.3 5555
:wav:form?;wav:mode?       
BYTE;NORM
:wav:mode raw
:wav:form?;:wav:mode?;:wav:star?;:wav:stop?
BYTE;RAW;1;1200
^C
rogeorge@debian80:~$

Linux run:
Code: [Select]
rogeorge@debian80:~/Downloads/v4$ ./ds1054z
Version 4
Usage: ds1054z ip.address.of.the.scope
rogeorge@debian80:~/Downloads/v4$ ./ds1054z 192.168.1.3
Version 4
Connected
Received 24001152 bytes in 22.286 sec.
rogeorge@debian80:~/Downloads/v4$

Windows manual preset:
Code: [Select]
c:\>nc 192.168.1.3 5555
:wav:form?;:wav:mode?;:wav:star?;:wav:stop?
BYTE;RAW;23750001;24000000
^C
c:\>

Windows run:
Code: [Select]
c:\v4>ds1054z.exe 192.168.1.3
Version 4
Connected
Received 24001152 bytes in 22.531 sec.

c:\v4>

The test signal was a 24ms sawtooth (raising ramp) between -2.5 to 2.5V. Trigger level 0V on raising edge. Scope in STOP mode. Before each program run, the "SINGLE" button was pressed.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: gmb42 on December 04, 2016, 11:22:06 am
AFAIK all the SCPI responses end up in \n. I saved all the packets from the previous test, so you can check this. The PCAPNG files can be opened with WireShark (files were saved with WireShark v1.12.4 for Windows) and you can see the timing, the handshake, the TCP payload, pretty much everything. The sniffer was set to record all the LAN traffic between the PC and the DS1054Z for the v2 of your zip.

Are you sure the captures in your attached zip were from the different OS's you stated?  I ask, because I had a look at them to see what differences there were if any, and they look remarkably similar.  The PC in both cases uses the same IP address (192.168.1.2) and MAC Address (d0:50:99:2b:77:fb), but you could be dual booting, but the time to live (TTL) for the initial SYN in both cases is 128, this is the expected value for Windows, but Linux usually uses 64.

That's also a rather old version of Wireshark for Windows, the latest release is 2.2.2.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 04, 2016, 12:07:26 pm
The Windows (real) machine is running on the (real) hardware. The Linux (virtual) machine is running inside VMware. Both VMware and WireShark were running all the time on the Windows real machine only.

Maybe that is why you don't see the expected values. No test was run and sniffed from a real Linux machine. I expect the speed transfer on a real vs. a virtual Linux machine to be the same, but that would be indeed an interesting benchmark.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 04, 2016, 12:26:03 pm
Thank you for running the tests.

So 22.5 sec it is, roughly 1Mbyte/sec, for either Windows or Linux, almost as if Rigol had 10Mbit PHY.

Anyone with DS2000 or DS4000 would agree to measure the transfer speed? Only a slight modification is required for DS2000 - changing from 24Mb to 56Mb.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: gmb42 on December 05, 2016, 07:36:53 am
The Windows (real) machine is running on the (real) hardware. The Linux (virtual) machine is running inside VMware. Both VMware and WireShark were running all the time on the Windows real machine only.

Maybe that is why you don't see the expected values. No test was run and sniffed from a real Linux machine. I expect the speed transfer on a real vs. a virtual Linux machine to be the same, but that would be indeed an interesting benchmark.

Not really an apples to apples comparison then.  The linux VM also has the VMWare drivers, and then the host OS drivers in the path.  In the end that doesn't really matter as the slow part is at the other end of the wire, the oscilloscope.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 05, 2016, 10:21:55 am
Not really an apples to apples comparison then.

I was curious about this too, so I tested it one more time by booting from a Debian 8.6 live CD and installing WireShark on the real Debian live machine. The transfer speed is similar:
Code: [Select]
user@debian:~/Downloads$ ./ds1054z 192.168.1.3
Version 4
Connected
Received 24001152 bytes in 22.208 sec.
user@debian:~/Downloads$

The compressed pcapng file is more then 2MB, too big to attach it. Can be download from:
http://s.go.ro/bms6l45m (http://s.go.ro/bms6l45m) | password: 862441

Here I will attach just a pic with some nice charts instead:
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 06, 2016, 02:55:26 am
This is a very interesting graph. Looks like it spends half of the time fetching data from the acquisition memory, then half of the time transmitting. If we could somehow parallel these processes, it would be faster. So, I though why not to try issuing the next command immediately after we receive the first packet of the response. I've corrected the program to do this. Would you please see if this improvement works and if it actually makes things faster? V5 is attached.

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 06, 2016, 07:26:30 am
It is a little bit faster.

Three runs for each OS. The scope's "SINGLE" button was pressed once before each run:
Code: [Select]
rogeorge@debian80:~/Downloads/v5 - interleaved request$ ./ds1054z 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 22.061 sec.
rogeorge@debian80:~/Downloads/v5 - interleaved request$ ./ds1054z 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 22.162 sec.
rogeorge@debian80:~/Downloads/v5 - interleaved request$ ./ds1054z 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 21.973 sec.
rogeorge@debian80:~/Downloads/v5 - interleaved request$

==========================

C:\v5 - interleaved request>ds1054z.exe 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 22.078 sec.

C:\v5 - interleaved request>ds1054z.exe 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 22.203 sec.

C:\v5 - interleaved request>ds1054z.exe 192.168.1.3
Version 5
Connected
Received 24001152 bytes in 22.171 sec.

C:\v5 - interleaved request>

The data request was indeed interleaved. All test results available at:
http://s.go.ro/hcy7ogyq (http://s.go.ro/hcy7ogyq) | password: 763674

Other options might be:
- make multiple requests at once, like 3-4 of 250 000 consecutive requests separated with semicolon.
- make single interleaved request but for 1 000 000 points instead of 250 000. Depending on how much free memory the scope might have at that moment, sometimes it can serve up to 1 700 000 points at once. If the response starts with a number after #9000 that is not equal with the expected length, then the request was too big. 1 mil points is usually safe for a single trace and no other scope functions activated.
- open a new socket for each request of a new datapoints batch, etc
but I suppose there will be no spectacular increase in transfer speed because, looking at the charts, it seems most of the time is spent waiting for the scope to prepare the next batch of datapoints to be sent.

Some charts for v5 results:
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 06, 2016, 09:16:17 am
Here's the version with 1000000-byte sections (although it is more than the maximum suggested by the specs).

I don't expect it to get much faster - it still needs to fetch all the points from the acquisition memory, then send all data through the TCP link. If we cannot make it do both operations at the same time, we won't get much gain. We can remove some overhead at the stitches, but the bulk of the process will still be the same.

Edit: Sorry, forgot to bump the version number :(
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 06, 2016, 12:06:23 pm
Better!  :D

Result files:
http://s.go.ro/d06c0jl8 (http://s.go.ro/d06c0jl8) | password: 466991

Code: [Select]
rogeorge@debian80:~/Downloads/v6 - 1mil points$ ./ds1054z 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.436 sec.
rogeorge@debian80:~/Downloads/v6 - 1mil points$ ./ds1054z 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.435 sec.
rogeorge@debian80:~/Downloads/v6 - 1mil points$ ./ds1054z 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.446 sec.
rogeorge@debian80:~/Downloads/v6 - 1mil points$

===================================

c:\v6 - 1mil points>ds1054z.exe 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.422 sec.

c:\v6 - 1mil points>ds1054z.exe 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.437 sec.

c:\v6 - 1mil points>ds1054z.exe 192.168.1.3
Version 6
Connected
Received 24000288 bytes in 13.391 sec.

c:\v6 - 1mil points>

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 06, 2016, 03:17:27 pm
That's surprising. Looking at your diagrams, it appears that they have a fixed-length delay approximately 120 ms. This delay occurs when they receive the "DATA?" request. It's hard to imagine what they are doing for 120 ms.

v6 (compared to v5) has eliminated 72 out of 96 delays. Now there's only 24 left. If we go to 1500000-byte segments, it should eliminate 8 more of them, which is about 1 more second - not that much.

Actual transmission speed appears to be almost about 7 sec + 3 sec of delays + 3.5 sec for Rigol internal memory transfers.

Anyway, 13 sec is not bad. I suspect DS2072A and DS4014 should be faster - looking at the documentation they have a continuous memory reading thread which should run in parallel, so one would expect speeds close to 3Mbytes/sec, but I guess we'll never know unless I buy and test one.

I also noticed that new Siglent SDS2104X has 140Mpts memory, but there are not many of them around, so it won't be easy to evaluate the speed for them.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 06, 2016, 08:27:57 pm
...DS2072A and DS4014 should be faster - looking at the documentation they have a continuous memory reading thread which should run in parallel...

Can you point me to that info, please?
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 07, 2016, 02:39:31 am
...DS2072A and DS4014 should be faster - looking at the documentation they have a continuous memory reading thread which should run in parallel...

Can you point me to that info, please?

The programming specification for DS2072A:

http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-0508/1/-/-/-/-/MSO2000A%26DS2000A_ProgrammingGuide.pdf (http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-0508/1/-/-/-/-/MSO2000A%26DS2000A_ProgrammingGuide.pdf)

It describes two methods to read data. One is the same as for DS1054Z. Another is opening a reading thread then reading data with series of consecutive ":WAV:DATA?".

For DS4014 only the thread method exists, but there's no PDF for the programming specifications, only CFM. I can't find a link to it. The PDF for DS4014E is here:

http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-06f3/1/-/-/-/-/MSO%26DS4000_programming.pdf (http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-06f3/1/-/-/-/-/MSO%26DS4000_programming.pdf)

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 07, 2016, 07:24:40 am
Thank you for your benchmark program. Now I am very curious about transfer speed for DS2000 an DS4000.

Does anybody have access to a Rigol DS2000 or DS4000 and willing to run the benchmark program, please?

It is very simple to run the benchmark program written by NorthGuy:
No installation is required. Just run the benchmark.
The benchmark program is working for network only, so the oscilloscope must be connected with the PC by a LAN cable, not by USB. No NI VISA, or IVI, or other kind of drivers are necessary to install. If some drivers are already installed, it doesn't matter.

The program is called ds1054, but AFAIK it should work with DS2000 or DS4000 series too.
The benchmark will transfer 24 million data points from the oscilloscope to the PC and will type the transfer time.

Please let me know the result, and many thanks!
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 07, 2016, 07:54:25 am
My results on a DS4024 with Firmware Revision 00.02.02.SP5, memory set to 140MPoints (if it matters). Executed on a Win7 64Bit system:
Code: [Select]
Version 6
Connected
Received 24000288 bytes in 46.598 sec.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 07, 2016, 08:43:34 am
Thank you very much, Twoflower!

This is amazing, why a DS4024 will do the transfer slower then a DS1054 ?!? :o

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 07, 2016, 09:29:44 am
Thank you Twoflower!

Wow! That is very slow - 4 times slower than DS1054Z. It's almost 10 minutes for whole 280 MBytes of memory. I expected it to be lots faster.

May be this is because of the older firmware. I guess 00.02.03 is the latest.

Another possibilty is that is because the test doesn't use ":WAV:BEG".

@Twoflower: Would you agree to run more tests if I modify the benchmark program a little (to include ":WAV:BEG")?

@RoGeorge: They don't have ":WAV:BEG" in DS1054Z. Perhaps they just removed it because it was redundant, but the functionality is still there. Then, we can set ":WAV:STAR" to 1 and ":WAV:STOP" to 24000000 and then keep calling ":WAV:DATA?" until we read out everything. This will require some changes to the program since I'll have to parse  responses, not only count bytes. Would you want to try this?

Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 07, 2016, 04:36:31 pm
Yes, I retested it and it was about the same transfer rate.  :-//

Sure I'm happy to run some more tests as the outcome might be useful not just for me. And if you need some other tests I might be able to use wireshark as well (a little at least).

On request I could also boot a Linux to see if it changes.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 08, 2016, 02:14:38 am
MSO2072A, Software version 00.03.04.SP1
Memory set to 56 MB

Code: [Select]
pedre@linux-m9pe:~/bin/DS1054Z> ./ds1054z 192.168.1.52
Version 6
Connected
Failure. Not enough data received.
Received 2824 bytes in 0.024 sec.

Peter

Thank you Peter. Looks like it didn't work. Apparently DS2072A handles things differently. May I ask you to re-run the test with version 4 of the benchmark program? It is attached to the post #27 of this thread.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 08, 2016, 07:10:47 am
Could you please try another one on your DS2072A. Here I rolled back the early response which we have introduced in v5 (looks like it caused problems on DS2072A), but it still has the big block size (same as in v6).
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 08, 2016, 08:10:25 am
Request while receiving data (v6) didn't worked on DS1054Z/DS1104Z either, but I didn't write this here because it happened only when the scope had a direct cable with the PC (without router) and only under Linux.

For v7, the time is the same as for v6 with my DS1054:
Code: [Select]
C:\>"C:\v7 - for DS2000\ds1054z.exe" 192.168.1.3
Version 7
Connected
Received 24000288 bytes in 13.703 sec.

C:\>

============================

rogeorge@debian80:~/Downloads/v7 - for DS2000$ ./ds1054z 192.168.1.3
Version 7
Connected
Received 24000288 bytes in 13.554 sec.
rogeorge@debian80:~/Downloads/v7 - for DS2000$

============================

request 1 mil points, not interleaved with the answer


All v7 test:
http://s.go.ro/01xaihpf (http://s.go.ro/01xaihpf) | password: 606507

LATER EDIT: I forgot to increment the name of the picture file, it should have been Win10.ds1054v7.save1.chart.png, not v6.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 08, 2016, 08:28:46 am
OK, I did updated my DS4024 to the latest FW (00.02.03) no changes in the transmission speed:
Code: [Select]
>ds1054z.exe 192.168.178.44
Version 6
Connected
Received 24000288 bytes in 45.068 sec.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 08, 2016, 08:57:15 am
no changes in the transmission speed:

How about the speed tested with the new v7 of the benchmark attached in Reply #49: https://www.eevblog.com/forum/testgear/ethernet-transfer-speed-for-rigol-ds2072a-and-ds4014/msg1086650/#msg1086650 (https://www.eevblog.com/forum/testgear/ethernet-transfer-speed-for-rigol-ds2072a-and-ds4014/msg1086650/#msg1086650) , please?
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 08, 2016, 09:06:50 am
No difference:
Code: [Select]
Version 7
Connected
Received 24000288 bytes in 45.131 sec.
The bad thing is I need to go to work early in the morning. It seems that the time zones tend add a huge latency here :-(
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 08, 2016, 09:20:12 am
Thank you very much for the time spent testing, and please forgive my bad English. I didn't wanted to push for more testing this night, nor to be rude, sorry if I did any of that.
Have a nice day tomorrow!  ^-^
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 08, 2016, 10:29:39 am
Thank you for running the benchmarking tests. The results are very interesting and very surprising too.

I have prepared another test. This one is doing as suggested in the manual and tries to get it all at once. It sends  ":WAV:RES\n:WAV:BEG", then it keeps calling ":WAV:DATA?" until 24000000 bytes are received. It prints the first byte of every chunk received. This is to check that it gets correct data. If the scope has captured a ramp, these numbers will increase through the read.

I had to split it into two separate executables. The "ds4014" executable is for DS2000A and DS4000.

The "ds1054z" executable is only for DS1000Z - it does the same, but skips the ":WAV:RES\n:WAV:BEG" commands. I suspect this approach may work for DS1054Z.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: CustomEngineerer on December 08, 2016, 11:19:10 am
As far as knowing how to transfer data off of the Rigol DS2000 series scopes, marmad is probably the best reference. I would strongly recommend you look into some of his posts for more insight. I good thread discussing some of the issues, and fairly recently was:

https://www.eevblog.com/forum/testgear/rigol-ds1000z-series-font-size (https://www.eevblog.com/forum/testgear/rigol-ds1000z-series-font-size)

Something else to keep in mind, the DS2000A series has up to 56Mpts (megapoints) memory, not megabytes. And you definitely need to do the transfer differently than on the DS1000Z series for optimal results. Sorry I can't give you more than that, but its not something I've played much with.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: RoGeorge on December 08, 2016, 11:28:20 am
...but for 1 000 000 points instead of 250 000. Depending on how much free memory the scope might have at that moment, sometimes it can serve up to 1 700 000 points at once. If the response starts with a number after #9000 that is not equal with the expected length, then the request was too big. 1 mil points is usually safe for a single trace and no other scope functions activated.

The 1 mil points is a trick I discovered when I was developing my open source tool, it is not guaranteed to always work. I didn't looked in the programming manuals for the DS2000/4000 yet, so I don't know if they have too a maximum number of points requested at once.

DS1000 definitely can't work with 24 mil points at once, as the v8 benchmark is trying to request:
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 08, 2016, 11:44:10 am
...If the response starts with a number after #9000 that is not equal with the expected length, then the request was too big..

I misinterpreted this. I though it would return a number other than zero, but somehow less than the number requested, which would be followed by a certain amount of data.

If it returns zero, then, obviously, it cannot work. So, "ds1054z" executable from v8 should not be used. I guess v7 is the best that we can do for DS1000Z, unless some new ideas come up.

The "ds4014" executable should work for DS2000A and DS4000 because that's the way it is documented.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 09, 2016, 01:02:50 am
Thank you Peter. What was the final time at the end?

Let me remove this "Bank of 0 bytes received ..." message. It creates too much clutter. It'll now print this message only when receives something. Still, if it uses 125K Bytes blocks, it'll be about 200 lines of output :)

I've also inserted a 50ms wait after each zero is received.

This version only works on DS2000A and DS4000.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 09, 2016, 05:07:06 am
Thank you very much for the time spent testing, and please forgive my bad English. I didn't wanted to push for more testing this night, nor to be rude, sorry if I did any of that.
Have a nice day tomorrow!  ^-^
No, your English is perfect! :-+ My sentence was too easy to misunderstand as English isn't my native language either. I actually meant that I would actually spend more time to help.

Is there anything I can stress my scope with?
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 09, 2016, 05:58:01 am
Is there anything I can stress my scope with?

It would be nice if you could try v9 (attached to post #61).
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Twoflower on December 09, 2016, 06:13:07 am
Here's the result from the v9:
Code: [Select]
Version 9
Connected
Block of 406682 bytes starting with 179 received at 0.328 sec.
Block of 1984 bytes starting with 83 received at 0.858 sec.
Block of 456320 bytes starting with 82 received at 1.092 sec.
Block of 3968 bytes starting with 82 received at 1.701 sec.
Block of 29760 bytes starting with 83 received at 1.716 sec.
Block of 23808 bytes starting with 82 received at 1.779 sec.
Block of 21824 bytes starting with 82 received at 1.825 sec.
Block of 25792 bytes starting with 82 received at 1.872 sec.
Block of 460288 bytes starting with 82 received at 2.137 sec.
Block of 450368 bytes starting with 82 received at 2.964 sec.
Block of 1984 bytes starting with 82 received at 3.557 sec.
Block of 452352 bytes starting with 82 received at 3.791 sec.
Block of 454336 bytes starting with 178 received at 4.618 sec.
Block of 462272 bytes starting with 178 received at 5.445 sec.
Block of 3968 bytes starting with 179 received at 6.053 sec.
Block of 31744 bytes starting with 179 received at 6.084 sec.
Block of 456320 bytes starting with 179 received at 6.365 sec.
Block of 1984 bytes starting with 178 received at 6.958 sec.
Block of 466240 bytes starting with 178 received at 7.207 sec.
Block of 430528 bytes starting with 88 received at 8.034 sec.
Block of 31744 bytes starting with 82 received at 8.611 sec.
Block of 462272 bytes starting with 82 received at 8.892 sec.
Block of 1984 bytes starting with 82 received at 9.501 sec.
Block of 464256 bytes starting with 83 received at 9.750 sec.
Block of 438464 bytes starting with 82 received at 10.577 sec.
Block of 3968 bytes starting with 83 received at 11.154 sec.
Block of 31744 bytes starting with 82 received at 11.185 sec.
Block of 460288 bytes starting with 82 received at 11.466 sec.
Block of 448384 bytes starting with 179 received at 12.293 sec.
Block of 460288 bytes starting with 180 received at 13.104 sec.
Block of 438464 bytes starting with 178 received at 13.947 sec.
Block of 1984 bytes starting with 179 received at 14.508 sec.
Block of 470208 bytes starting with 179 received at 14.758 sec.
Block of 5952 bytes starting with 83 received at 15.382 sec.
Block of 29760 bytes starting with 83 received at 15.413 sec.
Block of 21824 bytes starting with 82 received at 15.460 sec.
Block of 21824 bytes starting with 82 received at 15.507 sec.
Block of 25792 bytes starting with 83 received at 15.553 sec.
Block of 442432 bytes starting with 83 received at 15.819 sec.
Block of 1984 bytes starting with 83 received at 16.396 sec.
Block of 458304 bytes starting with 82 received at 16.630 sec.
Block of 436480 bytes starting with 82 received at 17.457 sec.
Block of 1984 bytes starting with 82 received at 18.034 sec.
Block of 454336 bytes starting with 83 received at 18.268 sec.
Block of 430528 bytes starting with 179 received at 19.079 sec.
Block of 3968 bytes starting with 179 received at 19.656 sec.
Block of 31744 bytes starting with 179 received at 19.672 sec.
Block of 470208 bytes starting with 178 received at 19.953 sec.
Block of 3968 bytes starting with 178 received at 20.577 sec.
Block of 33728 bytes starting with 179 received at 20.608 sec.
Block of 468224 bytes starting with 178 received at 20.889 sec.
Block of 1984 bytes starting with 178 received at 21.497 sec.
Block of 468224 bytes starting with 178 received at 21.747 sec.
Block of 7936 bytes starting with 179 received at 22.371 sec.
Block of 470208 bytes starting with 179 received at 22.620 sec.
Block of 3968 bytes starting with 82 received at 23.229 sec.
Block of 29760 bytes starting with 83 received at 23.260 sec.
Block of 21824 bytes starting with 82 received at 23.307 sec.
Block of 21824 bytes starting with 83 received at 23.354 sec.
Block of 25792 bytes starting with 83 received at 23.400 sec.
Block of 478144 bytes starting with 83 received at 23.681 sec.
Block of 452352 bytes starting with 82 received at 24.539 sec.
Block of 450368 bytes starting with 82 received at 25.366 sec.
Block of 136896 bytes starting with 82 received at 26.021 sec.
Block of 464256 bytes starting with 178 received at 26.442 sec.
Block of 436480 bytes starting with 178 received at 27.269 sec.
Block of 3968 bytes starting with 179 received at 27.846 sec.
Block of 29760 bytes starting with 178 received at 27.878 sec.
Block of 23808 bytes starting with 179 received at 27.924 sec.
Block of 466240 bytes starting with 178 received at 28.205 sec.
Block of 1984 bytes starting with 179 received at 28.814 sec.
Block of 470208 bytes starting with 179 received at 29.063 sec.
Block of 31744 bytes starting with 179 received at 29.687 sec.
Block of 476160 bytes starting with 169 received at 29.984 sec.
Block of 3968 bytes starting with 83 received at 30.608 sec.
Block of 31744 bytes starting with 82 received at 30.639 sec.
Block of 462272 bytes starting with 82 received at 30.920 sec.
Block of 1984 bytes starting with 82 received at 31.512 sec.
Block of 466240 bytes starting with 82 received at 31.762 sec.
Block of 456320 bytes starting with 83 received at 32.604 sec.
Block of 3968 bytes starting with 82 received at 33.197 sec.
Block of 31744 bytes starting with 82 received at 33.228 sec.
Block of 452352 bytes starting with 82 received at 33.509 sec.
Block of 450368 bytes starting with 178 received at 34.320 sec.
Block of 31744 bytes starting with 179 received at 34.929 sec.
Block of 460288 bytes starting with 178 received at 35.210 sec.
Block of 450368 bytes starting with 179 received at 36.052 sec.
Block of 1152704 bytes starting with 178 received at 37.238 sec.
Block of 452352 bytes starting with 83 received at 38.954 sec.
Block of 450368 bytes starting with 83 received at 39.780 sec.
Block of 1984 bytes starting with 82 received at 40.373 sec.
Block of 472192 bytes starting with 82 received at 40.623 sec.
Block of 460288 bytes starting with 178 received at 41.481 sec.
Block of 446400 bytes starting with 178 received at 42.308 sec.
Block of 444416 bytes starting with 178 received at 43.119 sec.
Block of 3968 bytes starting with 178 received at 43.696 sec.
Block of 31744 bytes starting with 178 received at 43.727 sec.
Block of 25792 bytes starting with 179 received at 43.790 sec.
Block of 80934 bytes starting with 179 received at 44.024 sec.
Received 24001200 bytes in 44.320 sec.
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: NorthGuy on December 09, 2016, 03:30:06 pm
I want to thank all the participants for the tests. Here's the summary of what we have found:

Connections

The benchmark program connected to port 5555 of the scope over LAN. We used two distinct methods to read data.

Block method

This is explained in programming manual for DS1000Z series.

:WAV:STAR <block begin>
:WAV:STOP <block end>
:WAV:DATA?

for example

:WAV:STAR 1
:WAV:STOP 250000
:WAV:DATA?

Once the data is received, the next set of commands can be sent to the scope. In most cases, it is possible to send the next set of commands earlier - as soon as the scope starts sending the data back. But sometimes such early response doesn't work. Since the early response only gains milliseconds per MByte, there's no reason to use it.

The block method works for every scope and is implemented in version 4 of the benchmark software (attached to post #27). Instructions are in post #41.

Even though the documentation limits the block size to 250000, it is possible to use block size of 1000000, which considerably speeds the transfer up, but we had a failure with DS2072A (see post #51). This method is implemented in version 7 of the benchmark software (attached to post #49).

Stream method

This is explained in programming manual for DS4000 series, except we do not do status queries.

First, these two commands are sent

:WAV:RES
:WAV:BEG

The data command follows.

:WAV:DATA?

Each command gets a response which contains some of the data (possibly zero). If we receive zero, we wait 50ms, then issue the data request command again. If it's not zero, we retrieve the data, then send the data command.

This is continuing until all the data is retrieved.

At the end, we do

:WAV:END

This method works on DS4000 series (as expected) and even provides slight improvement over the block method.

On DS2000A, this method is very inefficient because the scope doesn't organize the continuous data stream but rather sends data in blocks of 126976 (0x1f000) bytes. Because block size is small, this is much slower than the block method.

This doesn't work on DS1000Z series at all.

This is implemented in version 9 of the benchmark software (attached to post #61).

Data rates

Approximate data rates are:

DS1000Z - 1.7 MBytes/sec
DS2000A -  0.2 MBytes/sec (could be better if 1000000 block size worked, but it didn't)
DS4000 -  0.5 MBytes/sec

Approximate times to transfer all the deep memory data from the scope to PC:

DS1000Z (24 MBytes) - 15 sec
DS2000A (56 MBytes) - 4 min 40 sec
DS4000 (280 MBytes) - 8 min 50 sec
Title: Re: Ethernet transfer speed for Rigol DS2072A and DS4014
Post by: Karel on December 09, 2016, 07:53:25 pm
I can confirm the datarate for the DS1000z series using USB connection and using a slightly modified version of DSRemote.
When changing the value of macro SAV_MEM_BSZ in line 29 in save_data.cpp from 250000 to 1000000,
the downloadspeed increases from 0.73Mbytes/sec to 1.7Mbytes/sec.

Somebody else already did the same test using DSRemote some time ago. The question is, do you want to increase the risk
of data corruption while, even when following exactly the programming guide, the firmware has proven to be unreliable?
Everybody has to decide that for himself. Personally, I prefer to stick with the limit written in the programming guide,
i.e. the slow download speed, in order to stay on the "safe" side...(as far as that is possible with Rigol stuff...)