EEVblog Electronics Community Forum

Electronics => Projects, Designs, and Technical Stuff => Topic started by: Warhawk on May 13, 2015, 02:36:33 pm

Title: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on May 13, 2015, 02:36:33 pm
Dear colleagues,
I've been fiddling with an analog signal processing recently and I needed to import data from DS1074z scope to LTSpice (or similar application) and play with them then.
There are some apps or python scripts but mainly for the older 1052 series and none of them satisfied my needs. Thus I created my own minimalistic (~6 kB) utility in pure C.

Warning: The source code may cause nausea or seizure.. :scared:
Please take into account that I am not a SW or FW developer. I just use C for my hobby embedded projects.

Description: The utility takes CSV data from Rigol DS1000z series scope, recognizes a header and creates a *.txt file with PWL data for every available channel. It has been tested in Windows 7 and 8.1 x64. The utility also handles even very large CSV files. Conversion of 20 MB CSV takes couple seconds.

License: Code is available for everyone under "I don't care" license. I would be glad if someone considered the utility as useful.

Usage: Simplest way is to copy rigol.exe to the same folder as your CSV data, open a cmd line and execute "rigol.exe mydata.csv". The utility then creates ch1_out.txt, ch2_out.txt etc. Then you can import a text file to LTspice : http://www.linear.com/solutions/1814 (http://www.linear.com/solutions/1814)
You can use "-kt" parameter which sets time "0" to the trigger point.

Modifications: I believe that the code is simple and easy to understand for everyone who is familiar with ANSI C. For modifications you just need a notepad and TCC (Tiny C Compiler) from: http://bellard.org/tcc/ (http://bellard.org/tcc/) . You don't need any special tools or libraries. Everything is included in that 400 kB archive and without any installation... :-+
To get started, just put "rigol.c" into TCC folder, open a cmd line and put there "tcc.exe rigol.c". Then you find rigol.exe in the same folder.

Known bugs: (this list will probably grow...)

All input/comments/suggestions/bug reports welcomed. :blah:

Jiri

*source code added
Title: Re: Rigol DS1000z CSV to LTspice (PWL) parser [Testers welcomed !]
Post by: moffy on May 14, 2015, 12:44:40 am
Very cool! Thanks for making your work available.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on May 14, 2015, 11:18:05 am
Thanks for your reply moffy.
Today I found a way how to import data to popular WFM reader from
http://meteleskublesku.cz/wfm_view/ (http://meteleskublesku.cz/wfm_view/)

I am in touch with the author but however He does not have a time to make the ds1000z support. I understand that of course. Reverse engineering of the header is very tedious. I think some folks here on forum tried to do that but with partial success only. Honestly I don't fully understand why to care about wfm format when we have CSV instead.
At the moment I use matlab* to convert the CSV data to WAV. Right after I check the proper functionality I will try to update the utility to support WAV export as well. There are some open source codes in C for that.

If you want to try it, just remove the Rigol's CSV header and
"importdata('towav.csv')" and "wavwrite(ans,500e6,32,'out.wav')" do the trick.

*It seems that scilab can do it too.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on July 07, 2015, 05:16:36 pm
Interesting.
So far, I have been using EDFbrowser to inspect waveforms downloaded from Rigol oscilloscopes: http://www.teuniz.net/edfbrowser/ (http://www.teuniz.net/edfbrowser/)
This way I don't need to do any conversion of file formats.



Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on July 08, 2015, 07:36:05 am
Interesting.
So far, I have been using EDFbrowser to inspect waveforms downloaded from Rigol oscilloscopes: http://www.teuniz.net/edfbrowser/ (http://www.teuniz.net/edfbrowser/)
This way I don't need to do any conversion of file formats.

Nice software you linked, Karel ! I will check it out. What type of files it accepts ? On the web I see nothing regarding rigol binary data or even CSV ?
 
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on July 08, 2015, 10:09:54 am
It doesn't accept the waveform files from Rigol scopes.
Instead, I'm using this software that saves the waveform data from the scope (via LAN or USB)  in EDF format that can be read immediately without the need for any conversion:

http://www.teuniz.net/DSRemote/ (http://www.teuniz.net/DSRemote/)

Otherwise, you can use CSV files but EDFbrowser will convert them to EDF anyway.

Have a look at the manual here for the conversion from CSV to EDF:

http://www.teuniz.net/edfbrowser/EDFbrowser%20manual.html#ASCII_to_EDF_converter (http://www.teuniz.net/edfbrowser/EDFbrowser%20manual.html#ASCII_to_EDF_converter)

EDF files can also be read or imported in Scilab, Octave, Matlab, etc.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on July 08, 2015, 02:17:02 pm
It doesn't accept the waveform files from Rigol scopes.
Instead, I'm using this software that saves the waveform data from the scope (via LAN or USB)  in EDF format that can be read immediately without the need for any conversion:

http://www.teuniz.net/DSRemote/ (http://www.teuniz.net/DSRemote/)

Otherwise, you can use CSV files but EDFbrowser will convert them to EDF anyway.

Have a look at the manual here for the conversion from CSV to EDF:

http://www.teuniz.net/edfbrowser/EDFbrowser%20manual.html#ASCII_to_EDF_converter (http://www.teuniz.net/edfbrowser/EDFbrowser%20manual.html#ASCII_to_EDF_converter)

EDF files can also be read or imported in Scilab, Octave, Matlab, etc.

Thanks for clarifying that. Does DSremote allow to download whole acquisition memory (up to 24 Megsamples) or just the screen memory ?
Anyway, that EDF browser is a wonderful SW. I wish I had found it earlier.  :-DD
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on July 08, 2015, 05:00:29 pm
Does DSremote allow to download whole acquisition memory (up to 24 Megsamples) or just the screen memory ?

There are three choices. When you click on the save button, it will show you a dropdown list to select screenshot, screen waveform data or deep memory waveform data up to the maximum (24Mp for the DS1054Z).
The last option depends on the mem depth setting in the acquire menu.

Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: icpart on January 01, 2016, 04:39:36 pm
Hi Warhawk.
Today I tried to convert csv from my new 1054Z scope to LTtspice but it doesn't work. I received only message from program for Unsupported header.
I exported CSV file via Ultrascope PC program. Also firmware of my scope is 00.04.03.

PS.
Guys do you know is there any detailed explanation of Rigol CSV waveform format, because I don't know what means separated values in file?
I want to try to makes some chart with waveform in Excel.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on January 01, 2016, 04:45:57 pm
Hi Warhawk.
Today I tried to convert csv from my new 1054Z scope to LTtspice but it doesn't work. I received only message from program for Unsupported header.
I exported CSV file via Ultrascope PC program. Also firmware of my scope is 00.04.03.

PS.
Guys do you know is there any detailed explanation of Rigol CSV waveform format, because I don't know what means separated values in file?
I want to try to makes some chart with waveform in Excel.

Hi, could you please post here the CSV file you have ? I will have a look at it  ;)

The file description is very easy. Look at the picture.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: icpart on January 03, 2016, 06:16:05 pm
Hi Warhawk.
Today I tried to convert csv from my new 1054Z scope to LTtspice but it doesn't work. I received only message from program for Unsupported header.
I exported CSV file via Ultrascope PC program. Also firmware of my scope is 00.04.03.

PS.
Guys do you know is there any detailed explanation of Rigol CSV waveform format, because I don't know what means separated values in file?
I want to try to makes some chart with waveform in Excel.

Hi, could you please post here the CSV file you have ? I will have a look at it  ;)

The file description is very easy. Look at the picture.
Thanks for help. It was my mistake  ::). I realized later that your converter is for CSV files saved from oscilloscope and not for CSV files exported from Rigol Ultrascope PC soft. Also after I looking in depth from your example files from first post I realized the data format of that CSV.
Btw more interesting is CSV format exported from Ultrascope software. It is little confusing.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: marth on December 03, 2016, 02:49:03 pm
Hi Warhawk.
Data transmission in LTspice - very interesting. I have a model of 1054z. It seems there different header format. Here is an example. The file received from the oscilloscope.
X,CH1,Start,Increment,
Sequence,Volt,-3.000000e-03,5.000000e-06
0,-1.80e-02,
1,-1.80e-02,
2,-1.84e-02,
3,-1.84e-02,

Perhaps to make the program option for the 1054z model?
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: marth on December 10, 2016, 03:43:09 pm
author closed the development of this program?
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: marth on December 10, 2016, 04:17:28 pm
Converting to LTSpice format interesting features wide mathematical analysis and the possibility of combining several measurements. It is also interesting to combine with the simulation results.
p.s. Sorry for my english. I use an electronic translator.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: TinkeringSteve on December 11, 2016, 12:31:47 am
Indeed, CSV is awfully wasteful and takes a lot more time than I have to save.
I wonder why Rigol doesn't publish the specs of the files... I was under the impression that "secret file formats" and such have somewhat fallen out of vogue...
Oh wait, somebody could make their product more useful, we can't have that ^^
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on December 11, 2016, 08:49:57 am
I wonder why Rigol doesn't publish the specs of the files... I was under the impression that "secret file formats" and such have somewhat fallen out of vogue...
Oh wait, somebody could make their product more useful, we can't have that ^^

Have you asked Rigol support? If yes, what did they say? If not, why didn't you?

Why bother with their file format if there's a free to download programming guide that describes how to download
the waveform data? Writing a program that saves that data in a format of your preference isn't exactly rocket science.

And if you are not able to write it yourself, grab one from the internet and use that. There are free & opensource
programs that do that for you and you'll be able to analyze your data in Octave, Scilab and matlab.

Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: TinkeringSteve on December 11, 2016, 11:33:21 am

Have you asked Rigol support? If yes, what did they say? If not, why didn't you?

Why bother with their file format if there's a free to download programming guide that describes how to download
the waveform data? Writing a program that saves that data in a format of your preference isn't exactly rocket science.

I searched for the format, did not ask support as this doesn't seem like a support issue. If they wanna publish it, they do. Might try, though.

Not sure what you're talking about after that. Are you saying there is a guide which describes how to open a connection e.g. via sockets or in some way using USB and talk to the Rigol to get the data?
I was not aware of that, I had assumed that the connections only works with vendor provided software.
This is my second scope, before that, I had one of the older, crappier china scopes.
For stationary use here that would be awesome, of course.

Anyway - that thing has a USB port that lets you save wfm data for a reason... It would be a convenient enough way to record the data, if one was able to actually read it. Well I can see the wave data allright, but it is not very useful without the information from the header. I can use the scope + USBstick everywhere, whereas a LAN cable is not everywhere, and carrying more stuff is always bad ;-)
So even if I get the direct data connection working, I'd still like the header format.
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on December 11, 2016, 11:42:50 am
Are you saying there is a guide which describes how to open a connection e.g. via sockets or in some way using USB and talk to the Rigol to get the data?
I was not aware of that, I had assumed that the connections only works with vendor provided software.

Here's the programming guide:

http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-0386/1/-/-/-/-/DS1000Z_Programming%20Guide_EN.pdf (http://beyondmeasure.rigoltech.com/acton/attachment/1579/f-0386/1/-/-/-/-/DS1000Z_Programming%20Guide_EN.pdf)

Here's an example of a tool that let you download all the waveform data in a format that can be opened in lot's of other software,
and it doesn't require Rigol software or drivers:

http://www.teuniz.net/DSRemote/ (http://www.teuniz.net/DSRemote/)

Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: TinkeringSteve on December 11, 2016, 11:52:00 am
Ah, thanks!
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: dpenev on November 13, 2017, 05:23:14 pm
Hi,

I have tested DSRemote for my Rigol DS2102  but I got Unknown device ID message.
I am not sure if the author reads here but if he does probably he can advice?

Thanks.
 
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Karel on November 13, 2017, 06:14:55 pm
I have tested DSRemote for my Rigol DS2102  but I got Unknown device ID message.
I am not sure if the author reads here but if he does probably he can advice?

You can open an "issue" here: https://github.com/Teuniz/DSRemote/issues (https://github.com/Teuniz/DSRemote/issues)
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: silelis on December 17, 2018, 08:59:31 am
@Warhawk

I am trying to use Your tool but do not understand this bug description correctly. Can You explain it more like "for dummies". PLS. :)

  • When the scope saves data from the "memory" (not from the screen), the increment value is not right and has to be changed manually (Rigol's BUG ?). You must edit CSV file manually and put there sampling rate^-1

I am making CSV form memory (I think it is more accurate) and my parameters are like bellow on RIGOL screen screeshot, CSV screenshot.

Unfortunately LT Spice gives me error. I suppose that it is because I do not know how to change this "sampling rate^-1". PLS HELP me. :)
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: Warhawk on January 04, 2019, 08:59:26 am
Hi, I am sorry it took me so long to reply.
Could you please share your original file? I did not use the tool for a while so I need to find out how it works again  :-DD

ad. Sampling rate^-1

- This practically means conversion to sampling period (1/Sampling Rate). Your CSV seems to be right. Either rigol has already fixed the problem or you did the modification correctly. Your sampling rate is 1GS which means 1ns increments.



Hmmm, I also see that your CSV now has some increments (1,2,3,4,5 etc.)
Which firmware do you use?
Title: Re: Rigol DS1000z CSV to LTspice & WFM Viewer 1.0.0 parser
Post by: silelis on January 14, 2019, 09:41:37 am
My firmware is: 00.04.04.SP3.
Here is the begging of "STP" file with settings:
Quote
VZ8 DS1104Z             00.04.04.SP3

Strange because I am using DS1052Z not DS1104Z.

The thing I see is that my Rigol with this firmware adds additional column in CSV (named "X", which is sequence)

CSVs at this moment I am removing this additional column with CSVed.exe.

If You don't have this firmware in Your Rigol You can answer me on PM and We can work together on update.