Products > Test Equipment
Download speed from Rigol DS1054Z or similar oscilloscope to a PC
RoGeorge:
--- Quote from: switchabl on November 20, 2022, 01:37:12 pm ---In order to minimize latency, the other socket option you might want to try is TCP_QUICKACK (in addition to or instead of TCP_NODELAY).
--- End quote ---
Thanks for the hint. Searched today about TCP_QUICKACK, and it seems to be Linux specific. Other TCP stacks don't have the TCP_QUICKACK setting (e.g. the BSD one in FreeBSD or MacOS).
--- Quote --- TCP_QUICKACK (since Linux 2.4.4)
Enable quickack mode if set or disable quickack mode if
cleared. In quickack mode, acks are sent immediately,
rather than delayed if needed in accordance to normal TCP
operation. This flag is not permanent, it only enables a
switch to or from quickack mode. Subsequent operation of
the TCP protocol will once again enter/leave quickack mode
depending on internal protocol processing and factors such
as delayed ack timeouts occurring and data transfer. This
option should not be used in code intended to be portable.
--- End quote ---
Source: https://man7.org/linux/man-pages/man7/tcp.7.html
--- Quote from: lundmar on November 20, 2022, 06:24:51 pm ---I've added some example code here which demonstrates how to send a SCPI command to request screenshot image data from an instrument and receive all the image data:
https://github.com/lxi-tools/liblxi/blob/master/test/receive-image-data.c
--- End quote ---
You are too kind, thank you. I understand now the need for '\n' with lxilib in RAW mode, makes sense. Also it clarifies my question about consolidating consecutive SCPI commands before sending them:
--- Quote from: RoGeorge on November 19, 2022, 09:34:37 am ---Tried liblxi with TCP RAW, and I don't know how to handle the transfers.
- it puts consecutive lxi_send() strings in the same data packet, which the oscilloscope won't understand. I don't know how to flush the transmit buffer after each lxi_send(), so as a workaround I'm adding an *OPC? at the end of each command.
--- End quote ---
'liblxi' works just fine, no need to flush. The instrument will understand multiple commands from a single TCP packet, just that I was sending a wrong SCPI command, a bug in my code, sorry.
alm:
--- Quote from: RoGeorge on November 19, 2022, 12:04:57 am ---- A (telnet) data transfer will drop if inside any data packets it receives a 0x00. This is some RFC spec (for text Telnet IIRC).
- The ADC samples are coming as bytes, which means the incoming data packets will be truncated if one of the bytes is zero, because in the Telnet RFC 0x00 is considered a terminator, and will drop any bytes coming after a 0x00.
- This issue is only observed when the input signal in the ADC is less than 5 divisions on the screen (ADC outputs 0x7f for zero volts, and 0x00 for any input voltage that is -5*volts/div or under)
- Both the pyvisa->pyvisa-py->socket and the socketscpi are affected
--- End quote ---
I'm not sure why this is happening. As far as I know these libraries should use plain TCP sockets, and not implement the Telnet protocol. I would be careful with the word "raw TCP sockets", because raw sockets is generally understood to mean something else in network programming.
I just tested it with PyVISA, and it works as expected for me:
I used this as a mock server returning data containing null characters: echo '\0ff\0' | nc -vl -p 1052
And then succesfully retrieved this data:
--- Code: --->>> import pyvisa
>>> rm = pyvisa.ResourceManager()
>>> res = rm.open_resource('TCPIP0::127.0.0.1::1052::SOCKET')
>>> res.write_termination = '\n'
>>> res.read_termination = '\n'
>>> data = res.read()
>>> data
'\x00ff\x00'
--- End code ---
You don't happen to have the read termination set to \00, have you? Because then I could understand this happening.
RoGeorge:
Might have been something I was doing wrong. Can not reproduce that error any longer. :-//
MiDi:
Had similar problems:
On one PC the readout of 24MSamples took more than 40s and on another ~22s (Both Win10 latest updates, no TCP mods, but different python & module versions).
Turning Nagle off in Win did not help - as was expected as PyVisa turns it off by default for the connection itself.
Turning delayed ACK off in Win did improve the readout times a bit (~36s), but still slow.
As it turned out changing the code did the trick to get ~22s on both PCs:
instead single commands:
--- Code: ---instrument.write(f":WAV:STAR {interval[0]}") # start index of memory
instrument.write(f":WAV:STOP {interval[1]}") # stop index of memory
data += (instrument.query_binary_values(f":WAV:DATA?", datatype='B')) # get the data, B = unsigned char
--- End code ---
all commands at once did it:
--- Code: ---data += (instrument.query_binary_values(f":WAV:STAR {interval[0]}\n:WAV:STOP {interval[1]}\n:WAV:DATA?", datatype='B')) # get the data, B = unsigned char
--- End code ---
For reference the full source code used:
--- Code: ---#!/usr/bin/env python
# -*- coding: utf-8 -*-
# developed & tested with Python 3.6, pyvisa 1.11.3
# Rigol DS1000Z data acquisition
#from ctypes.wintypes import FLOAT
#from unicodedata import decimal
import pyvisa as visa # needs a backend e.g. NI-VISA or PyVISA-py
import sys
from time import time, sleep
from matplotlib.ticker import EngFormatter
from decimal import *
def ef(value, places=None):
return EngFormatter(sep="", places=places).format_eng(float(value)) #round(value, 4) - need to round in exp form
def crange(start,end,step):
i = start
while i < end-step+1:
yield i, i+step-1
i += step
yield i, end
def wait_ready(instrument):
#instrument.write("*OPC")
instrument.write("*WAI")
ready = instrument.query("*OPC?").strip()
#print(ready)
while ready != "1": # never occured, needed?
ready = instrument.query("*OPC?").strip()
print(f"\n-------------------not ready: {ready}-----------------------")
#pass
def is_error(instrument):
wait_ready(instrument)
status = instrument.query("*ESR?").strip()
if status not in ('0', '1'):
wait_ready(instrument)
return instrument.query(":SYST:ERR?").strip() #:SYSTem:ERRor[:NEXT]?
else:
return False
# connect to scope
# initialize scope
##*RST p. 96 - Restore the instrument to the default state
##*WAI p. 97 - Wait for the operation to finish (maybe not helpful as it is only for parallel execution of commands)
##:ACQ:MDEP? p. 21 - Set or query the memory depth of the oscilloscope (namely the number of waveform points that can be stored in a single trigger sample)
##:ACQ:TYPE? p. 22 - Set or query the acquisition mode of the oscilloscope
##:ACQ:SRAT? p. 23 - Query the current sample rate. The default unit is Sa/s.
#def connect(timeout = 10000, chunk_size = 20*1024):
def connect():
try:
# Get the USB device, e.g. 'USB0::0x1AB1::0x0588::DS1ED141904883'
rm = visa.ResourceManager() # "@py" for PyVISA-py
instruments = rm.list_resources() # rm.list_resources('USB?*')
print(instruments)
usb = list(filter(lambda x: 'USB' in x, instruments)) # TODO filter for ::DS1Z
print(usb)
#if len(usb) != 1:
# print(f"No or multiple instruments found: {instruments}")
# sys.exit(-1)
#from pyvisa.highlevel import ascii, single, double
#instrument = rm.open_resource(usb[0])
# instrument = rm.open_resource(f"TCPIP0::192.168.1.26::5555::SOCKET", write_termination = '\n', read_termination = '\n', timeout = timeout, chunk_size = chunk_size)
instrument = rm.open_resource(f"TCPIP0::192.168.1.26::5555::SOCKET", write_termination = '\n', read_termination = '\n')
#instrument = rm.open_resource("TCPIP::192.168.1.26::INSTR", write_termination = '\n', read_termination = '\n')
#instrument.write_termination = '\n'
#instrument.read_termination = '\n'
#instrument = rm.open_resource("USB0::0x1AB1::0x04CE::DS1ZA172316530::INSTR")
#instrument.encoding='latin1' # hack - better do ask raw
#rm.timeout = timeout
#rm.chunk_size = chunk_size
return instrument
except Exception as e:
print(f"Cannot open instrument: {str(e)}")
sys.exit(-1)
def setup(instrument, timebase_scale, ch1_scale, mem_depth):
#timebase_scale = Decimal("50") # timebase scale in s
#ch1_scale = Decimal("1e-3") # set channel scale in units (V, A, W)
#mem_depth = "24000000" # set memory depth in points: 12000, 120000, 1200000, 12000000, 24000000
timebase_scale = Decimal(timebase_scale) # timebase scale in s
ch1_scale = Decimal(ch1_scale) # set channel scale in units (V, A, W)
mem_depth = mem_depth # set memory depth in points: 12000, 120000, 1200000, 12000000, 24000000
instrument.write(f"*CLS") # Clear event registers and error queue
wait_ready(instrument)
status = instrument.query(":TRIG:STAT?").strip() # Stop instrument: defined state
while status != "STOP":
instrument.write(":STOP")
wait_ready(instrument)
status = instrument.query(":TRIG:STAT?").strip()
set_timebase_scale = Decimal(instrument.query(":TIM:SCAL?").strip()) # set timebase scale
while set_timebase_scale != timebase_scale:
print(f"set_timebase_scale: {set_timebase_scale}, timebase_scale: {timebase_scale}")
instrument.write(f":TIM:SCAL {timebase_scale}")
wait_ready(instrument)
set_timebase_scale = Decimal(instrument.query(":TIM:SCAL?").strip())
print(f"Timebase scale: {ef(float(set_timebase_scale))}s/div")
set_ch1_scale = Decimal(instrument.query(":CHAN1:SCAL?").strip()) # query channel scale
while set_ch1_scale != ch1_scale:
print(f"set_ch1_scale: {set_ch1_scale}, ch1_scale: {ch1_scale}")
instrument.write(f":CHAN1:SCAL {ch1_scale}")
wait_ready(instrument)
set_ch1_scale = Decimal(instrument.query(":CHAN1:SCAL?").strip())
print(f"Ch1 scale: {ef(float(set_ch1_scale))}V/div")
set_mem_depth = instrument.query(":ACQ:MDEP?") # memory depth = bytes to read
if set_mem_depth != mem_depth:
status = instrument.query(":TRIG:STAT?").strip() # Start instrument: needed for setting MDEP
while status == ("STOP"):
print(f"status: {status}")
instrument.write(":RUN")
wait_ready(instrument)
status = instrument.query(":TRIG:STAT?").strip()
while set_mem_depth != mem_depth:
print(f"set_mem_depth: {set_mem_depth}, mem_depth: {mem_depth}")
instrument.write(f":ACQ:MDEP {mem_depth}")
wait_ready(instrument)
set_mem_depth = instrument.query(":ACQ:MDEP?")
print(f"Memory depth: {ef(float(set_mem_depth))}pts")
trig_pos = set_timebase_scale * 6 # horizontal position of trigger on left edge (no pre-trigger)
set_trig_pos = Decimal(instrument.query(":TIM:OFFS?").strip())
while set_trig_pos != trig_pos:
print(f"set_trig_pos: {set_trig_pos}, trig_pos: {trig_pos}")
instrument.write(f":TIM:OFFS {trig_pos}")
wait_ready(instrument)
set_trig_pos = Decimal(instrument.query(":TIM:OFFS?").strip())
print(f"Trigger position: {ef(float(set_trig_pos))}s")
# read data (p. 241)
# :STOP - better use single and wait until finished (if that is possible)
# :WAV:FORM WORD - are there values beyond 255? Else BYTE would be sufficient
##S1. :STOP Set the instrument to STOP state (you can only read the waveform data in the internal memory when the oscilloscope is in STOP state)
##S2. :WAV:SOUR CHAN1 Set the channel source to CH1
##S3. :WAV:MODE RAW Set the waveform reading mode to RAW
##S4. :WAV:FORM WORD Set the return format of the waveform data to WORD
##Perform the first reading operation
##S5. :WAV:STAR 1 Set the start point of the first reading operation to the first waveform point
##S6. :WAV:STOP 125000 Set the stop point of the first reading operation to the 125000th waveform point
##S7. :WAV:DATA? Read the data from the first waveform point to the 125000th waveform point
##Perform the second reading operation
##S8. :WAV:STAR 125001 Set the start point of the second reading operation to the 125001th waveform point
##S9. :WAV:STOP 250000 Set the stop point of the second reading operation to the 250000th waveform point
##S10. :WAV:DATA? Read the data from the 125001th waveform point to the 250000th waveform point
##Perform the third reading operation
##S11. :WAV:STAR 250001 Set the start point of the third reading operation to the 250001th waveform point
##S12. :WAV:STOP 300000 Set the stop point of the third reading operation to the 300000th waveform point (the last point)
##S13. :WAV:DATA? Read the data from the 250001th waveform point to the 300000th waveform point (the last point)
# improved read data
##S1. :SING p. 19 - Set the oscilloscope to the single trigger mode
##S2. :TFOR p. 19 - Generate a trigger signal forcefully. This command is only applicable to the normal and single trigger modes (see the :TRIGger:SWEep command) and is equivalent to pressing the FORCE key in the trigger control area on the front panel.
##S3. wait for scope to finish
##S4. :WAV:SOUR CHAN1 Set the channel source to CH1
##S5. :WAV:MODE RAW Set the waveform reading mode to RAW
##S6. :WAV:FORM BYTE p. 239 - Set the return format of the waveform data to WORD|BYTE|ASCii (WORD: higher byte is always 0) - default: BYTE
def capture_data(instrument):
force_trigger = True
set_trig_swe = instrument.query(":TRIG:SWE?").strip() # query trigger sweep
print(f"set_trig_swe: {set_trig_swe}")
while set_trig_swe == "SING": # reset single trigger (defined state)
instrument.write(":TRIG:SWE AUTO")
wait_ready(instrument)
set_trig_swe = instrument.query(":TRIG:SWE?").strip()
while set_trig_swe != "SING": # single trigger
instrument.write(":SING")
wait_ready(instrument)
set_trig_swe = instrument.query(":TRIG:SWE?").strip()
print("measurement started")
# TODO refactor
# :TRIG:STAT? returns TD, WAIT, RUN, AUTO, or STOP
# RUN (Pre-Trigger measurement) - WAIT (for trigger, may not be read out if triggered by signal) - TD (Post-Trigger) - STOP (measurement finished)
#wait until measurement finished
start_time = time()
run_time = start_time
current_time = start_time
wait_ready(instrument) # even with wait_ready sometimes
status = instrument.query(":TRIG:STAT?").strip() # get trigger status: TD, WAIT, RUN, AUTO or STOP
prev_status = status
wait_count = 0
finished = False
while not finished:
if status != prev_status:
print()
run_time = time()
current_time = time()
if status in ("RUN", "TD"): # RUN/TD - measurement is runnning
print(f"{status} {current_time - run_time:7.3f}s ", end="\r")
elif status == "WAIT": # WAIT - waiting for trigger
print(f"{status} {current_time - run_time:.3f}s")
if force_trigger:
wait_ready(instrument)
instrument.write(":TFOR") # force trigger (may need multiple tries to start)
wait_count += 1
print(f'trigger forced {wait_count}x')
elif status == "STOP": # end on STOP (measurement finished)
finished = True
print(f"{status} {current_time - run_time:.3f}s")
else: # AUTO - should not happen
print(f"{status} {current_time - run_time:7.3f}s ", end="\r")
prev_status = status
status = instrument.query(":TRIG:STAT?").strip() # get trigger status: TD, WAIT, RUN, AUTO or STOP
#sleep(0.01)
current_time = time()
print(f"measurement finished in {current_time - start_time:.3f}s")
def read_data(instrument, ch: int = 1):
mem_depth = int(instrument.query(":ACQ:MDEP?")) # memory depth
# :WAV needs to be en bloc, at least between STAR & STOP
instrument.write(f":WAV:SOUR CHAN{int(ch)}") # data from channel 1-4
wait_ready(instrument)
instrument.write(":WAV:MODE RAW") # data from internal memory
wait_ready(instrument)
instrument.write(":WAV:FORM BYTE") # data as byte
data = []
start_time = time()
for interval in crange(1, mem_depth, 250_000): # 250_000 is max chunk size for bytes
#instrument.write(f":WAV:STAR {interval[0]}") # start index of memory
#instrument.write(f":WAV:STOP {interval[1]}") # stop index of memory
#data += (instrument.query_binary_values(f":WAV:DATA?", datatype='B')) # get the data, B = unsigned char
data += (instrument.query_binary_values(f":WAV:STAR {interval[0]}\n:WAV:STOP {interval[1]}\n:WAV:DATA?", datatype='B')) # get the data, B = unsigned char
print(f"{len(data)/mem_depth:3.0%} memory read in {time() - start_time:.3f}s ({len(data)}/{mem_depth}) reading: {interval[0]}:{interval[1]}", end="\r")
print()
# debug
print(mem_depth)
sps = float(instrument.query(":ACQ:SRAT?").strip()) # query the current sample rate in Sa/s
print(f"SPS: {ef(sps)}")
timebase_scale = float(instrument.query(":TIM:SCAL?").strip()) # query timebase scale
print(f"Timebase scale: {ef(timebase_scale)}s/div")
# debug end
# debug
#instrument.write(":WAV:FORM ASCii") # data as ASCII
#ascii_data = instrument.query(":WAV:DATA?") # get the data
#print("ASCII:")
#print(ascii_data[11:1000])
#print("ASCII END")
# debug end
return data
# scale data
# formula for converting in V (p. 242): scaled value = (value - YORigin - YREFerence) x YINCrement
##:WAV:YINC? # p. 244, return in scientific notation, see details
##:WAV:YOR? # p. 244, return as integer, see details
##:WAV:YREF? # p. 245, returns 127 always?, see details
##:WAV:PRE? # p. 246, returns 10 waveform parameters separated by ",", see details
def scale_data(instrument, data):
wait_ready(instrument)
mem_depth = Decimal(instrument.query(":ACQ:MDEP?").strip()).normalize() # memory depth
wait_ready(instrument)
wf_parameters = instrument.query(":WAV:PRE?").strip().split(",") # get waveform parameters: <format>,<type>,<points>,<count>,<xincrement>,<xorigin>,<xreference>,<yincrement>,<yorigin>,<yreference>
print(f"Waveform parameters: {wf_parameters}")
while Decimal(wf_parameters[2]).normalize() not in (mem_depth, mem_depth/2, mem_depth/3, mem_depth/4):
wait_ready(instrument)
wf_parameters = instrument.query(":WAV:PRE?").strip().split(",")
print(f"Waveform parameters: {wf_parameters}")
yincrement = float(wf_parameters[7])
yorigin = float(wf_parameters[8])
yreference = float(wf_parameters[9])
scaled_data = []
for byte in data:
scaled_data.append((float(byte) - yorigin - yreference) * yincrement)
return scaled_data
def save_data():
pass
def screenshot(instrument):
sleep(2) # wait until all messages are gone (e.g. can operate now)
start_time = time()
screenshot = instrument.query_binary_values(":DISP:DATA? ON,OFF,PNG", datatype='B') # ON,OFF,PNG - BMP is a bit faster
stop_time = time()
print(f"Screenshot took {stop_time - start_time:.3f}s")
return screenshot
def view_image(image):
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
img = mpimg.imread(image)
imgplot = plt.imshow(img)
plt.show()
def main():
#global measurement_filename
timebase_scale = "1" # timebase scale in s
ch1_scale = "1e-3" # set channel scale in units (V, A, W)
mem_depth= "24000000" # set memory depth in points: 12000, 120000, 1200000, 12000000, 24000000
data_path = r"D:\Dokumente\Software-Projekte\Rigol DS1000Z data acquisition"
data_dir = ""
data_ext = "csv"
data_file = "test"
scope = connect()
setup(scope, timebase_scale, ch1_scale, mem_depth)
# TODO get from capture_data and read_data as return
probe_ratio = Decimal(scope.query(":CHAN1:PROB?").strip()).normalize()
ch1_scale_ef = ef(Decimal(ch1_scale), places=0)
sps_ef = ef(Decimal(scope.query(":ACQ:SRAT?").strip()).normalize(), places=0)
acq_mode = scope.query(":ACQ:TYPE?").strip()
mem_depth_ef = ef(scope.query(":ACQ:MDEP?").strip(), places=0)
acq_time_ef = ef(Decimal(timebase_scale)*12, places=0)
for n in range(1, 2):
capture_data(scope)
data = read_data(scope)
print()
print(f"data length: {len(data)}")
#print(data[:100])
scaled_data = scale_data(scope, data)
print(f"scaled data length: {len(scaled_data)}")
#print(scaled_data[:100])
data_file_tail = f"x{probe_ratio} {ch1_scale_ef}V {sps_ef}SPS {acq_mode} {mem_depth_ef}pts {acq_time_ef}s"
data_file_full = f"{data_path}\{data_dir}\{data_file} {n:02} - {data_file_tail}"
import csv
with open(f"{data_file_full}.{data_ext}", "w", newline="") as f:
writer = csv.writer(f)
for item in scaled_data:
writer.writerow([format(item, ".4g")]) #to reduce the file size further, limit number to significant decimal digits / resolution (8bit/10div) depending on channel range
#import numpy as np
##np_sd = np.asarray(scaled_data)
#np.savetxt("D:\Dokumente\Software-Projekte\Rigol DS1000Z data acquisition\DS1000Z_DAQ.csv", scaled_data, fmt = "%.4g") #to reduce the file size further, limit number to significant decimal digits / resolution (8bit/10div) depending on channel range
print (f"{data_file_full}.{data_ext} written")
image = screenshot(scope)
#view_image(image)
with open(f"{data_file_full}.png", "wb") as f: # write screenshot to file
f.write(bytearray(image))
print (f"{data_file_full}.png written")
if __name__ == "__main__":
main()
--- End code ---
For the curious, the relevant parts of the wireshark logs are attached.
Navigation
[0] Message Index
[*] Previous page
Go to full version