On error resume next in Python from concurrent temperature sweeps and measuremen

[aronake]

Hi,

I use an Agilent E5810A to control a number of instruments. Control programming usually done in Python with vxi11.

It is not too uncommon I run one python program to collect measurements and one to control temperature sweeps. Sometimes a couple of python programs to do some different measurements.

Once in a while I get Error vxi11.vxi11.Vxi11Exception: 15: IO timeout [read] and a program stops. I suspect reason for this is that two communications on the GPIB bus collide.

How can this be solved?

A not so elegant solution would be to just to on error resume next. This could mean that i miss a temperature step once in a while or collection of a data point. typically this is not so critical. How is this done in python?

Or is there some on error retry that is not too complicated to program?

Or other better solution to this problem?

[voltsandjolts]

To ignore that exception, maybe...

Code: [Select]

try:
    something
except Vxi11Exception:
    pass


[alm]

Indeed, as voltsandjolts shows, by catching the exception and either doing nothing or printing a warning.

A better solution would be to prevent the error from occurring. In the case of timeouts the solution is either to increase the vxi-11 timeout or make the commands take shorter. For increasing the timeout, if you're using the Python VXI-11 module directly (as opposed to through PyVISA or python-ivi), something like:

Code: [Select]

instrument.timeout = 60
Should help.

To make commands faster it can help to switch from waiting until a slow command is finished to starting the command, then periodically checking the status registers if the command is done, and only fetching the results when you know they're ready. Alternatively triggering the command, then sleeping for some time, and then fetching the results can also work. For this you need to study the specific instrument if it supports that. SCPI DMMs do via initiate and fetch? commands.

[aronake]

To ignore that exception, maybe...

Code: [Select]

try:
    something
except Vxi11Exception:
    pass


Hi! Thanks! This is the correct answer to my question, but what i forgot to write is that i would want some way to deal with any except Vxi11Exception in a certain way (some initial declaration or something), not having to add code for each case where there is a Vxi11 read or write. Any way to do this, or Python do not allow for such lazy and bad programming?

[aronake]

Indeed, as voltsandjolts shows, by catching the exception and either doing nothing or printing a warning.

A better solution would be to prevent the error from occurring. In the case of timeouts the solution is either to increase the vxi-11 timeout or make the commands take shorter. For increasing the timeout, if you're using the Python VXI-11 module directly (as opposed to through PyVISA or python-ivi), something like:

Code: [Select]

instrument.timeout = 60
Should help.

To make commands faster it can help to switch from waiting until a slow command is finished to starting the command, then periodically checking the status registers if the command is done, and only fetching the results when you know they're ready. Alternatively triggering the command, then sleeping for some time, and then fetching the results can also work. For this you need to study the specific instrument if it supports that. SCPI DMMs do via initiate and fetch? commands.

Removing the source of the error is a very good advice!

I am using non-blocking reads for anything that takes longer time, like collecting average of 50 reading at 10 NPLC on 3458a by checking read_stb with some suitable recurrence say every 10th second. Other things i do typically are very quick things, like reading TEMP? on the 3458a or setting temperature or reading temperature of power on an ILX lightwave tec controller. Then occasionally on reads this error happens. And the higher frequency I read or write things from two python programs at the same time, the more likely these errors are to occur.

So it seems to me it is not that the GPIB bus is blocked that cause the error, which could be solved with longer time outs (or better use of the GPIB bus) but that the two python programs issue a GPIB transaction at the same time and they collide. I wonder how the E5810 responds and how python VXI11 handle the case of a busy bus and a new transaction initiated from python. I would have thought the E5810 would have responded with bus busy, and the python VXI11 would just have waited until bus available, but it may be that it then just throws an error.

[voltsandjolts]

To ignore that exception, maybe...

Code: [Select]

try:
    something
except Vxi11Exception:
    pass


Hi! Thanks! This is the correct answer to my question, but what i forgot to write is that i would want some way to deal with any except Vxi11Exception in a certain way (some initial declaration or something), not having to add code for each case where there is a Vxi11 read or write. Any way to do this, or Python do not allow for such lazy and bad programming?

Write your own read and write functions, which do nothing more than have that exception wrapper in them.

[alm]

I am using non-blocking reads for anything that takes longer time, like collecting average of 50 reading at 10 NPLC on 3458a by checking read_stb with some suitable recurrence say every 10th second. Other things i do typically are very quick things, like reading TEMP? on the 3458a or setting temperature or reading temperature of power on an ILX lightwave tec controller. Then occasionally on reads this error happens. And the higher frequency I read or write things from two python programs at the same time, the more likely these errors are to occur.

I have been running multiple parallel scripts (obviously not for the same instrument) on VXI-11 a lot.Mostly using HP E2050A but also E5810A. I do increase the timeout especially with the 3458A on the bus, but it's extremely rare that I get an error. The Keithley 2000 might give an error with a garbled response once every week or so (so I run long running scripts in a while true loop in the shell that keeps invoking the script), but I blame this on flaky Keithley GPIB implementation. I haven't seen this with HP or Datron / Wavetek equipment.

[Microdoser]

Personally, I would always log the specific exception with a timestamp before moving on with the rest of the code, instead of just simply passing. Then you can check the log later and look at exactly what happened and when, and try to find ways of stopping it happening again. If nothing else, you have a record of how often it happens.

IMO It's always better to stop errors from happening in the first place if possible, than to blindly pass and carry on. Sure, your program will work, but it's not the ideal solution.

[aronake]

I am using non-blocking reads for anything that takes longer time, like collecting average of 50 reading at 10 NPLC on 3458a by checking read_stb with some suitable recurrence say every 10th second. Other things i do typically are very quick things, like reading TEMP? on the 3458a or setting temperature or reading temperature of power on an ILX lightwave tec controller. Then occasionally on reads this error happens. And the higher frequency I read or write things from two python programs at the same time, the more likely these errors are to occur.

I have been running multiple parallel scripts (obviously not for the same instrument) on VXI-11 a lot.Mostly using HP E2050A but also E5810A. I do increase the timeout especially with the 3458A on the bus, but it's extremely rare that I get an error. The Keithley 2000 might give an error with a garbled response once every week or so (so I run long running scripts in a while true loop in the shell that keeps invoking the script), but I blame this on flaky Keithley GPIB implementation. I haven't seen this with HP or Datron / Wavetek equipment.

Rather than just concluding there has been an error I now keep track of which piece of equipment is involved in the error. Over the last two pretty measurement intensive days, there has been two errors. Both involving the ILX Lightwave LDT-5980 Tec Controller. So may be that error is caused by some bad GPIB implementation in this, and that it does not lock the bus and this cause the errors. I will keep looking if there is some other equipment that cause errors in the comming days. The equipment i run now is 5 X 3458a, one 34790a for switching, the ILX lightwave and a DMM6500 (on lan, so not GPIB)

[aronake]

To ignore that exception, maybe...

Code: [Select]

try:
    something
except Vxi11Exception:
    pass


Hi! Thanks! This is the correct answer to my question, but what i forgot to write is that i would want some way to deal with any except Vxi11Exception in a certain way (some initial declaration or something), not having to add code for each case where there is a Vxi11 read or write. Any way to do this, or Python do not allow for such lazy and bad programming?

Write your own read and write functions, which do nothing more than have that exception wrapper in them.

This seems to be the way forward. And add retry functionality and include error logging in these functions.

[miro123]

You already pinpoint the root cause of the problem. It is concurrence of VXI11 transactions from concurrent running processes.
My solution - I have chosen to use RAW socket instead of any high level protocols.
Advantage - it works is all Ethernet devices so far.includion Prologix ethern dongle.
Disadvantage - Requires basic software knowledge of writing multythreaded application.
Here is an example init function - is is part of bigger class
socket_tx_function and socket_rx_function are two thered that handle the two way socket communication. Building a semaphore in both function avod any concuancy on ether socket level. Time  out with error reporting is good practice to avoid deadlocks or network glitches
      
   def init_eth(self):
      HOST='10.0.0.89'   #prologix dongle . ToDo crete self discovery Prologix function
      PORT=1234
      buf_len=10
      self.s=socket.socket(socket.AF_INET,socket.SOCK_STREAM,socket.IPPROTO_TCP)
      a=self.s.connect((HOST,1234))
      self.eth_rx = threading.Thread(target=self.socket_rcv_function)
      self.eth_tx = threading.Thread(target=self.socket_tx_function)
 
      time.sleep(0.3) # start prologix specific code
      self.eth_rx.start()      
      self.s.sendall(b'++addr 9\r\n')
      time.sleep(1)
      self.s.sendall(b'++auto\r\n')
      time.sleep(1)
      self.s.sendall(b'Read?\r\n')
      time.sleep(1)
      self.s.sendall(b'Read?\r\n')
      time.sleep(1)      
      self.eth_tx.start()

[alm]

You already pinpoint the root cause of the problem. It is concurrence of VXI11 transactions from concurrent running processes.

This works just fine for me on Agilent E5810A and HP E2050A. Concurrent commands wait until the GPIB bus is free again, unless you reach the timeout. What hardware did you have the problems with?

[aronake]

You already pinpoint the root cause of the problem. It is concurrence of VXI11 transactions from concurrent running processes.

This works just fine for me on Agilent E5810A and HP E2050A. Concurrent commands wait until the GPIB bus is free again, unless you reach the timeout. What hardware did you have the problems with?

Thanks for confirming. Thats how I thought it was supposed to work out.

I previously just took notice there was an error happening and not what device. But since starting to dig into this, it seems all error are related to an ILX Lightwave tec controller. So my suspicion is that thisone have a not to great GPIB implementation

more here:
https://www.eevblog.com/forum/metrology/on-error-resume-next-in-python-from-concurrent-temperature-sweeps-and-measuremen/msg5314045/#msg5314045