Raspberry Pi2/3 logging platform for Voltnuts

[z01z]

This Noobs 1.4.1 doesn't boot on my PI3. If you observe, the newer releases (I've checked the 2.1) have files like bcm27*, which seem to correspond to the different hardware versions. The 1.4 has no such files.

[bingo600]

This Noobs 1.4.1 doesn't boot on my PI3. If you observe, the newer releases (I've checked the 2.1) have files like bcm27*, which seem to correspond to the different hardware versions. The 1.4 has no such files.

Sigh ....  Sorry about that

A v2 would boot afaik

/Bingo

[FrankBuss]

USB tested OK, so both USB and LAN are fully functional on the Rigol DM3068.   I have the Rigol DM3068 scripts working.  Is there a GITHub I can do a pull request to get the DM3068 added.  Then other people can enjoy. 

plesa created this repository:
https://github.com/PlesaEEVBlog/RPi_LogNut
@plesa please accept my pull request for the initial content, so that it doesn't get out of sync for too long.

[Dwaine]

USB tested OK, so both USB and LAN are fully functional on the Rigol DM3068.   I have the Rigol DM3068 scripts working.  Is there a GITHub I can do a pull request to get the DM3068 added.  Then other people can enjoy. 

plesa created this repository:
https://github.com/PlesaEEVBlog/RPi_LogNut
@plesa please accept my pull request for the initial content, so that it doesn't get out of sync for too long.

Ok.  I'll wait until the Initial pull happens and then do my get/pull for the Rigol DM3068. 

[Dwaine]

I have not tried the http.server and I would like to see how easy it is to implement.

I just tried the webserver, now I can read the value of my benchtop multimeter which has only RS232, in a webpage and even from the internet, if I would add a NAT rule to my router 

https://github.com/FrankBuss/RPi_LogNut/blob/master/pub/python/HM8012_Webserver.py

How it looks like (the real version reloads automatically every second) :



Of course, this is just a quick hack. You would not create the web pages in the Python code, but state of the art is probably to use some WebSocket API, serve local static files and then some JavaScript communicates over the WebSocket with JSON requests with the Python script.

I will also do an example web server view for the Rigol DM3068.

[Dwaine]

The Rigol DM3068 running on the Raspberry PI logging.  Check it out  Give it an Internet test.....

http://99.232.17.239/DM3068_Logs.html

Dwaine

[TiN]

Nice

plesa

How do you create images? Just simple plain bit-bit clone of SDcard, or there is some tool?
I'd like to make few images too, for DE1-SoC FPGA board running linux-gpib. Got I2C working today with BME280 on it. Next step is GPIO and SPI, so I can control ADCs/DACs as well.

[plesa]

Nice

plesa

How do you create images? Just simple plain bit-bit clone of SDcard, or there is some tool?
I'd like to make few images too, for DE1-SoC FPGA board running linux-gpib. Got I2C working today with BME280 on it. Next step is GPIO and SPI, so I can control ADCs/DACs as well.

There are several ways.Resizing It tried but it is too complicated. I used bit clone and compress it by gzip/tar.

USB tested OK, so both USB and LAN are fully functional on the Rigol DM3068.   I have the Rigol DM3068 scripts working.  Is there a GITHub I can do a pull request to get the DM3068 added.  Then other people can enjoy. 

plesa created this repository:
https://github.com/PlesaEEVBlog/RPi_LogNut
@plesa please accept my pull request for the initial content, so that it doesn't get out of sync for too long.

OK, I will try tomorrow.

[bingo600]

Someone here claims to have linux-gpib running w. an agilent on a rpi v3.
http://www.messtechniklabor.de/artikel-h0014-raspberry_pi_3_model_b_mit_linux_gpib_und_kernel_4_1.html

Seems like he has the old a model not the later b adapter.

I did try to build it (1h20m for kernel) , on a kernel 4.4.34 platform. 
Build/boot ok , but same linux-gpib result ... timeout.


One correction though

Where he writes

#raspberry pi 2/3
KERNEL=kernel7

You need to either do an

Code: [Select]

export KERNEL=kernel7
Or maybe better replace $KERNEL.img on the below line with kernel7.img
sudo scripts/mkknlimg arch/arm/boot/zImage /boot/$KERNEL.img



My startingpoint was NOOBS-1.9 https://downloads.raspberrypi.org/NOOBS/images/NOOBS-2016-03-18/

But i did the distupgrade before build , that lifted the platform to kernel 4.4.34.
Maybe someone can try wo. the distupgrade, where it's platform 4.1.19 i think.

/Bingo

[z01z]

Did the procedure on 2016-03-18-raspbian-jessie (4.1.21 kernel), it doesn't work afterwards either.
Actually it seems to work less with this than the v2 image, with that at least the instrument detects that it is being addressed. With this, nothing. It can easily be though, that I did something wrong.

[FrankBuss]

@FrankBuss

SPD3303 is very slow at handling commands.

You need to put a sleep after the command or it get messed.

I use something like this:

siglent.write("INST CH2")
time.sleep(1)
siglent.write("CH2:VOLT 0")
time.sleep(1)
siglent.write("CH2:CURR 1.3")
time.sleep(1)
siglent.write("OUTP CH2, ON")
time.sleep(1)


you can also lower sleep to 0.1 seconds if you need more speedy setup.

I guess this is the same problem I found out when I use the USBTMC Linux kernel driver on my PC, and might be really a bad firmware on the power supply. I can lower the delay down to 10 ms and it works always. With 5 ms sometimes it doesn't work. So might be safe to wait 20 ms. The instrument itself seems to be fast. I can do a "CH2:VOLT 9" command and 10 ms later a "CH2:VOLT 8" command and it is set to 8 V.

But I still can't read anything reliably. Could you try this on your system?

Code: [Select]

print(siglent.ask("*IDN?"))

Sometimes it shows the device, but most of the time it generates an error: "Operation timed out". I tried to edit /opt/python-usbtmc/build/lib.linux-armv7l-2.7/usbtmc/usbtmc.py and insert a time.sleep(0.1) after line 629 in the ask function, between the write and read, and it looks like it helps a bit, but I'm still getting the timeout error often.

At least writing works with the delay. But would have been nice to get the current reading working, too.

[Karel]

Additional benefit of using the USBTMC kernel driver: you can use it from other programs as well, not just from Python. On my PC I can do this:

echo "*IDN?" > /dev/usbtmc0
cat /dev/usbtmc0

and it answers with

Siglent Technologies,SPD3303,SPD00002130137,1.01.01.01.05,V1.1

(but still with the timeout problem: after printing this line it hangs for about 5 seconds and then it says "cat: /dev/usbtmc0: Connection timed out")

The timeout "problem" isn't a problem at all but just how the usbtmc protocol is implemented.
The cat program keeps on reading till it receives an EOF.
Usbtmc protocol uses a newline character at the end of a message, not an EOF.
Cat ignores the newline character and will continue to ask for more data till the driver timesout which is after 5 seconds.
A program that is written to read from an usbtmc device will stop reading when it detects a newline character in the message.

[bingo600]

Guyzz

Dave Penkler (linux-gpib maintainer) , has ansvered my e-mail , and suggests to build linux-gpib w. debugging enabled.
./configure  --enable-driver-debug

He uses latest kernels ie. 4.9 , but only have i386 based machines.

So he lacks a Raspi v3 to debug on.


He'll borrow one

/Bingo

[cdev]

Two things.

1.) A *logging* platform should definitely log to a real hard drive because the process of logging inherently makes a great many individual writes which will destroy flash memory cards eventually. (For the same reason compiling software on them will, hopefully people know that)

Good practice would be to put the boot OS on the SD card and do all logging - also any compiles, should take place on a real HD which can be a small one. SD cards have improved a lot but this is still good practice. You should also use the real HD for swap.

Set the SD card to "noatime" so that a separate write is not done when files are accessed also. USB SATA enclosures can be had for as little as $5 and a small low current requirement SATA drive is similarly cheap. Two can be used for data redundancy using software RAID.(use a powered USB hub because the RPI can only supply a fairly modest amount of power to power other devices.)

2.) David Taylor has done a lot of research into running NTP on an RPI using a GPS and kernel 1pps to provide an accurate timing reference. (Most people here have probably already seen his web pages I'm sure but worth mentioning anyway as lots of useful info) . I recently saw a link there to this discussion about a small change  which reduces the latency of the onboard USB-based Ethernet. Seems worth looking into.

[FrankBuss]

The timeout "problem" isn't a problem at all but just how the usbtmc protocol is implemented.
The cat program keeps on reading till it receives an EOF.
Usbtmc protocol uses a newline character at the end of a message, not an EOF.
Cat ignores the newline character and will continue to ask for more data till the driver timesout which is after 5 seconds.
A program that is written to read from an usbtmc device will stop reading when it detects a newline character in the message.

Ok, but why does it work sometime and sometimes not with pyusb? And the USBTMC kernel driver works always, if I use the code from here, but I guess "os.read(self.FILE, 4096)" doesn't wait for EOF, but reads all currently available data. So probably the pyusb implementation has some other problems, in combination with the Siglent instrument, because the HP gear seems to work, probably with the Linux kernel driver, too. So might be a good idea to use the Linux kernel driver instead of the Python implementation, if there are no problems with the HP devices.

[FrankBuss]

1.) A *logging* platform should definitely log to a real hard drive because the process of logging inherently makes a great many individual writes which will destroy flash memory cards eventually. (For the same reason compiling software on them will, hopefully people know that)

Good practice would be to put the boot OS on the SD card and do all logging - also any compiles, should take place on a real HD which can be a small one. SD cards have improved a lot but this is still good practice. You should also use the real HD for swap.

Another idea is to use a network drive. I have a NAS (with RAID0) and copied the /pub directory to it. Then I mounted this directory from my Raspberry Pi, now doing some long time logs for the Batteroo test. But probably not a good idea to use swap over ethernet. Best would be to disable swap and mount the main filesystem read-only, using the rest from the NAS.

Some time ago for another platform I could even load the Linux kernel over network with u-boot, and then even mount the root filesystem over NFS. This is really helpful for developing, because then you don't have any size limitations of the platform and testing new kernel versions is faster, and it can't brick, if you backup the last kernel (and modules).

[plesa]

1.) A *logging* platform should definitely log to a real hard drive because the process of logging inherently makes a great many individual writes which will destroy flash memory cards eventually. (For the same reason compiling software on them will, hopefully people know that)

Good practice would be to put the boot OS on the SD card and do all logging - also any compiles, should take place on a real HD which can be a small one. SD cards have improved a lot but this is still good practice. You should also use the real HD for swap.

Another idea is to use a network drive. I have a NAS (with RAID0) and copied the /pub directory to it. Then I mounted this directory from my Raspberry Pi, now doing some long time logs for the Batteroo test. But probably not a good idea to use swap over ethernet. Best would be to disable swap and mount the main filesystem read-only, using the rest from the NAS.

Some time ago for another platform I could even load the Linux kernel over network with u-boot, and then even mount the root filesystem over NFS. This is really helpful for developing, because then you don't have any size limitations of the platform and testing new kernel versions is faster, and it can't brick, if you backup the last kernel (and modules).

Raspbian has swap disabled. Booting form network should be possible ( U-Boot on RPi is running ), but it will limit majority of potential users. It will be too difficult for them to setup DHCP and PXE.
I will check the USBTMC kernel implementation. I do not accoutered any issue, but I was using only HP/Agilent and Keysight gear over USB.

[mimmus78]

@FrankBuss - or any siglent power supply user

I use following sequence to initialise the thing.

siglent = usbtmc.Instrument("USB::0x0483::0x7540::INSTR")
siglent.clear()
time.sleep(0.1)
siglent.write("*IDN?")
time.sleep(0.1)
print (siglent.read())
time.sleep(0.1)

This works 100% for me, I have a complicate process working for days and programming the V of power supply every second without any problem.
Only when you stop the program sometime the power supply don't work at first connection, in this case just stop python program and start again (or maybe just close USBTMC connection and reopen).

You cannot use ask because (I didn't checked, but I think) ASK is making WRITE and READ one after the other and this will confuse the thing.
So to make it work just use [write, sleep, read] sequence.

@FrankBuss

SPD3303 is very slow at handling commands.

You need to put a sleep after the command or it get messed.

I use something like this:

siglent.write("INST CH2")
time.sleep(1)
siglent.write("CH2:VOLT 0")
time.sleep(1)
siglent.write("CH2:CURR 1.3")
time.sleep(1)
siglent.write("OUTP CH2, ON")
time.sleep(1)


you can also lower sleep to 0.1 seconds if you need more speedy setup.

I guess this is the same problem I found out when I use the USBTMC Linux kernel driver on my PC, and might be really a bad firmware on the power supply. I can lower the delay down to 10 ms and it works always. With 5 ms sometimes it doesn't work. So might be safe to wait 20 ms. The instrument itself seems to be fast. I can do a "CH2:VOLT 9" command and 10 ms later a "CH2:VOLT 8" command and it is set to 8 V.

But I still can't read anything reliably. Could you try this on your system?

Code: [Select]

print(siglent.ask("*IDN?"))

Sometimes it shows the device, but most of the time it generates an error: "Operation timed out". I tried to edit /opt/python-usbtmc/build/lib.linux-armv7l-2.7/usbtmc/usbtmc.py and insert a time.sleep(0.1) after line 629 in the ask function, between the write and read, and it looks like it helps a bit, but I'm still getting the timeout error often.

At least writing works with the delay. But would have been nice to get the current reading working, too.

[Dwaine]

Ummmm.  I did not use any sleeps in my Rigol DM3068 python script.  Right now I'm letting the USB connection run continuously.   So far no problems with the python USBTMC. 

http://99.232.17.239

Dwaine

[Dwaine]

I have a Keysight u1272a meter. Would anyone object if I submitted the python, HTML, web server display for it?

Dwaine

[cdev]

Okay, I realize this is likely stuff people here already know about. But it cant hurt to remember it.

I wanted to encourage you to consider using a real hard drive on your logger from the start.. You cannot argue with the price of these "Pi Drives" - These hard drives are designed to be used with the Raspberry Pi. They incorporate a USB to SATA converter and an additional USB plug for bringing in additional power.

http://wdlabs.wd.com/category/wd-pidrive/

But if you must use flash memory, one way to optimize logging to SD cards for flash memory longevity is to cache and delay writes for several seconds, then do the writes at the same time, as one. That will reduce the raw number of writes. Of course, then if you lose power you will have lost the last few seconds of writes. A good search term to learn more is "wear leveling".  Another option is doing your writing to an NFS mounted share on another computer.

There is no reason not to simply be careful. You could run a cron job to back up your SD card every few hours.

That said, I usually only hook up a real hard drive to my Raspberry Pis when I want to compile something.

 I usually just use the SD normally and set the options on software to only log the needed data and errors and not all diagnostic messages.  Most pre-built software on Raspbian should use sensible defaults for flash memory, which will be different than the ones used on desktops and laptops.
On programs that write big logs by default, you have to tell them to log less or eventually your SD cards will need replacement. Luckily its usually possible to get most of the data off them.

They get significantly better every year at handling lots of writes. But still there is a huge level of variability in SD card quality. Bunnie Huang has written extensively on this. You might want to check out his web site.

You cant treat an SD card as if its a hard disk or even an SSD.

[Dwaine]

Ummmm.  I did not use any sleeps in my Rigol DM3068 python script.  Right now I'm letting the USB connection run continuously.   So far no problems with the python USBTMC. 

http://99.232.17.239

Dwaine

I have the KeySight U1272A example completed.   I actually have both the Rigol DM3068 and KeySight U1272A running concurrently taking measurements on the same raspberry PI.  No issue to report.   Both devices are connected by USB...  I wonder how many devices we could hook up to the raspberry pi before it becomes a problem?

Dwaine

[Karel]

Ok, but why does it work sometime and sometimes not with pyusb?

Probably because of

... the pyusb implementation has some other problems, ...

[mimmus78]

Ummmm.  I did not use any sleeps in my Rigol DM3068 python script.  Right now I'm letting the USB connection run continuously.   So far no problems with the python USBTMC. 
http://99.232.17.239
Dwaine

This come out from some Siglent firmware developer comment on some support request.
It's not driver but this Siglent power supply firmware that has this problem.

[Dwaine]

I think all these companies should invest in the software side of things.  It really does make the hardware less useful when the firmware/software has problems.  I was reading the thread on the Rigol DP832 power supply rebooting during remote controlling and measuring.  It makes the power supply useless for some applications now. 

If they did firmware updates on a monthly or bi-monthly release.  It would help their product lines.

Dwaine