How to process data?

[SigrokLover]

Hi,

I'm a last year student in engineering in electronics and I'm working on a final project. I'm struggling at the moment .The goal of my project is to read some SPI lines (MISO, MOSI, CLK and several SS (slave selects)) and some analog signals. I attached a bloc diagram to better understand what I want to do.

Bassically I have a connector with some digital signals and analog signals.

• The idea is to deal with the analog signals with an Arduino. I will digitalise the analog signal into bunch of bytes in the Arduino and send those bytes in the logic Analyser.
  
  
• For the SPI lines, I would directly send them in the logic analyser.

Once this is done, the logic analyser does its job and sample the digital signals and send them to the PC.

Now, I can have the data exported in some external file in the PC and I can decode the information send in the SPI bus (typically, according to the Slave selected, I will make sense of the information (it can be the value of a temperature, a voltage,...)). This will have to be done continously.

The logic analyser I'm using, the Dslogic Plus, is compatible with Sigrok. Sigrok is an open source softaware that could be used to decode the SPI protocol and save the data on a file (.txt,.csv,..). Typically, with sigrok-cli, i could use the following command :

• sigrok-cli -c "samplerate=10k:voltage_threshold=2.5-2.5:continuous=on" --driver=dreamsourcelab-dslogic --continuous

The final goal of the project is to display the differents information in a GUI. I attached an example of what should be the final result. It's just to give you an idea because GUI is quite a generic term. I was thinking about Java because it's the simplest way of doing a GUI.

So, here is my main issue : how to use correctly sigrok to exploit the data and display them on a GUI? Is there a way I could use only java where I could initiate the reading of the logic analyser using some line of codes and process the data just after on the same code? Or the only solution is to save the data on external file and then process the data on java and display it on the GUI.
The thing is that I'm a bit confused on the way I could do the is task.

The last requirement is to have some log file where I store every second all the data that are beeing displayed on the GUI.

I hope I was clear enough. If I miss some important information, don't hesitate, I'm really stucked at the moment :/

Thanks in advance for the answers!

[ebastler]

Hmm, your data flow seems a bit awkward. Are you sure you want to approach it like this? Is the choice of hardware components already a given, or can you choose something else?

It seems that the logic analyzer has 16 inputs. According to your block diagram, you need 12 for your SPI and CS lines. Are the remaining 4 sufficient for your digitized analog data? How do you want to transfer these data -- parallel, serial? How many analog channels, and how do you multiplex them?

Also, the two separate devices (Arduino and logic analyzer) are not very elegant. The Arduino -- maybe a version with a few more pins -- could certainly capture both, the analog and SPI signals, and decode SPI. You would need to define a different PC interface then, of course. Some Arduinos support USB nicely.

Finally, I don't think SigRok is the right platform for defining a complex custom GUI like the mockup (?) you have attached. Not sure whether there is any way of defining your own GUI from a user level in SigRok (i.e. short of changing the SigRok source code). LabView is a typical tool for this, but you would need to select an interface supported by LabView. Maybe others have additional proposals for suitable software.

Edit: You have not indicated the required data rates. What is the net data rate on the SPI line, and what are the sampling rates required for the analog signals? What update rate for the display, and how many data point per second have to go to the log file?

[SigrokLover]

Thank you for your answer!!!


Hmm, your data flow seems a bit awkward. Are you sure you want to approach it like this?

The thing is that I didn't find a logic analyser that could deal correctly with both analog and digital signal. Also, the number of channel is quite high so I choose the solution of dealing with the SPI separatly from the analog signals.  I did commit a lot on this solution and I already bought the Dslogic so I must find a way to make it work now I guess. Do you think I'm going in the wrong direction? Or that I could choose a simpler path? Because I though it was possible to do it that way but I could be wrong of course.


Is the choice of hardware components already a given, or can you choose something else?

The final solution must work, that's the main requirement.

Are the remaining 4 sufficient for your digitized analog data? How do you want to transfer these data -- parallel, serial? How many analog channels, and how do you multiplex them?

I was thinking about sending them from the Arduino in serial  (the digitalWrite function of the Arduino seems to be a correct solution). So, the 4 remaining are enough. And since I would know how I did encode the data, it should not be difficult to decode it afterwards.

Also, the two separate devices (Arduino and logic analyzer) are not very elegant

I understand your point, but the solution doesn't seem to bother my professor. The final solution must work, that's the main requirement.


Finally, I don't think SigRok is the right platform for defining a complex custom GUI like the mockup (?) you have attached. Not sure whether there is any way of defining your own GUI from a user level in SigRok (i.e. short of changing the SigRok source code). LabView is a typical tool for this, but you would need to select an interface supported by LabView. Maybe others have additional proposals for suitable software.

The main reasons I want to use Sigrok was because it can decode the SPI directly and export the data on a file quite easily. Once the file is written, I could process it and display it on a java-based GUI. That was my main idea. But the idea of written on a file by using sigrok-cli and then opening it with java to read and process the data didn't seem optimal.

Then, I saw some API binding for java (I have some basics in programming but when it comes to API and stuff like that, I'm quite lost :/) and thought it was possible for me to by-pass sigrok-cli (https://www.sigrok.org/api/libsigrok/unstable/bindings/java/) . That way, I would have a piece of code that is only based on java so that I don't have to write in some file and process it afterwards, but directly process it. And still use some useful function like the decoding one.

The GUI I attached was  just an example I found on google to give you an idea of the final product I have to achieve.  The solution of labView is not an option according to my professor. (I think he doesn't like labView for some reasons). So, a solution based on LabView is not possible unfortunately.

You have not indicated the required data rates.

You're right!
For the SPI, the maximum data rate is 4MHz and the logic analyser can sustain until 20Mhz sampling when the 16 channels are used.
For the analog data, they are all DC. I must sample their value every second. It's not the critical part I think.
The update rate for the display is every second. The data I record is the effective information which has been decoded. For example, if a temperature measurement is sent throught the SPI lines, I must only record the value of the temperature in Celsius, not the bits. So, all the informations displayed will have to be stored in some log file.

[shteii01]

I had something similar for my senior design project, so same as you just a few years earlier.

It was cooling system. 
- I had three thermocouples for high temperature connected to special boards that digitized the temperature and sent it to Arduino board using SPI.  (Useful trick in case you don't know.  You can use same data and clock lines for all your SPI devices, only SS lines need to be separate.)
- I had analog temperature sensor (TMP36 or 35) for ambient temperature.  Also connected to Arduino board.

Here is what we did. 
- The Arduino board we used was 3.3V system so all signals (analog and digital are 3.3V max.)  The SPI devices were selected to match this system (3.3V digital signals).  They fed directly to Arduino board. 
- The analog sensor also matched Arduino board system.  The output of the analog sensor was fed to Arduino board ADC, there it was digitized.
- At this point all the data is digital.  No need to waste your time on logic analyzer.
- I found a neat macro that works with M$ Excel.  The data from Arduino board is sent to PC using USB and fed directly into Excel spreadsheet.  In Excel you do your analysis and feed your results to other software.

Software: https://medium.com/@islamnegm/quick-start-to-simple-daq-system-using-plx-daq-excel-arduino-d2457773384b

[ebastler]

Thanks shtei01. That’s roughly what I had in mind as well.

[SigrokLover]

Thanks shtei01 for your answer!

I didn't know that the Arduino could handle several SPI device at the same time. I've also looked the excel macro and I was surprised we could linked excel to an Arduino that easily!

About your project, I think it's quite different from what I want to do. Correct me if I'm wrong but in your project you had an Arduino where the three thermocouples was using the MISO, MOSI, CLK line of the Arduino and 3 SS pins was defined, one for each thermocouple (6 SPI lines in total) and two analog inputs of the Arduino were used for ambiant temperature. So, the Arduino was bassically the master of the communication and was requesting the temperature from the thermocouples (so it was in charge of setting the clock,...) and was sampling the ambient temperature when needed.

In my projet, the SPI communication is between a microcontroller and 9 modules. I do not control the communication. The goal of my project is to "sniff" the data that are sent between the microcontroller and the different modules. So, the EPS connector is there to obtain a monitoring of the status (different voltages, temperatures,... for example 3 modules are just ADCs sending data to the microcontroller that saves it in its own memory) through what is being sent throug the SPI lines, not to control the communication.

So, what you are suggesting is that I could use an Arduino and set all the necessary pin in readingMode and write the Arduino code to trigger when a slave is selected, tranform the bit received in hexadecimal for example,... and then send the data using the macro you suggest. But, the logic analyser can do all this so I don't think it's interresting to rewrite the code in Arduino if the sigrok software can do it. And still even if I'm using an Arduino, the problem will remain the same because I would need to directly link the Arduino to a java-based GUI.
To be clear, my main issue is to avoid saving the information in a file and then having to open it with an other software to display it on a GUI.

But, I think you are saying that if I manage to have a file where I can save all the information and then feed this information into the software that will be in charge of the GUI, it's a good solution. I will certainly do that as a last resort.

Thanks again for your answer, it's very nice of you.

I also want to share something. When doing some research, I have something similar to what I want to do : https://blog.savoirfairelinux.com/en-ca/2014/logic-analyzer-visualize-latency-with-sigrok-and-matplotlib/

Maybe the way to approach this would be to modify the SPI decoder of sigrok to directly handle the data before being saved and exported . Then, I will just have to display it on a GUI.
The question is : The thing he is doing whit matplotlib could be generalized to a Python GUI ? I don't know I'm clear enough, please let me know if it's not the case.

[HalFET]

Hmm, your data flow seems a bit awkward. Are you sure you want to approach it like this?

The thing is that I didn't find a logic analyser that could deal correctly with both analog and digital signal. Also, the number of channel is quite high so I choose the solution of dealing with the SPI separatly from the analog signals.  I did commit a lot on this solution and I already bought the Dslogic so I must find a way to make it work now I guess. Do you think I'm going in the wrong direction? Or that I could choose a simpler path? Because I though it was possible to do it that way but I could be wrong of course.

Have you looked at the National Instruments DAQ cards or modules? Some of the PCI cards should be able to do this without too much issues, and they can do synchronous sampling making this a trivial task. Think the data rate might be a bit high to sustain on the Analog Discovery kits sadly.

Are the remaining 4 sufficient for your digitized analog data? How do you want to transfer these data -- parallel, serial? How many analog channels, and how do you multiplex them?

I'm not sure what the capabilities of this logic analyser are, but I recall we used the Agilent ones with a pattern generator module to drive a simple ADC and had it run off a clock generated by the LA which was in sync with the sampling. So if it supports pattern generation or at least clock signal generation you might be able to force a sync between the two. This is especially fun when combined with flash ADCs.

The GUI I attached was  just an example I found on google to give you an idea of the final product I have to achieve.  The solution of labView is not an option according to my professor. (I think he doesn't like labView for some reasons). So, a solution based on LabView is not possible unfortunately.

LabVIEW code is difficult to manage if it's been worked on by non-experienced users, so whatever students work on is usually nearly impossible to decipher. I had the "joy" of supervising LabVIEW lab sessions, and I can't recommend the experience to anyone. LabVIEW is great if you know ahead of time what you're going to make and have it worked out completely and make use of module definitions and some pieces of C code in the right spots. But lets be honest, when does that occur?

[shteii01]

Regarding SPI.
In my project we were simply monitoring temperature.  In this case the SPI devices (https://www.adafruit.com/product/269) were sending sensor readings to the master (Arduino Due).  So.  I needed MISO, CLK, SS.  Since master never sent anything to the devices, I did not use MOSI at all.  All three SPI devices were using same MISO and CLK, each device had its own SS line (SS1, SS2, SS3).  For each SS you can simply use any digital pin, your SS1 can be Digital Pin X, SS2 is Digital Pin Y, etc.  Turn the pin On to select the device, device start sending data to master.  Turn the pin Off, device stop sending data, MISO and CLK are now clear for next device.

Regarding project.
You are correct that our projects have different scopes. 
The scope of my project was to monitor external input (heat) to a known system.  Meaning, our team knew exactly how much heat we were going to get (minimum and maximum), monitor for changes, based on the changes adjust coolant flow through radiator and adjust speed of the radiator fan.  We were in complete control of the system.  Your project seem to be more exploratory.  You don't know what is being communicated.  So you need to figure out first what is being sent (logic analyzer), then display it on PC.  I did not realize that you did not have the complete control.