Electronics > Beginners
help with rapid ADC data aquizition
jhpadjustable:
--- Quote from: powerfly on December 07, 2019, 08:04:07 pm ---Thanks, that sounds good. the thing with the arduino (and the STM32F103) is that you wouldn't be able to batch up lots of data before writing as their RAM is so small (2kB or 20kB). so whilst the old STM32F103 would be able to read the ADC at a high rate, how could it be storing the data at the high rate with 20s intervals? I mean, if I'm using 7 channels, 20kB would get used up in ~0.2 s. Am I misunderstanding something here?
--- End quote ---
You're missing nothing! You are absolutely correct. You would either have to write more often, like every 50-100ms or so still might be feasible, or choose another STM32 with more RAM (e.g. the STM32F3/STM32F4 range), or go with that external RAM for buffering with all that entails. I can't think of any STM32s that have worse specs on their ADCs than the F103, except that some in the STM32F0 range might come with only one unit instead of two. #justsayin on the chance you were making a bunch and wanted to push the cost down a bit.
--- Quote ---I'm planning on using a Teensy 3.6 anyway that has 256K (I think that means 256KB) of RAM so that should be able to write then dump to the micro SD card via the native port without a problem (I think).
--- End quote ---
That sounds fine. You'll have plenty of room to work with if you drop the write interval to about 1/2 second. How long did you intend to leave this device in operation for, and how critical is it that you not lose data? I'm getting a little concerned about burning out the flash from overuse, which could happen if you write the metadata block back too often. If you intend to run for months, you might think about pre-allocating a big file on the flash, using the file data section as a big ring buffer, and not writing to the metadata at all.
--- Quote from: rstofer on December 07, 2019, 08:28:17 pm ---This doesn't really matter because the ADCs could be external on an SPI or I2C bus and run through the DMA channel.
--- End quote ---
Why, they're free with the chip. :)
radiolistener:
--- Quote from: powerfly on December 06, 2019, 05:59:19 pm ---1. Are there any microcontrollers that would be able to write at these sorts of speeds straight to an SD card or memory stick?
--- End quote ---
yes, you can do it with STM32 microcontrollers. But they have different performance and different peripherals, so you're needs to choice right type of STM32 microcontroller.
--- Quote from: powerfly on December 06, 2019, 05:59:19 pm ---3. Is a microcontroller the best option?
--- End quote ---
yes
radiolistener:
--- Quote from: powerfly on December 07, 2019, 01:41:35 pm ---cheap integrated ADCs aren't usually 12 bit and are often not 10kHz either.
--- End quote ---
cheap STM32 microcontrollers have 1-2 ADC with 12 bit resolution which can work with sample rate up to 1 MHz. And about 8 channels with analog multiplexer.
That's enough to capture all 8 channels with 10 kHz sample rate and write it to sdcard. Such solution will needs to share 1-2 ADC between 8 channels. For example you can use first ADC to digitize odd channels and second ADC to digitize even channels.
rstofer:
--- Quote from: powerfly on December 07, 2019, 10:08:47 pm ---
--- Quote from: rstofer on December 07, 2019, 08:28:17 pm ---In theory, you can transfer multiple megabytes per second to an SD card depending on the model
https://www.expertreviews.co.uk/storage/1404380/how-to-choose-an-sd-card-class-and-speed-ratings-explained
But you probably need something faster than an Arduino to do it and the code probably needs to be tightened up A LOT! DMA and multiple buffers will help.
Horsepower matters! How about the Teensy 4? It is blazing fast (600 MHz) has two 12-bit ADCs along with 14 analog pins. This doesn't really matter because the ADCs could be external on an SPI or I2C bus and run through the DMA channel. Given a bunch of internal memory, creating packets for the SD card will be very fast.
https://www.pjrc.com/store/teensy40.html
Not that I would necessarily go there but there is an Arduino compatible library for the chip called TeensyDuino which can be installed under the Arduino IDE although it escapes me why anyone would voluntarily use that IDE. But the library installation includes the proper toolchain for the chip (ARM, not ATmega, obviously). Keep the library, moved to anything else for the IDE.
The ARM chip itself:
https://www.nxp.com/docs/en/nxp/data-sheets/IMXRT1060CEC.pdf
--- End quote ---
I have been discussing using either the teensy 3.6 or 4.0 (see above comments). the advantage of the 3.6 is that it has the built in SD card reader and the processor is probably powerful enough. The 4.0 does have the added power and RAM, and I guess adding a card reader wouldn't be too difficult (although rapidly storing data to the card may be more difficult?)
--- End quote ---
Have you looked at the Horsepower Chart - the 4.0 is more than 5 times as fast as the 3.6
https://www.pjrc.com/store/teensy40.html
Speed is good!
--- Quote ---When you say the code needs to tightened up a lot, how would one go about doing this?
--- End quote ---
I haven't looked closely at the TeensyDuino libraries but the stuff for the ATmega chip doesn't use any features like DMA or fancy interrupt drivers. Mostly because the chip doesn't have much.
You would want to be able to concurrently transfer values from DMA buffers to the SD card. In a perfect world, the buffers would be exactly the size of a disk block (512 bytes) and every time the DMA filled one it would generate an interrupt to tell the SD write code to transfer more data to the card.
I don't know if the TeensyDuino library has most of this code. It could because the 3.6 code might be quite similar but I wouldn't necessarily count on library code if I want raw speed over portability across platforms. Bare metal programming comes to mind.
--- Quote ---
jhpadjustable mentioned some stufff he thinks will help, although having someone point me in the right direction of how to do this would be helpful. I am very much a beginner when it comes to coding. the teensyduino library is appealing as it seems like it would be simple to code in.
--- End quote ---
I do have a couple of Teensy 4.0 boards but I haven't played with them yet. Some years back I used the RawHID code and a early Teensy to add switches, knobs and dials to Microsoft Flight Simulator using USB HID. The code worked well so I have always been a fan of the Teensy products.
Sometimes, internal ADCs don't have the required performance and that's why I said it might not matter because you could use the 3 internal SPI buses (with 16 byte FIFOs) to connect to external ADCs with better performance. I haven't looked to see that the SPI gadget connects through DMA but I'll bet it does. So, basically, all of the ADC reading and transfer can be done in the background. When a buffer gets filled, the code sets a flag to tell the mainline to transfer the buffer to the SD card. This too should be done by DMA so very little code execution is required.
There are many examples of using SD cards with one of the common FAT filesystems with the entire project running under FreeRTOS and while this is an advanced topic, it truly simplifies the code by cleaning up the inter-communications between reading ADCs and writing the SD. Semaphores and queues (or buffer) plus some handshake signals control everything.
But, given an excess of horsepower, start simple and see how fast you can go. If it meets your needs, there is no reason to dive deeper into the code.
However, having an SD adapter onboard makes the 3.6 very compelling. I might start there and if I need more horsepower, I know where to get it.
powerfly:
--- Quote from: jhpadjustable on December 07, 2019, 10:36:53 pm ---
--- Quote from: powerfly on December 07, 2019, 08:04:07 pm ---Thanks, that sounds good. the thing with the arduino (and the STM32F103) is that you wouldn't be able to batch up lots of data before writing as their RAM is so small (2kB or 20kB). so whilst the old STM32F103 would be able to read the ADC at a high rate, how could it be storing the data at the high rate with 20s intervals? I mean, if I'm using 7 channels, 20kB would get used up in ~0.2 s. Am I misunderstanding something here?
--- End quote ---
You're missing nothing! You are absolutely correct. You would either have to write more often, like every 50-100ms or so still might be feasible, or choose another STM32 with more RAM (e.g. the STM32F3/STM32F4 range), or go with that external RAM for buffering with all that entails. I can't think of any STM32s that have worse specs on their ADCs than the F103, except that some in the STM32F0 range might come with only one unit instead of two. #justsayin on the chance you were making a bunch and wanted to push the cost down a bit.
--- Quote ---I'm planning on using a Teensy 3.6 anyway that has 256K (I think that means 256KB) of RAM so that should be able to write then dump to the micro SD card via the native port without a problem (I think).
--- End quote ---
That sounds fine. You'll have plenty of room to work with if you drop the write interval to about 1/2 second. How long did you intend to leave this device in operation for, and how critical is it that you not lose data? I'm getting a little concerned about burning out the flash from overuse, which could happen if you write the metadata block back too often. If you intend to run for months, you might think about pre-allocating a big file on the flash, using the file data section as a big ring buffer, and not writing to the metadata at all.
--- End quote ---
If I loose data I can just re-record it, it's not a huge issue. as for how much it's going to be used, it'll probably be used in ~1hr intervals a couple of times, not continuously for months. I don't really mind too much a $5 card burns out every couple of months anyway, they are fairly cheap.
I plan on progamming this on the Arduino IDE as it seems the most simple for a beginner, do you recommend this?
A new development is that apparently as I'm trying to amplify this signal from a strain gauge-wheatstone bridge setup there will be an issue using multiple channel inputs to the ADCs. apparently noise is increased as when using multiple channels you are multiplexing, I was unaware of this, let alone it being a problem. I guess I could just try 1 channel and then multiple channels and see if there is a difference. (you don't need to read this but the link to the discussion is here if you are interested: https://www.eevblog.com/forum/microcontrollers/getting-gain-on-adc-signals/ )
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version