Microcontroller Development Advise

[mawyatt]

New to microcontroller development and not that proficient in software/computer development in general, altho we have done some work with RPi and Python utilizing I/Os to drive various controls, and precision DACs and read back from precision ADCs and have used the SPI for some work. So basically a newb when it comes to microcontrollers, altho did play around with Arduino a little.

So we are looking for a development platform to develop a relatively simple microcontroller than can produce some specific I/O in the form of pulses and possible read back a few logic signals and maybe some analog input.

What we need the microcontroller to do is create a set of 8 sequential non-overlapping pulses of varying width (0.5 to 5ms), at a rate of ~4Hz to ~80Hz selectable. Another output is a selectable timer from ~60sec to ~1800sec. Everything is ~6 bit resolution, except the analog input at ~8bit resolution.

For UI a rotary dial, with a couple push buttons (Start, Stop, Select) and medium resolution small LCD. No need for USB or other type interfaces, or the need to upload firmware by the end user, and not concerned about protecting the design or code either, as it's going to be trivial anyway.

Per item/chip cost isn't an issue as these will likely be less than few hundred total units in production, but the main point is a good, simple, easy to learn and use development platform to work from.

We have accounts at Adafruit, Newark, Mouser, Digikey, and a few popular other sources.

Anyway, looking for advice from those microcontroller gurus that are well versed in this technology sector (read actually microcontroller developers), and please not the usual rhetoric from those no valued added folks.

Thanks in advance for any guidance/help provided.

Best,

[nctnico]

If you don't want to get deep into programming and cost is not an issue, then look at a PLC with HMI. Your requirements are well within the realm of what is possible using a PLC and these have straightforward to use development tools.

Otherwise just use an RPi compute module with some controls bolted onto it. Use Python + Qt to build a GUI.

Creating a GUI that works reasonably on a microcontroller with less resources means going through the learning curve of programming C, dealing with graphics libraries, implement a means to deploy updates, connectivity, etc.

[mawyatt]

The RPi is the option we've been considering since we are familiar with them. Just need to create a setup that boots up upon power application.

Will take a look at PLC, hadn't considered those, thanks!!

Best,

[nctnico]

Making something start automatically on a Linux system is not super difficult using systemd. If a device has a GUI, there has to be a dependence on the graphics system running with having the user logged in automatically.

Random Google find:
https://www.makeuseof.com/what-is-systemd-launch-programs-raspberry-pi/

[eugene]

I would not recommend a Raspberry Pi if you need precise control of timing. Maybe a RPi Pico....

From your description, it sounds like almost any microcontroller can do what you want, even the simplest Arduino or Arduino-like development board. To save development time, you might as well program it as an Arduino and take advantage of libraries to control the LCD, though that wouldn't be hard to do bare metal either. I'm fond of Teensies from pjrc.com, but I also bought a bag of Pro Micro clones on Aliexpress for something like $2 each. They would be more than adequate - and probably easier to make work than a Raspberry Pi.

[woofy]

.. altho did play around with Arduino a little.

.. but the main point is a good, simple, easy to learn and use development platform to work from.

There you go then, you've answered your own question.
I would have thought almost any arduino + an spi lcd module will do the job.

[NorthGuy]

LCD is obviously the biggest part to handle. That's huge variety here ranging from simple text to complex graphics etc. Otherwise, any MCU with enough pins will do.

If it really doesn't matter whether it costs $2 or $100, no consideration for size or power consumption, then there's no criteria to select. Get whatever you're more familiar with.

Although 300 pieces at $100 will cost $30k. That's more than enough to hire someone who would design an efficient solution for you.

[dobsonr741]

if you want the buy the platform ready for low code solutioning here is one: https://www.unihiker.com/

[Siwastaja]

Arduino plus a display module which comes with an easy-to-use library is not a bad idea: availability is good (including all the clones) and you are already familiar with it. For more complex projects or professional workflows, you can then move away from the Arduino ecosystem gradually, but it sounds like your project is achievable even with the limitations of Arduino libraries, and the ease of use is a plus for you if you are a newbie. Another advantage is you'll be able to find helping hands quite easily because nearly everyone has played around with Arduinos.

[nctnico]

I would not recommend a Raspberry Pi if you need precise control of timing. Maybe a RPi Pico....

You forget that SoCs like the ones used on the RPi have peripherals like you find on microcontrollers. Including hardware timers. On top of that it isn't hard to write a kernel module that reacts to a timer interrupt. This way you can create pretty decent response times from software as well.

[eugene]

LOL. And you forget all of the OPs requirements.

I'm glad that you will never attend one of my design reviews.

[sokoloff]

Most any microcontroller family will have a part to meet these fairly modest specs. Depending on your packaging requirements, you might start with what screen you want and work backwards from there. If it "doesn't matter, so long as it's functional", there are ESP32 boards with integrated OLED displays on them for under $20. Pair that with a rotary encoder with a pushbutton switch and the other buttons you need and you could be under $25 in parts with minimal assembly. ESP8266 has 2 10-bit ADCs and ESP32's are 12-bit. You will probably spend most of the complexity in the project doing the level shifting/protection to shape and protect the analog inputs. Most everything else is entirely straightforward.

If you need more control over the appearance, making a custom PCB and mounting a micro to it is also not hard, time-consuming, or particularly expensive.

A 500microsecond pulse is still fairly long and none of the other timing items specified are anywhere close to difficult to achieve.

I did read that you don't need other connectivity and accept that at face value but will still observe that the ESP chips have WiFi and Bluetooth, which might represent a value-add if you want to do computer or smartphone control either now or down the road.

[nctnico]

ESP8266 has 2 10-bit ADCs and ESP32's are 12-bit. You will probably spend most of the complexity in the project doing the level shifting/protection to shape and protect the analog inputs. Most everything else is entirely straightforward.

The ADCs in the ESP  chips are utter crap. It takes a whole lot of software and some extra hardware (reference input) to get some to something somewhat useful but there still is a fairly large piece of the lower / upper range you can't use.

[hans]

I think any technical solution is a product of their developer skillsets. An optimum in terms of bill of material costs or other "overheads" cannot always be reached unless development may take infinite resources. Using a SoC from a RPi is grossly overkill for the task at hand. But also tricky because desktop OS'es aren't intended for precise timing. That does not mean it cannot be done, but breaking out kernel modules to generate a 500u-5ms pulse is bonkers if OP is also clearly asking something thats managable for someone thats reasonably new to programming and microcontrollers.

Likewise I would argue the PLC is similarly bad idea. It may be fine for a handful of deployments, but OP is talking literally hundreds. Even if a cheap PLC costs 200$, if you need to purchase 200 of them you can easily pay a consultant or engineer 30k$ in wages (probably a good 1-2 months engineering time) for a solution that costs only 50$. Given the broad requirements I don't think someone experienced will take 2 months to complete this. Someone new maybe more, but those hours are also put into new skills.

Generating a few pulses and driving a LCD display is a manageable task even for a beginning developer with embedded systems. I'd recommended sticking to the Arduino eco system. ESP32 can also be programmed with the Arduino IDE, but you'll need to install their SDK. But Arduino Uno is perhaps the easiest to start.

Using Arduino has the benefit of a vast selection of libraries. I think this is most useful for a LCD display. If you want something graphic, I'm sure there are libraries for Arduino that can do a lot of the heavy lifting for you, and provide examples on how to write text on screens with fonts etc.  There are tons of LCD display shields and modules that work 2-wire I2C or 4-wire SPI. I'm not sure if the Uno is (best) suited with these libraries, because it only has 2K of RAM and thus cannot hold a framebuffer for a colour LCD of any usable size. Many people have moved beyond the Arduino Uno because of its memory limitations. However the beauty of Arduino is that you can quite easily migrate a sketch to a different Arduino-compatible board, such as a different Arduno board, or a ESP32 chip/module, or a Teensy.

Yes the ADC of the ESP32 is a bit crap. It has problems with rail-to-rail operation (offset and attenuation issues) and also IIRC linearity. It depends on the application though. If you need something a bit more accurate, you can also add an external ADC.

Although there is a lot of material on Arduino, its quality can vary. Some people may bash Arduino for this reason as "a proof-of-concept platform" at best. Although those arguments do have some merits, I don't think its an universal truth though. Like I said: different teams will develop different solutions to the same problem.

Choosing the right board can be a bit tricky. I'm not sure if longevity of a requirement is as well. Microcontrollers tend to stick around for quite a long time generally speaking, but if you were to buy a module such a Teensy, its an additional risk to see whether those boards also are still in stock, or are still being made. But at that point you could also look into designing your own PCB instead.

[woofy]

.. there are ESP32 boards with integrated OLED displays on them for under $20.

Yep, and the ESP32 can be programmed using the arduino environment, which is at least somewhat familiar to the OP. There are also lcd libraries available to make life easy.

[nigelwright7557]

Have a look at Arduino.
It comes with its own dev system.

I used to work for a Microchip consultancy so got stuck with PIC's.

[hubi]

What does medium resolution LCD mean to you? If that means 1024x768@24bpp, I would exclude small microcontrollers. If 320x240@16bpp is ok, I'd look at the Raspberry RP2040. I like the development environment, there are lots of examples, documentation and design resources. The chip has a poor ADC, but should meet your requirements. If you want, you can use the RP2040 with the arduino development environment, but the C/C++ SDK would be my preference by far. I really like the cmake based build system. There are fewer libraries available than for arduino, but I think you will find a graphics driver/display library pretty easily, e.g., TFT_eSPI and lvgl.

Also, it's cheap and available.

[MK14]

New to microcontroller development and not that proficient in software/computer development in general, altho we have done some work with RPi and Python utilizing I/Os to drive various controls, and precision DACs and read back from precision ADCs and have used the SPI for some work. So basically a newb when it comes to microcontrollers, altho did play around with Arduino a little.

So we are looking for a development platform to develop a relatively simple microcontroller than can produce some specific I/O in the form of pulses and possible read back a few logic signals and maybe some analog input.

What we need the microcontroller to do is create a set of 8 sequential non-overlapping pulses of varying width (0.5 to 5ms), at a rate of ~4Hz to ~80Hz selectable. Another output is a selectable timer from ~60sec to ~1800sec. Everything is ~6 bit resolution, except the analog input at ~8bit resolution.

For UI a rotary dial, with a couple push buttons (Start, Stop, Select) and medium resolution small LCD. No need for USB or other type interfaces, or the need to upload firmware by the end user, and not concerned about protecting the design or code either, as it's going to be trivial anyway.

Per item/chip cost isn't an issue as these will likely be less than few hundred total units in production, but the main point is a good, simple, easy to learn and use development platform to work from.

We have accounts at Adafruit, Newark, Mouser, Digikey, and a few popular other sources.

Anyway, looking for advice from those microcontroller gurus that are well versed in this technology sector (read actually microcontroller developers), and please not the usual rhetoric from those no valued added folks.

Thanks in advance for any guidance/help provided.

Best,

Given your mention of specific past experiences.

I'd suggest considering the raspberry PI PICO range.  As already mentioned (RP2040), by other(s).

It can readily come with MicroPython, and your low frequency signal requirements, may be doable quickly (development time wise), with its commands and library functions.

It is a real microcontroller, rather than an OS based Linux system (which a full on Raspberry PI, such as the Raspberry PI 4), would be.  Full OS's, can cause timing issues (errors), which may affect your program too badly (specification dependent).  Generally they are not considered acceptable for genuine/tricky hardware tasks, if precise timing etc is needed.
6 bits might be doable (with an OS), but it tends to introduce lots of jitter (uncertainty), which is best avoided, in many cases.
Bare metal on a big PI is possible, but is usually best left for experts in software, with huge amounts of time on their hands.  Whereas bare metal on a PI PICO is standard.

Also, increasingly these days it (The PI PICO), can be treated/programmed, as if it was almost an Arduino, as it is included in the Arduino studio IDE and is used by enough people (arguably lots, relatively speaking), to have lots of searchable information and tons of included documentation.

Supply of the PICO devices, which is available in both a very economically priced development board, useable in products or as a bare chip, has been excellent, even when other things were in severely, short supply.  They didn't even seem to price gouge.

There are even a range of ready made (somewhat plug in or connectable) accessories, such as displays and all sorts of things, at reasonable cost/quality, readily available.  A bit like the Arduino eco system.

It is available as a breadboard-able development board if wanted (header pins) and can include wi-fi and very recently bluetooth, built in as standard that can be useful, e.g. to ease development and debugging.  As you can readily connect it to stuff that way.

If things get tough with those timing signals, it has a (unfortunately rather complicated to program) PIO unit, which can handle very precise, complicated timing signals and other stuff.  Think of it as being an extremely tiny software like FPGA I/O peripheral.

Its downsides, are it doesn't have that much I/O stuff on the chip, limited number of I/O pins, and always needs an external flash (QSPI) chip, if going bare MCU chip.  It does have an A to D converter, but it is not an especially good one.

N.B. There are many other solutions, which also can have merits.  I've described only one possible solution/family.

[sokoloff]

ESP8266 has 2 10-bit ADCs and ESP32's are 12-bit. You will probably spend most of the complexity in the project doing the level shifting/protection to shape and protect the analog inputs. Most everything else is entirely straightforward.

The ADCs in the ESP  chips are utter crap. It takes a whole lot of software and some extra hardware (reference input) to get some to something somewhat useful but there still is a fairly large piece of the lower / upper range you can't use.

That seems like it could be a pretty good fit when the product brief includes "and [possibly] read back a few logic signals and maybe some analog input."

[nctnico]

What does medium resolution LCD mean to you? If that means 1024x768@24bpp, I would exclude small microcontrollers. If 320x240@16bpp is ok, I'd look at the Raspberry RP2040. I like the development environment, there are lots of examples, documentation and design resources. The chip has a poor ADC, but should meet your requirements. If you want, you can use the RP2040 with the arduino development environment, but the C/C++ SDK would be my preference by far. I really like the cmake based build system. There are fewer libraries available than for arduino, but I think you will find a graphics driver/display library pretty easily, e.g., TFT_eSPI and lvgl.

I think that route will still have a steep learning curve before you have a product you can deploy / sell without worries. I'm doing a project using an ESP32 right now and I need to dig deep into the build system here and there to make it do what I want (100% instead of the 99% that comes out of the box). Information found online is often outdated and/or incomplete.

Depending on timeline and budget it may make sense to start with something that has a more polished environment like a module that runs Linux (doesn't need to be a Rpi) and has some means to generate the signals in an easy way. Especially if the UI has to look good on a display with a reasonably high resolution. For a couple of 100 units it may make sense financially to go for a full-custom design based on a SoC or microcontroller but such a project takes quit a bit of time to finish (at least 9 months lead time just for development; time flies while testing & tweaking). And then you might need FCC / safety certifications as well. Using pre-certified hardware like a PLC suddenly looks like a quick way out...

[DrGeoff]

A PIC18 and an OLED display? The rest of the I/O is simple.

[mawyatt]

The RPi Pico and the RD2040 were a couple we had considered, but not the ESP32 which we'll look into.

To add some additional information, the "System".

Will need to display some text, a few lines, although needs to readable from 3~6 feet, so a small resolution LCD but larger area should suffice. A custom PCB is planned, as there's lots more "stuff" involved, like high current drivers for each pulse phase as well as overcurrent sensing and lockout (safety) precautions.

Later planned is having a multi-level final pulse for each phase where a high voltage and current is forced into the inductive loads, then quickly tapering off and decaying to a much lower holding level holding current, thus keeping the load and driver dissipation down. The timing of these multi-levels is yet to be determined, but the overall pulse-width and rep rate are roughly the same as previously mentioned. So each of the 8 pulse lines may need to be PWM to achieve the desired driver waveform decaying feature. As of now the concept involves creating the desired waveform in the HV and HC driver with PWM on each of the 8 pulse lines, with a shorter initial pulse, then a couple additional narrow pulses to create the holding current effect. We don't know enough about the various "loads" yet to have a good picture of what this waveform needs to be, and may need to be selectable for specific "loads". However, this is a much higher end System that will be at a Premium, and require another custom PCB and supporting components.

One note, we could have farmed this out and had a true pro create the controller, but we saw this as an opportunity as a means to learn something about microcontrollers and because we are semi-retired our time isn't as valuable as when directly billable while previously employed......and not having to create significant valued added for senior executives quarterly bonuses

Anyway, thanks folks for all the good information and discussions

Best,

[PCB.Wiz]

Will need to display some text, a few lines, although needs to readable from 3~6 feet, so a small resolution LCD but larger area should suffice.

So this may be best as two modules ?

You could start with common character LCD modules cheap, and available in larger glass models for better distance reading.
one example  https://www.aliexpress.com/w/wholesale-large-character-LCD-i2c.html
Then you can decide if you need to add more time/effort/cost to a fancier display
example https://www.aliexpress.com/item/1005005297304786.html

New to microcontroller development and not that proficient in software/computer development in general, altho we have done some work with RPi and Python utilizing I/Os to drive various controls, and precision DACs and read back from precision ADCs and have used the SPI for some work. So basically a newb when it comes to microcontrollers, altho did play around with Arduino a little.

So we are looking for a development platform to develop a relatively simple microcontroller than can produce some specific I/O in the form of pulses and possible read back a few logic signals and maybe some analog input.

What we need the microcontroller to do is create a set of 8 sequential non-overlapping pulses of varying width (0.5 to 5ms), at a rate of ~4Hz to ~80Hz selectable. Another output is a selectable timer from ~60sec to ~1800sec. Everything is ~6 bit resolution, except the analog input at ~8bit resolution.
.....
A custom PCB is planned, as there's lots more "stuff" involved, like high current drivers for each pulse phase as well as overcurrent sensing and lockout (safety) precautions.

Later planned is having a multi-level final pulse for each phase where a high voltage and current is forced into the inductive loads, then quickly tapering off and decaying to a much lower holding level holding current, thus keeping the load and driver dissipation down. The timing of these multi-levels is yet to be determined, but the overall pulse-width and rep rate are roughly the same as previously mentioned. So each of the 8 pulse lines may need to be PWM to achieve the desired driver waveform decaying feature. As of now the concept involves creating the desired waveform in the HV and HC driver with PWM on each of the 8 pulse lines, with a shorter initial pulse, then a couple additional narrow pulses to create the holding current effect. We don't know enough about the various "loads" yet to have a good picture of what this waveform needs to be, and may need to be selectable for specific "loads". However, this is a much higher end System that will be at a Premium, and require another custom PCB and supporting components.

That's starting to sound like a programmable waveform generator, which could need a MCU with a bit more RAM.

I'd start with that you already know, and try multiple paths.
Find which is easiest to get working.

The RPi may be to unwieldly, but the Pi-Pico + python might be quick enough.
A generic Ardunio may be 'good enough' ?

or, you might like to try to get your hands on the new Arduino UNO R4  using 5V Renesas RA4M1 (Arm Cortex®-M4)
5V is good for gate driving power MOSFETS

https://www.elecrow.com/blog/new-arduino-uno-r4-coming-in-may-upgraded-configuration-with-usb-c-and-wi-fi-support.html


 

[NorthGuy]

Will need to display some text, a few lines, although needs to readable from 3~6 feet, so a small resolution LCD but larger area should suffice. A custom PCB is planned, as there's lots more "stuff" involved, like high current drivers for each pulse phase as well as overcurrent sensing and lockout (safety) precautions.

You don't need much CPU power to control a text LCD - the LCD will have its own controller, you only need to pass some commands and the text.
 

Later planned is having a multi-level final pulse for each phase where a high voltage and current is forced into the inductive loads, then quickly tapering off and decaying to a much lower holding level holding current, thus keeping the load and driver dissipation down. The timing of these multi-levels is yet to be determined, but the overall pulse-width and rep rate are roughly the same as previously mentioned. So each of the 8 pulse lines may need to be PWM to achieve the desired driver waveform decaying feature. As of now the concept involves creating the desired waveform in the HV and HC driver with PWM on each of the 8 pulse lines, with a shorter initial pulse, then a couple additional narrow pulses to create the holding current effect. We don't know enough about the various "loads" yet to have a good picture of what this waveform needs to be, and may need to be selectable for specific "loads". However, this is a much higher end System that will be at a Premium, and require another custom PCB and supporting components.

RPi is totally unsuitable for this. Get a real MCU. If you want to use PWM you can do it with PWM modules (which generate precise PWM pulses for you), for which you would need an MCU with 8 PWM modules. Or, you can bit-bang your pulses without PWM modules. If it's hard for you to do all 8 channels from one MCU, you can have 8 very small MCUs each controlling a single FET driver. These small MCUs then can be controlled from a master MCU.

One note, we could have farmed this out and had a true pro create the controller, but we saw this as an opportunity as a means to learn something about microcontrollers and because we are semi-retired our time isn't as valuable as when directly billable while previously employed......and not having to create significant valued added for senior executives quarterly bonuses

Do it. Forget the Python stuff, get a real MCU and learn to program in C. This is not a rocket science, and is much easier to do than you think. Take practical (i.e. how do I do it?) as opposed to esoteric (i.e. what is the prescribed way to do it?) approach to shorten your learning curve. Remember that your goal is to learn something, as opposed to getting results without learning. Move in small steps (as small as you can make them). May be it'll take couple weeks, but the result will be way more reliable and manageable.

[rstofer]

I would do the development on an Arduino UNO for the libraries.  For production, I would design a specific purpose PCB and use the ATmega328 chip with the same code and libraries.  Everything that can be done with the Arduino has already been done and the projects are all over the Internet.

The Pi Pico with Micro-Python is a really good choice but I don't know if the infrastructure has all of the required libraries.  I  like the Micro-Python approach but I don't have a lot of experience with it.  The boards are small enough that your end product could be an interface PCB and a mounted Pico.

I'm starting to like the Micro-Python thing but C++ works on the Pico as well.

[eugene]

If it's hard for you to do all 8 channels from one MCU, you can have 8 very small MCUs each controlling a single FET driver. These small MCUs then can be controlled from a master MCU.

This is a technique that I think has yet to become fully appreciated, (except perhaps by the automotive industry.) I recently needed to coordinate 10 BLDC motors. Obviously, there are no MCUs with 30 PWM channels, so I went all-in with the objected oriented hardware and ended up with 16 individual MCUs, each with their own well defined job to do. It ended up being easier to implement than trying to get a quad core processor to do everything.

So yeah, I still don't understand the OP's application well enough, but I would consider a separate ATtiny for each FET as a possible solution. They're cheap and easy to get working on a breadboard before designing into a full system. And, if part of your goal is to learn about MCUs, there is no easier place than 8 bit AVRs

[PCB.Wiz]

.... I recently needed to coordinate 10 BLDC motors. Obviously, there are no MCUs with 30 PWM channels, so I went all-in with the objected oriented hardware and ended up with 16 individual MCUs, each with their own well defined job to do. It ended up being easier to implement than trying to get a quad core processor to do everything.

I'd agree it is easier to have local focused hardware, especially if each also has gate drivers and high power associated outputs.

That said, there actually are even 8 bit MCUs with 30 or more PWM channels   

The STC STC8G2K64S4 series have 45 channels of 15b PWM, and the SinOne SC95FW40 has 40 channels of 12b PWM - maybe drones with many motors is the market here ?

More common is 8 channels of PWM.

[mawyatt]

So this may be best as two modules ?

You could start with common character LCD modules cheap, and available in larger glass models for better distance reading.
one example  https://www.aliexpress.com/w/wholesale-large-character-LCD-i2c.html
Then you can decide if you need to add more time/effort/cost to a fancier display
example https://www.aliexpress.com/item/1005005297304786.html



That's starting to sound like a programmable waveform generator, which could need a MCU with a bit more RAM.

I'd start with that you already know, and try multiple paths.
Find which is easiest to get working.

The RPi may be to unwieldly, but the Pi-Pico + python might be quick enough.
A generic Ardunio may be 'good enough' ?

or, you might like to try to get your hands on the new Arduino UNO R4  using 5V Renesas RA4M1 (Arm Cortex®-M4)
5V is good for gate driving power MOSFETS

We aren't quite as naive as we indicated regarding the RPi and controlling things. Our last major project was a precision ~15 bit accurate 128 independent channel (expandable) AWG that produced up to +-100V independent outputs. Each channel needed to be Bi-phase modulated and the waveform "average" after modulation needed to be with a few millivolts of 0 on a cycle by cycle basis. All created and controlled from a RPi (developed on RPi 400 and also used with a Pi Zero) with mostly SPI and Python.

The application was highly proprietary, even for us, as we didn't get to attend the first demo of the overall system. We were just given some challenging specs, which were attained first pass, later to be developed into a custom chip set, which apparently Covid has postponed. We had gleaned from discussions which we can disclose, this was the steering controller for a new technology dynamically configurable electronic controlled rapid XY beam steering Phase Array antenna system in MMW based upon Liquid Crystal Technology. Results we got back from 1st test was everything was hooked up according to our documents, powered up, and a beam position command was entered and the narrow MMW beam immediately went to the precise position as commanded. After this initial test, everything went dark as we suspect the use may be sensitive and we have not been asked any questions or been made aware of any problems. This is a tribute to the folks behind this "System" as they are pretty darn good at what they do, and provided the proper specs we needed, but nothing more, so sometimes no news is good news 

Anyway, thanks for the information

Best

[mawyatt]

So yeah, I still don't understand the OP's application well enough, but I would consider a separate ATtiny for each FET as a possible solution. They're cheap and easy to get working on a breadboard before designing into a full system. And, if part of your goal is to learn about MCUs, there is no easier place than 8 bit AVRs

This is certainly a possibility, especially for the system that's requiring pulse shaping by means of PWM. We don't need to worry about the output level as 5V pulses are insufficient to drive the output MOS device, which are multiple balanced ~100A PMOS devices for each of the 8 phases. The gate drive will be upscaled to 0 to +12V, to fully enable the PMOS device channel.

The timing of the pulses is such that each "phase" of the 8 sequential pulses will require identical PWM waveforms, altho time displaced by the overall pulse-width. One idea we were considering in having a single PWM channel create the PWM waveforms, repeated at each time displaced phase, and have each of the 8 phase channels just act as a pulse enable signal to the high current load driver.

Anyway, thanks for the ideas

Best,

[NorthGuy]

The gate drive will be upscaled to 0 to +12V, to fully enable the PMOS device channel.

PMOS would require negative voltage to turn on the FET.

[mawyatt]

The gate drive will be upscaled to 0 to +12V, to fully enable the PMOS device channel.

PMOS would require negative voltage to turn on the FET.

They are arranged as Drain drivers with the Source at +12V, and the Drain drives the Load which has a common ground. So the Gate is driven from +12V to zero to turn ON the PMOS.

[nctnico]

That must be a special MOSFET then. Most MOSFETS have an optimum Vgs between 4V to 9V where the gate charge versus Rdson is minimal and thus switching speed is highest.

[mawyatt]

Switching speed isn't that important, Rdson is

[NorthGuy]

They are arranged as Drain drivers with the Source at +12V, and the Drain drives the Load which has a common ground. So the Gate is driven from +12V to zero to turn ON the PMOS.

So, you have 12V 100A. How big are your inductors?

[mawyatt]

No, the PMOS are >100 amp pulse rated, and must be significantly derated on long pulses or continuous use. Please study up on how power MOS FETs are used and specified. Quite misleading specifications often cause issues with designs that don't take the transient, including power, current and thermal effects into account.

Best,

[NorthGuy]

No, the PMOS are >100 amp pulse rated, and must be significantly derated on long pulses or continuous use. Please study up on how power MOS FETs are used and specified. Quite misleading specifications often cause issues with designs that don't take the transient, including power, current and thermal effects into account.

You're speaking as if you gave us the datasheet to look at. But you keep everybody in the dark ...

To select an MCU, you start by figuring voltages, currents, size of the inductors you switch etc. Then you use this information to calculate rise and fall times. This, in turn, will tell you how accurate your pulses may be. Say, if your rise time is 10 us, this is very long time for a slow MCU, and you can write a sloppy program in C to bit-bang everything sequentially and don't worry about accuracy. Or, if rise/fall times are fast you can place your pulses more accurately (if you need to). Or, if your inductors never saturate then you can use PWM to control inductor's current. These are all different approaches.

[mawyatt]

No, the PMOS are >100 amp pulse rated, and must be significantly derated on long pulses or continuous use. Please study up on how power MOS FETs are used and specified. Quite misleading specifications often cause issues with designs that don't take the transient, including power, current and thermal effects into account.

You're speaking as if you gave us the datasheet to look at. But you keep everybody in the dark ...

To select an MCU, you start by figuring voltages, currents, size of the inductors you switch etc. Then you use this information to calculate rise and fall times. This, in turn, will tell you how accurate your pulses may be. Say, if your rise time is 10 us, this is very long time for a slow MCU, and you can write a sloppy program in C to bit-bang everything sequentially and don't worry about accuracy. Or, if rise/fall times are fast you can place your pulses more accurately (if you need to). Or, if your inductors never saturate then you can use PWM to control inductor's current. These are all different approaches.

Nobody's in the dark except maybe you!! We don't need advice on inductive power switching design nor power MOS FET selection, quite well versed thank you!! And never asked for such, only about microcontroller selection which just needs to generate the basic low frequency pulses we've outlined in the 1st post. BTW the PMOS device we are considering is a AP120P30.

You need to read what we've posted earlier, please do so!! The PWM is for an advanced Premium version not completely specified yet, not the one we've been discussing!! Here's a quote from what we said earlier, note the highlighted text!!

Later planned is having a multi-level final pulse for each phase where a high voltage and current is forced into the inductive loads, then quickly tapering off and decaying to a much lower holding level holding current, thus keeping the load and driver dissipation down. The timing of these multi-levels is yet to be determined, but the overall pulse-width and rep rate are roughly the same as previously mentioned. So each of the 8 pulse lines may need to be PWM to achieve the desired driver waveform decaying feature. As of now the concept involves creating the desired waveform in the HV and HC driver with PWM on each of the 8 pulse lines, with a shorter initial pulse, then a couple additional narrow pulses to create the holding current effect. We don't know enough about the various "loads" yet to have a good picture of what this waveform needs to be, and may need to be selectable for specific "loads". However, this is a much higher end System that will be at a Premium, and require another custom PCB and supporting components.

The likely voltage for these loads will be ~90V not 12V, adaptable, and as mentioned the waveform detailed characteristics & shape are mostly unknown at this time for this Premium System. This system will require a completely different controlling scheme (both NMOS & PMOS switching devices), faster pulses with PWM, different Power MOS devices likely a different microcontroller.

So please lets stay with the initial system originally described, it's requirements, and focus on the microcontroller, which is where we need some quidance! We'll have plenty of time to discuss the advanced Premium System at some later date as the requirements materialize and we finalize the initial system to free up time for the Premium System design, hopefully by then we'll a little more proficient in microcontrollers

Best,

[NorthGuy]

So please lets stay with the initial system originally described, it's requirements, and focus on the microcontroller, which is where we need some quidance!

Any MCU, given enough pins, can do what you have described in your first post. It seems you have a good understanding of what you need to do. What sort of guidance do you need?

[PCB.Wiz]

So please lets stay with the initial system originally described, it's requirements, and focus on the microcontroller, which is where we need some quidance!

What we need the microcontroller to do is create a set of 8 sequential non-overlapping pulses of varying width (0.5 to 5ms), at a rate of ~4Hz to ~80Hz selectable. Another output is a selectable timer from ~60sec to ~1800sec. Everything is ~6 bit resolution, except the analog input at ~8bit resolution.
With a pulse train like that, you may be better to define to edge toggle times, and have small slaves you send a pulse-set to, and then trigger.
If that becomes firmly defined, that pulse set can go into code FLASH.

You may be able to scale this, to fit into the common 16 bits timer size
eg if you need 0.5ms to 6 bits, that 10us LSB can define to 0.655 seconds

AVR's have  timer prescalers with fairly coarse choices, but that may be  'good enough'.
The 8051 families have a PCA with a HSO mode, which toggles a pin on a match setpoint, and finer prescaler choices.
The SiLabs EFM8BB5x series have 50MHz sysclks, which gives good headroom.
The BB52 has 3 PCA channels, but you can shift them along the pins, so if your pulses never overlap, you can have more pulsing pins.

If you need precision PWM, the STC8H family have 8 x 16b PWM's with a nice fine 16b /N prescaler and TSSOP20/TQFP32 packages

Some 8b MCU's are getting > 100MHz PWMs, but you are not quite pushing that territory.
Maybe get some low cost eval boards, and try creating some example pulse trains.

If I was coding an experimental/flexible pin pulse generator, I might pick a EFM8BB52, and allocate 4 bytes per edge : The lower 2 are HW timer match, 3rd byte is SW timer match, and upper byte is pin select + toggle enable + spares.
Pulses longer than 2^16 would need some packer values, ie time stamps without toggles, as HW simply left alone will toggle once every 2^16.
Software updates times place a rule on shortest possible pulse, something less than maybe 100 sysclks would need care, ~2us tho more HW via CLU logic or external XOR could drop to 1 sysclk.

A RPi or any host could generate and send those pulse records and then trigger the MCU's on a trigger pin.

You did not explicitly state the edge precision and total number of edges, or the total timing precision (on chip OSC or external OSC) ?

[Siwastaja]

Jitter was not specified but assuming it's not critical, the frequencies sound low enough that I would just pick any random Cortex-M microcontroller and implement the whole generation logic in a periodic timer interrupt handler and run the UI in main loop.

[larsdenmark]

While it is not too difficult to obtain the pulse widths you're looking and driving a LCD display it is easy to screw things up when both have to be carried out unless you have more than one core.

RPi Pico and ESP32 have two cores and should require the same amount of coding to get everything going. Rpi Pico (RP2040) has programmable IO (PIO) that can be used for extremely fast and/or precise timing for the pulses (but PIO is not entry level stuff). Both can be programmed using Arduino libraries.

Running on some memory limited, single core Arduino processor doesn't make any sense for the number of devices you are talking about.

Being familiar with RPi is a plus when programming the first system, but you'll have a hard time purchasing them in the hundreds.

[mawyatt]

Any MCU, given enough pins, can do what you have described in your first post. It seems you have a good understanding of what you need to do. What sort of guidance do you need?

Mainly in a development platform and specific microcontroller(s) that are easy for a someone relatively new to the microcontrollers to get up and running that can produce the simple pulse scheme described.

Best,

[mawyatt]

To address some additional information on the pulse requirements, this is for the initial system, and NOT the Premium System which will employ PWM for waveform tailoring at a later timeframe.

The pulses are not critical at all, could be ~5% without issue, even have a little overlap, and many microseconds of jitter. The 8 pulses are all the same tho, just displaced in time, and should match within ~1%, i.e. pulse 1 should be the same as pulse 2 or 8, or 5, or any other pulse and pulse-width match within 1%, the pulse-width can vary ~5% without issue, as can the repetition rate vary by ~5% without issue.

In fact we have a test PCB that's going to be utilized to further understand and model the various loads, some unknown at this time, and this is just a single channel based upon a couple 555 timers with pots to set the pulse-width and repetition rate. Will also include the high current load driver to evaluate various PMOS devices under various loads.

Anyway, thanks folks for the valuable responses and information

Best,

[Siwastaja]

Many microseconds of jitter -> ISR & GPIO approach is more than fine. Pretty easy, just increment a counter and turn inputs on/off based on the counter value; sounds like it could even cater the "premium" PWM version. You can easily have a 10kHz periodic interrupt on a simple AVR, or a 100kHz interrupt on a higher-end 32-bit MCU.

Arduino ecosystem would soften the initial hit as you could just buy a display "shield" and start working with things like lcd.display("Hello World"); with the display library which would work out of the box.

[NorthGuy]

Mainly in a development platform and specific microcontroller(s) that are easy for a someone relatively new to the microcontrollers to get up and running that can produce the simple pulse scheme described.

For a project like yours, I don't think there is any significant difference between different MCUs. The development platform is provided by MCU's manufacturer. It consists of IDE and development tools (C compiler, linker). You'll also need a programmer, which may also be a debugger, although I don't think you'd need a debugger for such a simple project. Your development process will be roughly the same - you'll write C code and run it on your MCU.

[nctnico]

For writing code you'll need clear documentation and a microcontroller that works as described. In this alone differences are huge between manufacturers. Pile manufacturer supplied libraries on top and you can be hunting documentation / reality discrepancies for days.

I agree with Siwastaja here. A much better option for the OP is to look for something similar that already exists (and works well) as an Arduino project and start from there IF the goal is to actually finish the project within a reasonable timeframe. Depending on the OP's ability to spot the difference between bad/good code examples quickly, using existing software can help to save time.

At some point the timer control could be programmed directly if/when more control over the hardware is needed. But this will be a small, incremental step.

Another question is: how much can be re-used to make the second, premium version of this device? Does it need fancy graphics on the display or is it just more timer features?

My initial impression for this project was that it would need a GUI / display with fancy graphics to give it some appeal (wow factor) to the bean counters in order to get it over the counter. But that doesn't seem to be the case.

[sokoloff]

My initial impression for this project was that it would need a GUI / display with fancy graphics to give it some appeal (wow factor) to the bean counters in order to get it over the counter. But that doesn't seem to be the case.

That insightful comment gives me an idea:

Have you considered using a tablet (Android or iPad) or laptop, running a web browser, and talking to the prototype device over http over WiFi?

Serve that up from an ESP32 and you have a cheap/easy system to at least demo and can build whatever interface you want during development.

(Again, this is all spitballing without deep insight into the unstated product requirements.)

I have an ESP8266 connected to my old boiler and I just now opened up a port on my firewall to NAT that page out to the internet.
http://73.126.96.83:55555/

That page is not designed to be pretty, but you could certainly make a pretty one.

[PCB.Wiz]

To address some additional information on the pulse requirements, this is for the initial system, and NOT the Premium System which will employ PWM for waveform tailoring at a later timeframe.

The pulses are not critical at all, could be ~5% without issue, even have a little overlap, and many microseconds of jitter. The 8 pulses are all the same tho, just displaced in time, and should match within ~1%, i.e. pulse 1 should be the same as pulse 2 or 8, or 5, or any other pulse and pulse-width match within 1%, the pulse-width can vary ~5% without issue, as can the repetition rate vary by ~5% without issue.

That's quite relaxed, you might be able to even use the RPi for that, for initial testing
Google finds this
https://forums.raspberrypi.com//viewtopic.php?f=72&t=67741#p496306
which configures a PWM and says
You can't really get a regular gap, you seem to get at least 150 microseconds between pulses whatever you do.


Google also finds this for RPi

I also did some experimenting with the allowable range of values for pwmSetClock() and pwmSetRange(). As noted in one of the other answers, the valid range for pwmSetClock() seems to go from 2 to 4095, while the valid range for pwmSetRange() is up to 4096 (I didn't attempt to find a lower-limit).
The Raspberry Pi PWM clock has a base frequency of 19.2 MHz.

So it looks like 12 bit prescaler and 12 bit compare fields, with some ability to further change the default 19.2MHz.  ie Quite basic, but could be 'good enough'.


The PiPico may be better - tho this is still a more complex production solution than a 8 it MCU, it is easy to play around with.
Pi Pico HW PWM :   Duty cycle resolution 16 bits PWM Clock 125MHz
Prescaler 8.4 so up to 256,  so longest timebase 125M/2^16/256 = 7.45Hz

or there is the PIO state engine, if you dig that deep.

Addit : for the specs of  create a set of 8 sequential non-overlapping pulses one simplification you can apply here, to almost any solution, is to add a 74HC138/74HC238 as that avoids the need to re-map peripherals to pins.
You simply increment the MUX and have a single pin generating the pulse train.

[PCB.Wiz]

Mainly in a development platform and specific microcontroller(s) that are easy for a someone relatively new to the microcontrollers to get up and running that can produce the simple pulse scheme described.

Already mentioned are AVR and 8051 cored parts, which are well under $1 and able to manage this easily.

Low cost eval boards exist for both - SiLabs EFM8BB52
https://www.digikey.com/en/products/detail/silicon-labs/BB52-EK2701A/15276600

and new AVRs, with DEBUG on board
https://www.digikey.com/en/products/detail/microchip-technology/DM164151/12177481
https://www.digikey.com/en/products/detail/microchip-technology/EV35L43A/13174972

or, if you are ok with Chinese, STC have a wide range 
https://ec-buying.aliexpress.com/store/1762106/search?SearchText=STC

[tggzzz]

When using the RPi and equivalent computers for hard real time systems, you have to ensure the worst case performance matches your needs. Such computers have many optimisations to increase the average performance.

That raises an obvious question how do you know what the worst case performance actually is? If the principal timings are guaranteed by hardware (e.g. timers), then you still have to verify the software responds to inputs sufficiently fast and to verify the software is fast enough to generate all the outputs. The design tools provide zero help with this, since they cannot determine the effects of all the different types of cache misses.

The only way is by measurement, and hoping that you have captured the worst case performance. While that may be acceptable in some circumstances, it is unsatisfying.

If you are running a fully-fledged operating system such as linux, then the effects of that have to be assessed and measured.

Personally for hard real time operation I prefer having a very simple processor with "poor" but repeatable performance. Preferably it run on bare silicon without an OS, but an RTOS may be beneficial.

Ideally the IDE should inspect the compiled code to determine the min/max timings, and the equivalent of an RTOS should be implemented in hardware. Such (single source) systems do exist, and can be bought at DigiKey.

[nctnico]

The only way is by measurement, and hoping that you have captured the worst case performance. While that may be acceptable in some circumstances, it is unsatisfying.

If you are running a fully-fledged operating system such as linux, then the effects of that have to be assessed and measured.

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

[tggzzz]

The only way is by measurement, and hoping that you have captured the worst case performance. While that may be acceptable in some circumstances, it is unsatisfying.

If you are running a fully-fledged operating system such as linux, then the effects of that have to be assessed and measured.

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Maybe that's what you have seen[1] on your system. Now prove there won't be any problems under any conditions.

If a deadline was missed, what would be the consequences? Now answer the same question with the OP's application, hardware, and external system.

Given that the OP has highlighted the importance of Rdson in devices being switched at accurate times, I suspect relatively high energies are being controlled.

[1] BTW, that's not what I see on my (rather old) system.

[SiliconWizard]

That's cool. Talk is cheap though.
So, just show us an example of a "real-time task" running with a few µs latency on Windows and vanilla Linux. Measure real latency, not self-convincing tricks.

On Linux, it's just not possible with the vanilla kernel and base options due to how the scheduler works. Even interrupts are all handled in a single thread by default - that's not the case if you either use a RT-patched kernel, or if you enable the 'threadirqs' option on the kernel boot line. That helps. You can use benchmarks from the rt-tests package to get reliable figures. The 'cyclictest' program for instance.
With a RT-patched kernel, I can get max latencies (it's the max that matters, of course, not the min or average) of about 15-20µs, it's already pretty good and I've never seen better than this on any machine I've dealt with - maybe it can happen, if all stars are aligned. With a vanilla kernel (latest) with the threadirqs option, I can get a max. latency of about 150-200µs. All that with pretty fast machines.

That's a benchmark, and of course, to get that in your applications, you need to carefully craft your programs, configure your kernel properly and create threads with a high priority.

On Windows I don't have recent benchmarks on latency. I've never seen it have lower latencies than the Linux kernel though up to Win 7.

[PCB.Wiz]

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Of course, the examples you just gave, both use significant peripheral assistance, they are not purely OS delivered.

Yes, I have used Windows to design a DAC and logger, (no remote MCU) and that was able to stream continually and have defined timing to sub-milliseconds, but it was not the OS alone at work here.
It used a USB-UART bridge and those have significant buffers that tolerate the clumping effects of the OS.

It works both ways, if you see the CP2102 thread, that shows how just bad 'USB+OS' are at fine granular half-duplex communications.
There are many things to get in your way.

[Siwastaja]

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Audio & video is a specialized task for which the OS has been hand-tailored. Besides, things like smearing is commonplace, especially on linux, has always been. Missed frames, audio playback stutter, loss of syncronization etc. also happens every now and then. You just don't always notice it, and when you do, it's not catastrophic because audio/video on general purpose computer is entertainment.

Look at projects like linuxCNC and you'll see how difficult it is. It tries to bit-bang stepper motor direction/pulse controls on the good old parallel bus ("printer port"), using a customized linux kernel build and heavily tuned userland packaged into its own "distribution" (a debian fork IIRC), and it still usually fails so people got tired to it and started building actual motion controllers, many based on surprisingly simple microcontrollers like 8-bit AVRs, which buffer commands and act on their own.

[tggzzz]

That's cool. Talk is cheap though.
So, just show us an example of a "real-time task" running with a few µs latency on Windows and vanilla Linux. Measure real latency, not self-convincing tricks.
...
With a RT-patched kernel, I can get max latencies (it's the max that matters, of course, not the min or average) of about 15-20µs, it's already pretty good and I've never seen better than this on any machine I've dealt with - maybe it can happen, if all stars are aligned.
...
That's a benchmark, and of course, to get that in your applications, you need to carefully craft your programs, configure your kernel properly and create threads with a high priority.

Yup.

Even then you are still crossing your fingers and hoping you've spotted the max latency. I don't know how you can ensure all the caches are in the pessimal state required for max latency. Start with branch-predictions cache, move on to L1, L2, L3 caches, don't forget TLB caches, nor superscalar instruction pipelines - and then there's all the software techniques such as hashmaps.

Fundamentally, if you care about worst-case performance, then you have to avoid every technique used to improve average performance. Best way is to start with simple hardware and software - preferably designed to guarantee hard realtime.

Or you can say the equivalent of "you can use an elephant gun to kill a fly".

[tggzzz]

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Audio & video is a specialized task for which the OS has been hand-tailored. Besides, things like smearing is commonplace, especially on linux, has always been. Missed frames, audio playback stutter, loss of syncronization etc. also happens every now and then. You just don't always notice it, and when you do, it's not catastrophic because audio/video on general purpose computer is entertainment.

Yup. And that's even with heavily buffered i/o peripherals. For that application the increased latency won't matter.

Look at projects like linuxCNC and you'll see how difficult it is. It tries to bit-bang stepper motor direction/pulse controls on the good old parallel bus ("printer port"), using a customized linux kernel build and heavily tuned userland packaged into its own "distribution" (a debian fork IIRC), and it still usually fails so people got tired to it and started building actual motion controllers, many based on surprisingly simple microcontrollers like 8-bit AVRs, which buffer commands and act on their own.

I didn't realise many people are wasting their lives trying (and failing) to do things that way. Can't say I'm surprised though

[nctnico]

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Audio & video is a specialized task for which the OS has been hand-tailored. Besides, things like smearing is commonplace, especially on linux, has always been. Missed frames, audio playback stutter, loss of syncronization etc. also happens every now and then. You just don't always notice it, and when you do, it's not catastrophic because audio/video on general purpose computer is entertainment.

What you are describing are typically buffering problems stemming from hard drives not keeping up / being busy with something else. I'm watching quite a bit if video on my Linux computer and what you describe doesn't happen because I set the prebuffering to a larger size.

I have done a couple of ultra low latency video player projects using Linux and in those missing frames is not allowed to happen. Data and parameters need to be moved around at precise times governed by the display's refresh rate (which ultimately stems from the vertical retrace interrupt). I actually did measurement by wiggling I/O pins + oscilloscope to verify timing is met and it does. Response times into user space applications are actually quite good and consistent.

[tggzzz]

It is funny that people keep on clinging to this myth. The reality is that OSses like Linux and Windows are perfectly capable of doing realtime tasks requiring single digit microsecond timing accuracy. If they wheren't, they wouldn't be suitable for playing audio and video in a synchronised fashion. Video playback -for example- is tightly coupled to the monitor's refreshrate using the vertical interrupt at the lowest level to avoid flickering and smearing between frames.

Audio & video is a specialized task for which the OS has been hand-tailored. Besides, things like smearing is commonplace, especially on linux, has always been. Missed frames, audio playback stutter, loss of syncronization etc. also happens every now and then. You just don't always notice it, and when you do, it's not catastrophic because audio/video on general purpose computer is entertainment.

What you are describing are typically buffering problems stemming from hard drives not keeping up / being busy with something else. I'm watching quite a bit if video on my Linux computer and what you describe doesn't happen because I set the prebuffering to a larger size.

So? It is sufficent to counter your statement.

I have done a couple of ultra low latency video player projects using Linux and in those missing frames is not allowed to happen. Data and parameters need to be moved around at precise times governed by the display's refresh rate (which ultimately stems from the vertical retrace interrupt). I actually did measurement by wiggling I/O pins + oscilloscope to verify timing is met and it does. Response times into user space applications are actually quite good and consistent.

Remove the adjectives and provide the numbers of the worst case behaviour. Demonstrate why those numbers are more than the worst observed case.

I ran into this data point 15s before reading your response. It is about the PostGres database, and serves to illustrate the kinds of problems that must occur

I think we're starting to hit quite a few limits related to the process model, particularly on bigger machines. The overhead of cross-process context switches is inherently higher than switching between threads in the same process - and my suspicion is that that overhead will continue to increase. Once you have a significant number of connections we end up spending a *lot* of time in TLB misses, and that's inherent to the process model, because you can't share the TLB across processes.

https://lwn.net/SubscriberLink/934940/3abb2d4086680b78/

[nctnico]

Why would a video player suffer from TLB misses? By nature every frame is new data which must be fetched from an SSD and can be disgarded after showing it. That is repeating the same process over and over by a limited amount of code which easely fits in the cache. The limit is the data throughput of the system. A Postgresql database is a different beast entirely; you can't even begin to compare that.

An interesting experiment is to take a multicore SoC and create a square wave (based on the systick counter / monotonic counter) on a GPIO pin using a high priority process that is locked to a core. I'm pretty sure it will have very little jitter as very little gets in the way of executing it and SoCs typically run at >1GHz. I've seen surprisingly good behaviour on bit-banged I2C busses as well (not my choice to make it that way) on low end (single core, 400MHz SoCs) running Linux. It is not like the clock rate is allover the place or messages get interrupted a lot. Not that it matters from I2C though.

[tggzzz]

Why would a video player suffer from TLB misses? By nature every frame is new data which must be fetched from an SSD and can be disgarded after showing it. That is repeating the same process over and over by a limited amount of code which easely fits in the cache. The limit is the data throughput of the system. A Postgresql database is a different beast entirely; you can't even begin to compare that.

An interesting experiment is to take a multicore SoC and create a square wave (based on the systick counter / monotonic counter) on a GPIO pin using a high priority process that is locked to a core. I'm pretty sure it will have very little jitter as very little gets in the way of executing it and SoCs typically run at >1GHz. I've seen surprisingly good behaviour on bit-banged I2C busses as well (not my choice to make it that way) on low end (single core, 400MHz SoCs) running Linux. It is not like the clock rate is allover the place or messages get interrupted a lot. Not that it matters from I2C though.

Go on then. Demonstrate the jitter when bit banging a 1MHz 50% duty cycle square wave.

Then add 205619Hz 732997Hz and repeat. Then add comms to remote controlling PC.

And then provide an answer as to whether being able to use an elephant gun to kill a fly in the desert (but not a town) is a worthwhile exercise.

Don't forget to respond to the points made by various people, not just cherrypick one example.

[nctnico]

Why would a video player suffer from TLB misses? By nature every frame is new data which must be fetched from an SSD and can be disgarded after showing it. That is repeating the same process over and over by a limited amount of code which easely fits in the cache. The limit is the data throughput of the system. A Postgresql database is a different beast entirely; you can't even begin to compare that.

An interesting experiment is to take a multicore SoC and create a square wave (based on the systick counter / monotonic counter) on a GPIO pin using a high priority process that is locked to a core. I'm pretty sure it will have very little jitter as very little gets in the way of executing it and SoCs typically run at >1GHz. I've seen surprisingly good behaviour on bit-banged I2C busses as well (not my choice to make it that way) on low end (single core, 400MHz SoCs) running Linux. It is not like the clock rate is allover the place or messages get interrupted a lot. Not that it matters from I2C though.

Go on then. Demonstrate the jitter when bit banging a 1MHz 50% duty cycle square wave.

Then add 205619Hz 732997Hz and repeat. Then add comms to remote controlling PC.

And then provide an answer as to whether being able to use an elephant gun to kill a fly in the desert (but not a town) is a worthwhile exercise.

No time to set that up right now. Maybe later. But the reasoning to use Linux is that it makes it easier to create a sophistigated GUI (if that is required) without needing a lot of programming experience.

[tggzzz]

Why would a video player suffer from TLB misses? By nature every frame is new data which must be fetched from an SSD and can be disgarded after showing it. That is repeating the same process over and over by a limited amount of code which easely fits in the cache. The limit is the data throughput of the system. A Postgresql database is a different beast entirely; you can't even begin to compare that.

An interesting experiment is to take a multicore SoC and create a square wave (based on the systick counter / monotonic counter) on a GPIO pin using a high priority process that is locked to a core. I'm pretty sure it will have very little jitter as very little gets in the way of executing it and SoCs typically run at >1GHz. I've seen surprisingly good behaviour on bit-banged I2C busses as well (not my choice to make it that way) on low end (single core, 400MHz SoCs) running Linux. It is not like the clock rate is allover the place or messages get interrupted a lot. Not that it matters from I2C though.

Go on then. Demonstrate the jitter when bit banging a 1MHz 50% duty cycle square wave.

Then add 205619Hz 732997Hz and repeat. Then add comms to remote controlling PC.

And then provide an answer as to whether being able to use an elephant gun to kill a fly in the desert (but not a town) is a worthwhile exercise.

No time to set that up right now. Maybe later. But the reasoning to use Linux is that it makes it easier to create a sophistigated GUI (if that is required) without needing a lot of programming experience.

And that will be a world away from your 205619Hz bit-banged output. It will drag in all sorts of software that will make hard realtime behaviour difficult to guarantee by design (or measurement).

Have fun waggling the mouse and typing/entering many many numbers to see ever increasing worst case latency

[Siwastaja]

LinuxCNC did include a test program which instructed the user to move mouse cursor quickly while browsing web and watching Youtube videos trying to catch worst-case jitter. Quite obviously this does not work out in practice because one would need to move mouse cursor while watching Youtube videos for hours straight to get any meaningful idea of worse-case performance, which is quite critical because the CNC tool snaps on one single incident.

As usual, nctnico has not tested his hypothesis, which is not supported by practical experience by all the others.

[nctnico]

LinuxCNC did include a test program which instructed the user to move mouse cursor quickly while browsing web and watching Youtube videos trying to catch worst-case jitter. Quite obviously this does not work out in practice because one would need to move mouse cursor while watching Youtube videos for hours straight to get any meaningful idea of worse-case performance, which is quite critical because the CNC tool snaps on one single incident.

As usual, nctnico has not tested his hypothesis, which is not supported by practical experience by all the others.

I guess you read past my experience doing low latency video players which rely on realtime performance on Linux. The most complicated one involved putting the output on an EGL canvas on Wayland which has several threads in parallel (one for drawing, one for decoding and one for management) all tightly knit together while running a GUI application in the background that can be brought up using a magic key press. Ofcourse this wasn't build on some crappy SoC or is supposed to do all kinds of other random stuff in the background.

[sokoloff]

I guess you read past my experience doing low latency video players which rely on realtime performance on Linux. The most complicated one involved putting the output on an EGL canvas on Wayland which has several threads in parallel (one for drawing, one for decoding and one for management) all tightly knit together while running a GUI application in the background that can be brought up using a magic key press. Ofcourse this wasn't build on some crappy SoC or is supposed to do all kinds of other random stuff in the background.

If you were early or late getting the frame to the SoC or GPU's HDMI driver, your system didn't crash and nothing broke. You and the user might in fact never even notice a micro-stutter. Getting a usually smooth 50 or 60Hz display is a pretty different problem from driving a bunch of GPIOs directly with sub-microsecond jitter. If you were driving the video display directly from GPIOs rather than using a video driver, disregard my comment here.

[nctnico]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

Meanwhile I did a test with an iMX8 platform (I designed a couple of years ago) using this simple program:

Code: [Select]

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <time.h>
#include <stdint.h>
#include <fcntl.h>
#include <unistd.h>

uint64_t time_us()
{
struct timespec now;
uint64_t retval;
clock_gettime(CLOCK_MONOTONIC, &now);
retval = (uint64_t) now.tv_sec * (uint64_t) 1000000 + (uint64_t) now.tv_nsec / (uint64_t) 1000;
return retval;
}

#define INTERVAL_US 1000

int main()
{
uint8_t h = '1';
uint8_t l = '0';
int fd = open("/sys/class/gpio/gpio66/value", O_WRONLY | O_SYNC);
if (fd < 0)
{
printf("fd failed");
return 1;
}

uint64_t next = time_us() + INTERVAL_US;


while(1==1)
{
while(time_us() <= next);
write(fd, &l, 1);
next += INTERVAL_US;
while(time_us() <= next);
write(fd, &h, 1);
next += INTERVAL_US;
}

close(fd);

return 0;
}
Then I used cset to make the program have one of the 4 CPU cores for itself entirely which results in this square wave:



While running I decided to start Eclipse on the system and compile a piece of C++ code (about 16k sloc) which took about 4 minutes (make -j 2 to use 2 parallel build processes). This takes the system to it's limits where it comes to processing and memory usage.

Without doing all the system calls and mapping the GPIO controls + reading the systick counter directly into the user space program things can likely be made even better.

[sokoloff]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

Jitter and pulse width are two different specs; the only thing they share in common is the unit.

[tggzzz]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

Jitter and pulse width are two different specs; the only thing they share in common is the unit.

Tee hee  Beat me to it!

Conflating the two does, ahem,  raise doubts about his understanding the points made by several people in this thread.

[nctnico]

I'm perfectly aware of what jitter is but in this case the number Sokoloff threw on the table just doesn't apply to what the OP requires. OP wants 0.5ms with 6 bit resolution. 2^6 = 64. 500us / 64 = 7.8 us.

[NorthGuy]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

OP said 1%, aka 5 us. Your results definitely don't meet this. By a factor of 10.

Even if they did, why bother with 4-core Linux when even a simplest PIC16 can do better? Is this a "justify government spending" contest?

[nctnico]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

OP said 1%, aka 5 us. Your results definitely don't meet this. By a factor of 10.

Even if they did, why bother with 4-core Linux when even a simplest PIC16 can do better? Is this a "justify government spending" contest?

Because the Linux system can also drive a super fancy GUI application while your PIC16 can not. I'm using the particular iMX8 system as a desktop system to do development on. Anyway, I wanted to show that realtime performance is way better than people expected even with something super simple and as I noted there is much room for improvement by bypassing the kernel entirely.

[PCB.Wiz]

OP said 1%, aka 5 us. Your results definitely don't meet this. By a factor of 10.
Even if they did, why bother with 4-core Linux when even a simplest PIC16 can do better? Is this a "justify government spending" contest?

Both have their places.
The OP's project has both research and production phases.
It is often useful to use a more powerful host, to allow better testing and data collection, and then for production deploy a correct sized-MCU.
It also comes down to what the developer is most used to (and thus most productive on).
 
On the general idea of using a more powerful initial host, here is another find from google.

https://www.eevblog.com/forum/testgear/cy7c68013-based-bus-pirate-and-logic-generator/msg1582012/#msg1582012
It uses a FX2 HS-USB bridge, to
generate up to 8 channels of PWM signals is a welcome addition. It is capable of producing frequencies from 1Hz up to 1MHz at a sample rate of 4 MHz.

[tggzzz]

Nobody ever specified sub-microsecond jitter here. Original spec was in the 0.5ms ballpark.

OP said 1%, aka 5 us. Your results definitely don't meet this. By a factor of 10.

Even if they did, why bother with 4-core Linux when even a simplest PIC16 can do better? Is this a "justify government spending" contest?

Because the Linux system can also drive a super fancy GUI application while your PIC16 can not. I'm using the particular iMX8 system as a desktop system to do development on. Anyway, I wanted to show that realtime performance is way better than people expected even with something super simple and as I noted there is much room for improvement by bypassing the kernel entirely.

And even more improvement by throwing the Linux kernel out completely.

[mawyatt]

Didn't mean to create such a controversy, just requesting some help with the development system and microcontroller selections

Anyway, to clarify the pulse details. Each pulse of the 8 sequential pulses just need to match each other within ~1%, and the pulse-width needs to be accurate time wise overall within ~5% over the 0.5 to 5ms range. Regarding the pulse resolution of ~6 bits, this is for the pulse-width range and just needs a resolution of ~5ms/64, or ~78us, or another way of saying the pulse width increment needs to be better than ~78us.

These pulse characteristics are easy to achieve with just a couple 555 timers and support circuitry, and we will be sending off single channel test PCBs today for fabrication, which we'll utilize to evaluate and model load characteristics, many of which are unknown at this time.

Later while we are "learning" about microcontrollers with the kind help of folks here, we'll be developing another custom PCB which will incorporate all 8 pulse channels, load drivers, and some other functions. This will also use 555 timers, and a CD4017 to create the sequential 8 pulse train, and a CD4060 for the single long duration (60~1800 sec) timer pulse. This will be an entry level system, mainly to gather more information, and hone the final specifications/features (hopefully with customer input) for the microcontroller application(s).

Thanks again for all the microcontroller and development system advice & information.

Best,

[Nominal Animal]

Each pulse of the 8 sequential pulses just need to match each other within ~1%, and the pulse-width needs to be accurate time wise overall within ~5% over the 0.5 to 5ms range. Regarding the pulse resolution of ~6 bits, this is for the pulse-width range and just needs a resolution of ~5ms/64, or ~78us, or another way of saying the pulse width increment needs to be better than ~78us.

Have you considered using DMA SPI and the MOSI line for the pulse train?  As I understand it, the SPI peripheral clock is quite stable (and accurate, if the main clock source is accurate) on many microcontrollers, but me being a hobbyist I have not verified this.

At 16 kbits/s = 2000 bytes/second, the width of each possible high/low state is 62.5µs.  If you use a microcontroller with say 32k of RAM available for just the DMA buffers, that gives you a maximum 16 second pulse train, completely programmable.

At a SPI clock of 52,428,800 Hz ≃ 52 MHz, the 32k RAM buffer is long enough for 5 ms, but gives you a resolution of about 19 ns.  Obviously the base clock should be chosen to suit both the minimum resolution for transition edges and the hardware used, but to hobbyist me it sounds a quite powerful approach.

For generating parallel pulse trains, you could add a serial-to-parallel shift register with a latch, and a counter chip to strobe the latch on the N'th pulse of the SPI clock line.  These would be synchronous to each other to within the limits of the latching shift register and transmission line effects, which might be useful too.

[NorthGuy]

Later while we are "learning" about microcontrollers with the kind help of folks here, we'll be developing another custom PCB which will incorporate all 8 pulse channels, load drivers, and some other functions. This will also use 555 timers, and a CD4017 to create the sequential 8 pulse train, and a CD4060 for the single long duration (60~1800 sec) timer pulse. This will be an entry level system, mainly to gather more information, and hone the final specifications/features (hopefully with customer input) for the microcontroller application(s).

I think you grossly exaggerate the time needed to learn MCU basiics. It's no more than couple of days, a weak at most from scratch to blinking LED, and blinking LED is a pulse, so the distance from blinking LED to pulses bit-banged on 8 pins is  not that long neither. So, I suggest you build another board in parallel, this one with MCU. Select the one you like, get the datasheet to figure out minimum connections and place it on your PCB. Then you only need FET drivers.

PS. If you want to use high voltage logic ICs, I suggest using HEF instead of CD, e.g. HEF4017 instead of CD4017 etc. They are more or less identical, but newer and better.

[mawyatt]

Yeah, but we are slow, real slow 

Seriously tho, we'll be doing loads of other things in parallel, so allocating time as there's no rush...yet

Know CMOS of various flavors well, probably just use HC or whatever we find in stock, 555 and logic will run from 5V, drivers from 12V.

Our main concern at present is more acquiring details on the various loads with the test PCBs, not getting to know the microcontroller. Having a couple knobs to tweak while evaluating the loads is attractive, as we can "tune" the pulse parameters to search for load resonances and such.

We were just inquiring so we could order the proper development system and be ready to dive into the microcontroller when time permits, and not have to wait for delivery, or out of stock situations.

Also doing all the design, simulations, schematic capture, PCB layout, parts searching, procurement, quality control and receiving, accounting, and bunch of other "family" related issues with our small company Wyatt Labs LLC of three, being, me, myself and I 

Best,

[Nominal Animal]

Having a couple knobs to tweak while evaluating the loads is attractive, as we can "tune" the pulse parameters to search for load resonances and such.

That's the main way I'm using microcontroller dev boards (mainly Teensies, although I do have a few others from AVR to Cortex-M7) for: to interface to sensors and display modules, with a straightforward Arduino firmware that is controlled from an application on the host computer, typically using USB Serial for the data transfer.  Teensy 4.x have lots of RAM and a native high-speed USB 2.0 that can sustain 25 MBytes/sec over USB Serial, so generating arbitrarily long pulse trains at Teensy's SPI clock resolution (high, considering you can run it at up to 600 MHz) would be a very straightforward matter.  Add a digital isolator, say a TI ISO6720 or ISO6721, and you have an isolated pulse train source at 1.71-1.89V or 2.25-5.5V logic levels, by also supplying the appropriate VCC and GND.  Only need a couple of 1µF bypass caps, too.

Any microcontroller with full-speed USB Serial interface and DMA SPI support can sustain about a million bytes per second, so something like continuous 1µs resolution pulse train should not be a problem at all.  You could even generate and save the exact pulse trains into binary files, and play them back as desired; it's just 7.5 megabytes per minute, or 450 megabytes per hour.

The higher pulse resolution rates interest me because of some experiments with PDM I've done: you could use an analog low-pass filter on the output pin, and essentially control also the amplitude of the signal (relative to the output Vcc).  This is where the high-speed USB interface would matter.

Alternatively, one could just use an 8-bit SPI DAC like MCP48CVB01, and one byte per minimum pulse edge interval.  At 100kSps (or 10µs resolution, 10µs between level transitions) you'd need 100,000 bytes per second (not a problem even with full-speed USB) or 6 megabytes per minute.

[PCB.Wiz]

Anyway, to clarify the pulse details. Each pulse of the 8 sequential pulses just need to match each other within ~1%, and the pulse-width needs to be accurate time wise overall within ~5% over the 0.5 to 5ms range. Regarding the pulse resolution of ~6 bits, this is for the pulse-width range and just needs a resolution of ~5ms/64, or ~78us, or another way of saying the pulse width increment needs to be better than ~78us.

These pulse characteristics are easy to achieve with just a couple 555 timers and support circuitry, and we will be sending off single channel test PCBs today for fabrication, which we'll utilize to evaluate and model load characteristics, many of which are unknown at this time.

Later while we are "learning" about microcontrollers with the kind help of folks here, we'll be developing another custom PCB which will incorporate all 8 pulse channels, load drivers, and some other functions. This will also use 555 timers, and a CD4017 to create the sequential 8 pulse train, and a CD4060 for the single long duration (60~1800 sec) timer pulse. This will be an entry level system, mainly to gather more information, and hone the final specifications/features (hopefully with customer input) for the microcontroller application(s).

Note a 4017 will have no gaps between pulses, which might be an issue ? Usually gate drivers and FETs have dead time allowances for finite turn off times.

If you are comfortable with CMOS logic, I did a quick trial of a FT232H in 245 CPU FIFO mode - ie this streams 8 bits wide. from HS-USB link, so you simply copy a file to generate pulses.

It needs a simple 2 wire handshake, but a rough always-there RDN stream hangs the link, so I did a quick kludge with a 1k5/BAW62 OR gate where RXF# gates the CLKOUT.
That uses the CLKOUT from FT232H to control the stream rate, and it works 'pretty well' over a simple virtual COM port driver. (VCP)

At 7.5MHz CLKOUT (sadly their lowest setting) it does not quite sustain that, but gets annoyingly close, averaging 6.67467MBytes/sec, with sparse gaps of 150us  (averages ~89% of 7.5M)

That suggests sub 6MHz? might sustain without gaps  (another thread on here mentioned 6~8 MBytes/sec from FT232H)

I think fast logic like 74AC161 + 74AC02 can give a /N crystal paced, hardware read that should work at any of the 7.5M/15M/30Mhz clock rates.
It should support a hard-wired choice of 6M,5M,3M,2M,1.5M,1MHz,500k round-numbers crystal clocked 8 bit wide data stream rates, for easy testing. (plus other CLK/N speeds too, eg /9 is 300ns updates)

The FT245 mode floats data pins between RDN strobes, so either a latch or pin-keeper type buffer would keep that long term static after the stream.

Addit:

Having a couple knobs to tweak while evaluating the loads is attractive, as we can "tune" the pulse parameters to search for load resonances and such.

If you like the 'knobs to tweak while evaluating the loads' approach, you could also use a 74LV123 monostable, triggers on RXF =\_ and drives a ==\__/== (variable low time) into RDN, and clocks a HC374 latch to manage the bus floats when RDN is high. 
Data says a 2k~100k resistor with 100pF will give 200ns~10us tick rates.

[NorthGuy]

If you are comfortable with CMOS logic, I did a quick trial of a FT232H in 245 CPU FIFO mode - ie this streams 8 bits wide. from HS-USB link, so you simply copy a file to generate pulses.

It needs a simple 2 wire handshake, but a rough always-there RDN stream hangs the link, so I did a quick kludge with a 1k5/BAW62 OR gate where RXF# gates the CLKOUT.

I did the same thing with FT601 (same interface as FT245 but 32-bit wide), except I put all the signals through flip-flops and connected the FT's RXF# pin to the flip-flop's CE#. The flip-flops hold the last state during the gaps in transmission.

This worked very well at 100 MHz. However there are lots of gaps. Even if you transmit continuosly from PC, you can achieve only about 80% of the maximum 400 MB/s traffic, even though USB has enough bandwidth. I don't know whether the gaps are caused by hardware, FTDI driver, or their DLL. The gaps were Ok for my application. But there are too many of them, some shorter, some quite long. It wouldn't work very well for signal generation.

[SiliconWizard]

If you are comfortable with CMOS logic, I did a quick trial of a FT232H in 245 CPU FIFO mode - ie this streams 8 bits wide. from HS-USB link, so you simply copy a file to generate pulses.

It needs a simple 2 wire handshake, but a rough always-there RDN stream hangs the link, so I did a quick kludge with a 1k5/BAW62 OR gate where RXF# gates the CLKOUT.

I did the same thing with FT601 (same interface as FT245 but 32-bit wide), except I put all the signals through flip-flops and connected the FT's RXF# pin to the flip-flop's CE#. The flip-flops hold the last state during the gaps in transmission.

This worked very well at 100 MHz. However there are lots of gaps. Even if you transmit continuosly from PC, you can achieve only about 80% of the maximum 400 MB/s traffic, even though USB has enough bandwidth. I don't know whether the gaps are caused by hardware, FTDI driver, or their DLL. The gaps were Ok for my application. But there are too many of them, some shorter, some quite long. It wouldn't work very well for signal generation.

Yes, of course. This is not going to work. There is absolutely no possible guarantee here neither from the USB bus itself nor from the OS all the way up to the user application.

Just use a couple-$ MCU.
A RP2040 can toggle I/Os at over 200MHz for $1 and it has USB FS for communicating commands from and to a PC host if needed.

[PCB.Wiz]

Yes, of course. This is not going to work. There is absolutely no possible guarantee here neither from the USB bus itself nor from the OS all the way up to the user application.

That broad claim is not quite true.
Of course Audio PC playback manages just fine, and I've done sustained no-gaps data streaming for many minutes from USB-UARTS, in multiple applications.
The key point, is to be aware of the possible gaps at the highest speeds, and make sure the hardware buffers and pacing are always below that.
It is entirely possible and practical, if you follow the simple rules.

[PCB.Wiz]

I did the same thing with FT601 (same interface as FT245 but 32-bit wide), except I put all the signals through flip-flops and connected the FT's RXF# pin to the flip-flop's CE#. The flip-flops hold the last state during the gaps in transmission.

This worked very well at 100 MHz. However there are lots of gaps. Even if you transmit continuosly from PC, you can achieve only about 80% of the maximum 400 MB/s traffic, even though USB has enough bandwidth. I don't know whether the gaps are caused by hardware, FTDI driver, or their DLL. The gaps were Ok for my application. But there are too many of them, some shorter, some quite long. It wouldn't work very well for signal generation.

Did you try lower read out speeds ?
The FTDI parts at very highest rates will have gaps, but at lower readouts the buffers have the ability to smooth those gaps.

[tggzzz]

The XMOS xCORE devices work well for USB audio.

They do.have a number of key advantages for hard real time.

Many fully independent cores that cannot interfere with each other's operation (up to 32 cores.and 4000 MIPS per chip).

Timing guarantees the are guaranteed by design not by measurement. When the IDE states a min/max number of clock cycles, then that will be met.

Language support for many core operation, based on concepts developed and proven in the 70s.and 80s.

Simply dedicate a few cores to the USB functions, and use the other cores for "real" work

[nctnico]

Yes, of course. This is not going to work. There is absolutely no possible guarantee here neither from the USB bus itself nor from the OS all the way up to the user application.

That broad claim is not quite true.
Of course Audio PC playback manages just fine, and I've done sustained no-gaps data streaming for many minutes from USB-UARTS, in multiple applications.
The key point, is to be aware of the possible gaps at the highest speeds, and make sure the hardware buffers and pacing are always below that.
It is entirely possible and practical, if you follow the simple rules.

Agree. I have used high speed data acquisition systems streaming their data at several tens of MByte/s over USB3 continuously to a Linux machine for weeks without any problems. This system was using libusb to do deal with the USB data transfer. And there are also quite a number of high speed SDRs that use USB3 to stream data into a host as a continuous stream.

[SiliconWizard]

Yes, of course. This is not going to work. There is absolutely no possible guarantee here neither from the USB bus itself nor from the OS all the way up to the user application.

That broad claim is not quite true.

This was not a "broad claim", it was a reply to the specific thing NorthGuy was talking about. With pin toggling at 100MHz.

Of course Audio PC playback manages just fine, and I've done sustained no-gaps data streaming for many minutes from USB-UARTS, in multiple applications.
The key point, is to be aware of the possible gaps at the highest speeds, and make sure the hardware buffers and pacing are always below that.
It is entirely possible and practical, if you follow the simple rules.

Looks like we're running in circles and back several pages ago.
Concepts like throughput, latency and buffering seem to be conflated here.

Of course, when latency can't be guaranteed (/has too much jitter) but average throughput is high enough, you can accomodate it using buffering.
And then, the larger the buffer, of course the longer the resulting latency you'll get.
"PC audio" relies on proper buffering, and USB audio is *guaranteed* only using USB isochronous modes (which guarantee latency and throughput allocation). The usual USB interface chips (be it FTDI or other similar) all implement USB bulk mode, I've never encoutered one that implement isochronous.

Can you still do continuous data streaming using USB bulk transfers? Sure. With appropriate buffering, again, which will increase latency. It may not be a concern in some applications, and may be in others.
And, even with enough buffering, USB bulk transfers can't, by nature, guarantee anything timing-wise. It may work as your requirements expect in some cases, in some conditions, but can't be guaranteed in general. You'll have better luck of course if the host is not too busy, you're handling I/O in a thread with elevated priority, and the USB port your device is plugged to is not shared with any other port on the same USB host chip (meaning the USB traffic is not shared.)

A FT232H has an internal buffer of 512 bytes IIRC, so *at best*, 512 successive 8-bit values can be generated at a fixed frequency. If we say, target 1MHz, that's 512µs of buffering. There can't be any guarantees of what will happen between these 512-byte buffers. Might be a negligible gap, or possibly up to a few ms depending on the conditions, and that will jitter.

If you want to generate longer streams of data with a guaranteed period, you'll have to further buffer data externally and use external circuitry to then use the data to generate whatever you need generated with precise timings. Buffers holding below a few ms of data are usually not enough to guarantee anything on a conventional PC running Windows (or even Linux with no particular RT option.)

Speaking of audio, I have implemented an USB generator a few years back, precisely using bulk transfers in USB HS (target output: 24-bit/stereo/96kHz). The minimum buffering that would work reliably was 6ms of audio data, and that was with great programming care. Even so, if using some heavy graphics at the same time, it would occasionally get audible gaps. If you left the PC alone with nothing much running in the background and limited action on the GUI, it was running gap-less well enough. So, that was usable (in my case), but not what we can call "guaranteed" either. A much larger buffering would have avoided these occasional gaps (when using the GUI), but that wasn't an option in my case.

In the end, you're not really saying otherwise.

But if your goal is to directly toggle IOs at even just a few MHz (let alone 100MHz) with a guaranteed period, from the data output pins of a FTDI chip in FIFO mode, it is not going to work reliably.
If that all required more details.

[PCB.Wiz]

A FT232H has an internal buffer of 512 bytes IIRC, so *at best*,

Nope.  Perhaps you should look at data before posting ?

....
But if your goal is to directly toggle IOs at even just a few MHz (let alone 100MHz) with a guaranteed period, from the data output pins of a FTDI chip in FIFO mode, it is not going to work reliably.
If that all required more details.

I never talked about 100MHz, you seem to diverge into false territory so easily.
Even your "just a few MHz" claim is hopelessly vague. What does that even mean ? 30MHz or 2MHz ?

As a reminder, these are the OP's lab testing requirements

Anyway, to clarify the pulse details. Each pulse of the 8 sequential pulses just need to match each other within ~1%, and the pulse-width needs to be accurate time wise overall within ~5% over the 0.5 to 5ms range. Regarding the pulse resolution of ~6 bits, this is for the pulse-width range and just needs a resolution of ~5ms/64, or ~78us, or another way of saying the pulse width increment needs to be better than ~78us.

That's 12.82k Bytes/sec base spec.

[NorthGuy]

Did you try lower read out speeds ?
The FTDI parts at very highest rates will have gaps, but at lower readouts the buffers have the ability to smooth those gaps.

There are only two - 100 MHz and 66 MHz. I dodn't try 66 MHz.

[NorthGuy]

However there are lots of gaps. Even if you transmit continuosly from PC, you can achieve only about 80% of the maximum 400 MB/s traffic, even though USB has enough bandwidth. I don't know whether the gaps are caused by hardware, FTDI driver, or their DLL. The gaps were Ok for my application. But there are too many of them, some shorter, some quite long. It wouldn't work very well for signal generation.

Yes, of course. This is not going to work. There is absolutely no possible guarantee here neither from the USB bus itself nor from the OS all the way up to the user application.

Of course, there's no guarantee. Nothing you can do if there's a noise on USB line which bogs down the throughput.

However, FT601 has its own clock and its own buffers and USB has enough bandwidth. So, FTDI could've ensured continuous transmission, but they didn't. I was about to investigate the details, but the project was scraped because of shortages. But I did find two shortcomings.

1. They move data in buffers whose size coinsides with the size of USB packets. They then switch the buffers feeding the FIFO. There's always a gap between switched buffers. This is certainly a hardware problem. It is not hard to switch buffers without gaps. They certainly could do this, but they didn't. Even though you cannot fix their hardware, there's no theoretical barrier.

2. There are other gaps - longer, infrequent, and unpredictable. These must be caused by the inefficiencies in the drivers or the DLL. It should be possible to fix these.

Anyway, if you re-buffer what you already have, for example with FPGA, you can get stable 8-bit wide 300 Ms/s signal.

<edit>I am not suggesting doing such thing to solve the OP's task.

[SiliconWizard]

A FT232H has an internal buffer of 512 bytes IIRC, so *at best*,

Nope.  Perhaps you should look at data before posting ?

Hence the "IIRC" I cared to add. it's 1024. So multiply what I said above by 2. Doesn't change the principle.

....
But if your goal is to directly toggle IOs at even just a few MHz (let alone 100MHz) with a guaranteed period, from the data output pins of a FTDI chip in FIFO mode, it is not going to work reliably.
If that all required more details.

I never talked about 100MHz, you seem to diverge into false territory so easily.
[/quote]

NorthGuy did and that's what I was replying to initially with this. The "you" is as in a general you, not directed to you specifically.

Even your "just a few MHz" claim is hopelessly vague. What does that even mean ? 30MHz or 2MHz ?

It doesn't matter. I gave a simple obvious calculation for just 1MHz. 1024 bytes = 1ms of buffer @1MHz. In this case, for longer sequences than 1ms, there  *will* be gaps. Occasionally or all the time as I explained above. And even when the buffer is 1024 bytes, since the USB packets in HS are 512-byte long, in bulk mode there is no timing guarantee in between two packets. So if the internal buffer is larger than 512, it may not always get full within the time frame than fits some given latency requirements, in which case there will be a gap.

All this to get realistic with what can be expected if we need some hard guarantee.
Again USB bulk can't guarantee delivery time anyway, so if we have a max deliery time requirement, we'll know we'll be hoping for the best. But if we can buffer a data sequence entirely and "play it back" later at a given fixed frequency, then bulk works plenty fine. OTOH, USB isochronous guarantees delivery time but can't guarantee data integrity, so may not be ideal either if exact data is required.

[SiliconWizard]

1. They move data in buffers whose size coinsides with the size of USB packets. They then switch the buffers feeding the FIFO. There's always a gap between switched buffers. This is certainly a hardware problem. It is not hard to switch buffers without gaps. They certainly could do this, but they didn't. Even though you cannot fix their hardware, there's no theoretical barrier.

USB SS is a bit more complicated than USB HS. You can send data by bursts than can be of various lengths, potentially rather long (much longer than the 512-byte packets of USB HS).
I don't know how the FT60x chips are designed, but I wouldn't be surprised if they supported only a fixed length, or at least a limited range of lengths.

2. There are other gaps - longer, infrequent, and unpredictable. These must be caused by the inefficiencies in the drivers or the DLL. It should be possible to fix these.

That's not surprising, pretty much what I was describing above. Delays between USB bursts can vary and sure this is down to the drivers, but even the OS itself probably can't guarantee much better anyway.
The max burst length that the FT60x chips have is also probably not helping.

Anyway, if you re-buffer what you already have, for example with FPGA, you can get stable 8-bit wide 300 Ms/s signal.

Yes, that would be the only *guaranteed* way of doing it. If the avaialble throughput is significantly higher than the throughput needed to keep your buffer filled, it should be fine.
But the problem is that I don't think you can have a strict guarantee that the USB bursts  will not stop occasionally for a longer time.
A high troughput and large buffer will of course mitigate that to a degree that may be good enough in a given application.

To get a better guarantee, you would probably at least need to implement your own USB device core (optimized for your use case) and your own driver.
Although, as tggzzz and I keep saying, don't expect hard real-time guarantees on a system that can't provide any.

<edit>I am not suggesting doing such thing to solve the OP's task.

Yep, that was pretty clear. Always interesting to experiment with that though.

[SiliconWizard]

The XMOS xCORE devices work well for USB audio.

They do.have a number of key advantages for hard real time.

Many fully independent cores that cannot interfere with each other's operation (up to 32 cores.and 4000 MIPS per chip).

Timing guarantees the are guaranteed by design not by measurement. When the IDE states a min/max number of clock cycles, then that will be met.

Language support for many core operation, based on concepts developed and proven in the 70s.and 80s.

Simply dedicate a few cores to the USB functions, and use the other cores for "real" work

Yes. But on top of that, as I mentioned earlier, USB audio (UAC1 and UAC2) uses isochronous transfers, which is key to getting a predictable data stream over USB timing-wise.
UAC can be implemented on a MCU that has a USB device controller. Sure you don't have the same guarantees on the firmware level. But if you do things carefully, it's doable.
UAC strictly uses isochronous transfers. Bulk transfers are not meant for transfers with predictable timings.

[tggzzz]

... but even the OS itself probably can't guarantee much better anyway.

WinXP (and I presume its successors) used to deal with application level deadlock by randomly perturbing the priority of threads. Given the lack of control of the environment, that's probably a reasonable approach.

Anyway, if you re-buffer what you already have, for example with FPGA, you can get stable 8-bit wide 300 Ms/s signal.

Yes, that would be the only *guaranteed* way of doing it.

Er, no.

I have used a single mid-range xCORE chip to, with guaranteed timing:

• count (in software) the 0->1 transitions on two 62MS/s input streams. The IDE inspected the optimised binary to determine the time-critical loop has 5 clock cycles to spare
• deal with front panel i/o
• some other hard realtime computation
• communicate over USB with a host PC

There are application notes showing how (in software) you can encode/decode 100Mb/s ethernet streams

Although, as tggzzz and I keep saying, don't expect hard real-time guarantees on a system that can't provide any.

Musk/Tesla FSD-beta car fake-it-'till-you-make-it "engineering" does appear to be infecting the zeitgeist. Let's hope imprisoning Elizabeth Holmes has a salutary effect.

Ditto "the unit test suite is showing a green light, therefore it works" mentality.

[NorthGuy]

1. They move data in buffers whose size coinsides with the size of USB packets. They then switch the buffers feeding the FIFO. There's always a gap between switched buffers. This is certainly a hardware problem. It is not hard to switch buffers without gaps. They certainly could do this, but they didn't. Even though you cannot fix their hardware, there's no theoretical barrier.

USB SS is a bit more complicated than USB HS. You can send data by bursts than can be of various lengths, potentially rather long (much longer than the 512-byte packets of USB HS).
I don't know how the FT60x chips are designed, but I wouldn't be surprised if they supported only a fixed length, or at least a limited range of lengths.

This is not that. Imagine you have N fixed size buffers and half of them are already filled (or may be even all of them are filled). You finish transmitting a buffer and now you need to switch to the next buffer. It is possible to build hardware in such a way that you do not lose any clocks while switching. But this is not done in FT601 - switching takes extra time.

Yes, that would be the only *guaranteed* way of doing it.

There's no guarantees in communications. Given enough noise, it's always possible that the throughput you're counting on is not there. But you can create a system which guarantees continuous function if the required throughput is achieved by the communication line. In normal conditions, USB SS can sustain 500 MB/s, and this would be enough for a decent signal generator producing 400 MB/s on the other end. But FT601 cannot sustain this. Not because there's a theoretical impossibility, but because it's built that way.