One large microcontroller or several small ones?

[langwadt]

Large single controller real-time and safety conscious designs have have an appalling record for reliability.  Even if the programmers and programming tools are up to it, and neither is a given or even likely, a single programming error or soft error can clobber the whole system.

Nowadays that is not a very good reason. If process seperation is a concern then you can use a microcontroller which supports memory protection (MPU). Modern (automotive) SoCs even support virtualisation right down to the video system. If the entertainment software goes haywire then the dashboard display isn't affected at all. IOW Nowadays you don't need physical seperation to achieve process seperation.

but there is probably good reason to keep the internet connected entertainment system/instrument cluster and the engine/brakes/steering physically separated so some guy in a basement in Hackistan can't drive you off a cliff remotely

[Kjelt]

And on security: meltdown is a real problem for process seperation and virtualization.

[nctnico]

And on security: meltdown is a real problem for process seperation and virtualization.

That is not really relevant because it is known and thus it can be solved.

[Kjelt]

And on security: meltdown is a real problem for process seperation and virtualization.

That is not really relevant because it is known and thus it can be solved.

Nico the only real solution AFAIK is to replace the processor with a newly designed one, the current patches rely on skipping some opcodes but relies on the os to inforce this. But for this discussion it might be too far offtopic.
I still think there are valid system cases where multiple processors are a good decision. Take a look at a pc for example would you really want the main core processor to do everything? Even sample all the aux input devices like mouses, keyboards , etc? How about controlling the harddisks? Although the original IBM PC with 8086 might have some performance issues doing this the current ones could handle it with ease but why don't they. Why use a complex USB protocol?
Wouldn't you say that it is a necessity in this case to make a good documented interface in order for all manufacturers to design, build and test these aux devices seperately? How often did you need to upgrade the firmware of your keyboard or harddisk? Was it essential?

[obiwanjacobi]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

 

[mikeselectricstuff]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

 

Like the choice of single or multiple processors, it's entirely application dependent.
There is no "good" or "bad", only more or less appropriate for a given set of constraints

[Gibson486]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

 

Like the choice of single or multiple processors, it's entirely application dependent.
There is no "good" or "bad", only more or less appropriate for a given set of constraints

Exactly. If everything were so black and white, they would not being paying engineers. Not everything is binary and if it was, lots of us would be in unemployment. There is no book that says, do this when this occurs and if there is, it is misguided. The books that give us the answers are the text books in school that just give us equations to formulate our thinking. That is the reason why the internet has so much conflicting info.  The reasoning behind it has to do with the application and constraints. Lots of the designs that were deemed "bad", were more likely due to the iterative process.  They started off as good decisions, but they were probably rushed into the rest of the design. Not everyone will be fortunate enough to put into a situation where they can carefully make architectural decisions.

[Tomorokoshi]

Not everyone will be fortunate enough to put into a situation where they can carefully make architectural decisions.

Does anyone really ever get that opportunity?

I'm working through several projects using the DAMS process.

It's all the rage these days: Design to Arbitrary Management Schedules.

[obiwanjacobi]

Okay, I worded my question poorly. I understand that 'it depends' 

Can we say something on what things to look out for, how you solved what problems in what way...?

[SiliconWizard]

Not everyone will be fortunate enough to put into a situation where they can carefully make architectural decisions.

Does anyone really ever get that opportunity?

At least in situations involving safety-critical devices, not only is it an opportunity, it's also a requirement.
And if you ever work in a company that designs such devices and doesn't care about architectural choices, don't just walk. Run.

I'm working through several projects using the DAMS process.

It's all the rage these days: Design to Arbitrary Management Schedules.

I know what you mean. It's somewhat related to all the "lean management" concepts that are currently in favor. Unfortunately, a lot of managers (usually at the top management level, not the middle managers) have been convinced that what works in production environments can be happily applied in any area of a company, including R&D.

[Siwastaja]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

 

If on the same board, SPI is usually a good candidate: even the smallest MCUs have it, bigger have shitloads of separate SPIs, it's often DMA'able (is that a word?) (compared to, say, I2C, where actually utilizing DMA and getting any benefit of it is often left as a dream). It's fast and robust, simple to debug. SPI is simple to make self-reset after the transmission, which is delimited/framed by actual physical signal, compare this to, for example, the state horror and lockups of I2C.

If on the separate boards and you need reliability for longer links, against noise, etc., consider CAN and RS422/485. CAN is great when you have quite some many devices together and everyone shouts something; RS422 is great as a link between two devices. Latter works even with most cheap microcontrollers which have UARTs. CAN seems rather OK mostly because apparently most MCU manufacturers have outsourced and bought the same bxCAN peripheral IP, which is of usable quality and documentation compared to what it could have been if, for example, ST had designed their own 

Retry logics are often more difficult than you'd expect! If possible, design periodic resend of important data in any case; implement timeouts on important things. Additional safety with counters that prove that the data is recent. This strategy is often used with CAN bus and in automotive. "Just flood the data", "throw bad data away" and "timeout to safe state if no recent data available" work way more robustly than trying to design a two-way "did you get it?" "do we need to resend?" logic. For example, how do you deal with lost resend requests? How do you deal with buffering all the data that might need resending? Don't try to implement an abstacted high-level message transfer protocol (i.e., something like TCP), chances are you are solving the wrong problem.

[GeorgeOfTheJungle]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

Easy peasy one UART per uC, for long distances use differential/balanced signaling (rs-422/485), the software protocol is up to you, but the simpler the better.

CAN is designed for multiple modules on a bus with message priorities and is thus needlessly complicated, avoid it except when you CAN't :-) and SPI is not designed for long distances and is 4 signals + GND for gods sake are you kidding me? A UART is one signal (half duplex single ended, wired or) or two (full duplex single ended or half duplex differential) or 4 if full duplex differential, +GND, there's nothing easier to use than that.

[nctnico]

And on security: meltdown is a real problem for process seperation and virtualization.

That is not really relevant because it is known and thus it can be solved.

Nico the only real solution AFAIK is to replace the processor with a newly designed one, the current patches rely on skipping some opcodes but relies on the os to inforce this.

Well, we are discussing a new design so there is no need to replace anything.

[Kjelt]

No just have to wait till the hardware is made, Q3 i believe Intel is shipping first silicon

[mikeselectricstuff]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

Easy peasy one UART per uC, for long distances use differential/balanced signaling (rs-422/485), the software protocol is up to you, but the simpler the better.

UART is a good first choice, but not always ideal as you need to deal with packet framing - an advantage of SPI is the chip-select takes care of this. SPI is also easier to bit-bash for transmit if you don't have enough peripherals, and doesn't care about system clock accuracy or jitter.
 
 

[Siwastaja]

Yeah, UART is often simple and great and everything, but....

The fact that SPI provides reliable hardware framing signal with no restrictions to packet length makes it really easier to use than UART for many purposes. Some microcontrollers even support managing the chip select signal, for example, by DMA.

Extra complexity added by framing seems to sometimes surprise young players. How do you do that with UART? Often by actually designing a protocol and writing a parser for it, easily 100 lines of code even for a rather simple task + specification you need to keep up to date. Bugs can easily sneak in as you need parser state.

With SPI, synchronous data swaps of two memory areas (simple self-documenting structs, for example), arbitrary length, even tens of kilobytes if necessary, between two CPUs can happen almost 100% automatically and very robustly, often with a few lines of code, and you have the job done. With UART, you are building and specifying message formats and messages and designing how to frame them, i.e., at what character does the previous message end and the next start?

But of course, it depends. Sometimes the UART solution is way simpler than SPI. Sometimes you want to give a simple command. And sometimes 8 bits [for a complete message] is enough, and UART can frame that for you!

CAN can be simpler than UART, as well. Often it solves the framing problem way easier than UART. Often, the ID is data type directly (i.e., it tells you where to assign the data), and 8 bytes of payload is quite usable! Now you don't need headers, delimiters, or parser code either. Many CAN-based systems are complex as hell, but it's not because they are using CAN. CAN seems to be trendy among "let's do everything the complex way" folks, but you can still use it in a sane way if you want to. I have replaced a complex UART horror (my earlier revision) with a simple and robust CAN solution. It's always nice when you can just delete a few hundred lines of message parsers and just do the underlying thing "directly" by using signaling that's more suitable for the task.

[MT]

Any paper around describing "CAN for us who dont like convoluted complex system solutions" or "CAN the simple elegant way"?

[GeorgeOfTheJungle]

There's no need to deal with framing in software/parsing, you can use the hardware to signal the start of frame with a break (a start bit that lasts a bit longer than a character time). DMX512 does it so too.

void on_rxErrInterrupt () {
    newFrame();
}

[nctnico]

Yeah, UART is often simple and great and everything, but....

The fact that SPI provides reliable hardware framing signal with no restrictions to packet length makes it really easier to use than UART for many purposes. Some microcontrollers even support managing the chip select signal, for example, by DMA.

Extra complexity added by framing seems to sometimes surprise young players. How do you do that with UART? Often by actually designing a protocol and writing a parser for it, easily 100 lines of code even for a rather simple task + specification you need to keep up to date. Bugs can easily sneak in as you need parser state.

With SPI, synchronous data swaps of two memory areas (simple self-documenting structs, for example), arbitrary length, even tens of kilobytes if necessary, between two CPUs can happen almost 100% automatically and very robustly, often with a few lines of code, and you have the job done. With UART, you are building and specifying message formats and messages and designing how to frame them, i.e., at what character does the previous message end and the next start?

I don't quite agree with this approach. First of all an SPI transaction can be screwed up by a single short spike on the clock line. Secondly sharing structs between various parts of the software is a major pain in the ass. If it gets changed in one place, it must be changed everywhere and how do you deal with backward compatibility? In many protocols you see a message consists of seperate strucs of data each with header which has the type, length and sometimes a version. Still not ideal but at least it has some way to deal with incompatible versions. Last but not least I'm weary of sharing large / many data structures. That is usually a sign a distributed system hasn't been partioned properly and thus there are too many dependancies which need to cross the border from one device to the other.

What I did for a system which needed distributed processors is the following: I used a bit-banged multi-drop framing protocol using NRZ signalling over RS485 because I had that available but it could have been UART based just as well. The payload of the messages consists of commands in plain text (which also makes framing easy: STX and ETX are the characters to use when using a UART). Every product I make has a command line interpreter on a serial port anyway so getting the system to work is just a matter of putting the payload of the messages into the command line interpreter and sending the result (OK, a value, whatever) to the sender of the message. Debugging is very easy because you can enter the commands through the serial port too (in a debug built) and dump whatever is being send over the 'network'. Backward compatibility is also easy. If a device can't deal with a command or if there are extra parameters the system doesn't fall apart.

[mikeselectricstuff]

But of course, it depends. Sometimes the UART solution is way simpler than SPI. Sometimes you want to give a simple command. And sometimes 8 bits is enough, and UART can frame that for you!

Or you can use a single unique byte value for start-of-frame, which makes a lot of things very simple.
One characteristic of UART is that either end can initiate a message, whearas SPI is typically master-slave.  Either will have advantages and disadvantages depending on system requirements

[mikeselectricstuff]

There's no need to deal with framing in software/parsing, you can use the hardware to signal the start of frame with a break (a start bit that lasts a bit longer than a character time). DMX512 does it so too.

void on_rxErrInterrupt () {
    newFrame();
}

Out-of-band signals like breaks can sometimes be useful, but can also have subtle issues once you start having things like fifo's and UART error interrupts. At the very least these are often not well documented. It also makes life harder if you need to transport over other protocols.

[GeorgeOfTheJungle]

There's no need to deal with framing in software/parsing, you can use the hardware to signal the start of frame with a break (a start bit that lasts a bit longer than a character time). DMX512 does it so too.

void on_rxErrInterrupt () {
    newFrame();
}

Out-of-band signals like breaks can sometimes be useful, but can also have subtle issues once you start having things like fifo's and UART error interrupts. At the very least these are often not well documented. It also makes life harder if you need to transport over other protocols.

Yes, I know, right you are  , it was in response to this:

Extra complexity added by framing seems to sometimes surprise young players. How do you do that with UART? Often by actually designing a protocol and writing a parser for it, easily 100 lines of code even for a rather simple task + specification you need to keep up to date. Bugs can easily sneak in as you need parser state.

[Siwastaja]

I don't quite agree with this approach. First of all an SPI transaction can be screwed up by a single short spike on the clock line. Secondly sharing structs between various parts of the software is a major pain in the ass. If it gets changed in one place, it must be changed everywhere and how do you deal with backward compatibility? In many protocols you see a message consists of seperate strucs of data each with header which has the type, length and sometimes a version. Still not ideal but at least it has some way to deal with incompatible versions. Last but not least I'm weary of sharing large / many data structures. That is usually a sign a distributed system hasn't been partioned properly and thus there are too many dependancies which need to cross the border from one device to the other.

I'm sorry but I'm really unable to find any valid point in your reply.

UART is immune to "single short spike"?

Checksumming against electrical corruption applies to any of the mentioned schemes similarly. Clear hardware framing makes this task so much easier, you can resync right to the next packet.

I'm actually using UART as well quite a lot and have tried several ways of framing, message ID'ing, error correction, etc. Not super complex stuff, but if at all possible, I'll avoid it.

Keeping the communication structure on the both sides pain in the ass? Tough game, that's what you have in the multi-MCU system. I can't see how you are going to magically get rid of this task. In multi-MCU system, especially on the same physical board, a simple struct{} is by far the easiest, and can even share the same .h file, so only once place needs modification. It's also readable enough so that it self-documents. I can't figure out any way to get it more maintainable than that - what you suggest requires 3-4 orders of magnitude more maintaining and matching between two systems. Define packet structure - change on both sides when necessary. Define message types - change on both sides when necessary. Document everything. Keep the documentation up-to-date.

Unless, of course, the system is just simple enough. For one or two simple messages, it doesn't matter. Especially if it doesn't need to be super robust.

ASCII UART based systems are nice for human intervention; most I have seen are buggy as hell and total pain in the ass to automate. Yeah, it's OK to use a buggy ASCII parser when you are a human. Everything happens slowly when you type, so race conditions are left untested. Framing does not work reliably? You hit some random keys or reset manually until it works again. Some command is totally left unaswerred? Not a big deal, just command it again. Nothing works? The parser crashes due to wrong type of line feed in use - configure your terminal until it works. As a human, we have a great troubleshooting system called "brain" which "autonegotiates" the communication link.

Last time I needed to actually use - i.e., not test it on the lab table - such a product (a scanning 2D LIDAR using such a "simple" text-based UART interface for configuration, switching to binary communication after init so you cannot do anything with it on terminal, anyway ), it took about 300 lines of code and a few days of bug hunting development and reverse engineering to get the it work reliably. The documentation of communication is nicely laid out and looks professional, but it doesn't match the actual behavior 1:1. Things like "it randomly completely ignores a certain message" are extremely easy when you initialize it as a human. Or when it replies something completely unexpected back, you use your super capable AI Brain^tm technology to parse and decide what to do! But in actual production code, it's added complexity to the state machine, again. And they seem to be going out of business right now.

In these kind of systems, you typically have two layers of state machines (actual messaging, and message-level parsing/framing/error checking) on both ends.

And yes, with this LIDAR, it's just normal that it gets stuck or crashes and you need to reset it. They even offer a handy hardware signal to make that easy. But they provide checksumming! Never had a checksum failure, though.

Sure, you can make the ASCII UART interface work, and it has many appealing points. But for multi-MCU system? Nah, it's not super difficult. But compared to many much easier options (which I have covered already), this is completely on a different level of complexity than a simple memory swap using a framed low-level protocol such as SPI or CAN, equipped with a simple checksum automatically calculated over the framed data. Especially if you need robustness and/or high performance in the system.

My issue is that I'm doing such complex systems alone right now that I simply don't have time to play "let's design a nice protocol" games - the solutions just need to be simple and robust.

And in the MCU world, we always don't have full libc with all fancy-pancy sscanf() stuff available.

[donotdespisethesnake]

So if you would choose to use multiple processors. what would be a good protocol for the communication?
(both electronic design (diff-signals?) as software design (retry-logic?))

 

The discussion to that question will likely go on for another 10 pages. Which also shed lights on why it is not a good idea to use multiple processors if you have a choice.

To answer the original question another way, only consider using multiple processors if you have a good understanding of comms and networking. There are many traps for the unwary.

[nctnico]

I don't quite agree with this approach. First of all an SPI transaction can be screwed up by a single short spike on the clock line. Secondly sharing structs between various parts of the software is a major pain in the ass. If it gets changed in one place, it must be changed everywhere and how do you deal with backward compatibility? In many protocols you see a message consists of seperate strucs of data each with header which has the type, length and sometimes a version. Still not ideal but at least it has some way to deal with incompatible versions. Last but not least I'm weary of sharing large / many data structures. That is usually a sign a distributed system hasn't been partioned properly and thus there are too many dependancies which need to cross the border from one device to the other.

I'm sorry but I'm really unable to find any valid point in your reply.

UART is immune to "single short spike"?

Yes. Because a UART is sampling the signal (hence the 8x or 16x clock) the spike doesn't even register. Remember that UARTs are invented as part of a physical layer to transmit data over very long distances. Checksums are nice but it helps if the physical layer has immunity to spikes built in.

I think the whole world disagrees with you that text based protocols are bad. HTTP, VOIP, JSON, XML, SCPI are good examples.