How do I handle programming for safety standards.

[peter-h]

I'd be interested to hear of any actual legal case that hinged on the manufacturer's choice of which standards/directives to apply - have there been any?

In EMC, I've never heard of any enforcement anywhere, and especially not in the UK. The trade mags which arrive every few weeks used to talk about nasty fines etc etc with the articles planted by EMC labs, but AFAICT it was all fake. Same with ROHS; stories about random inspections checking for leaded solder.

With injury potential, that's going to a very different thing.

[Simon]

Our products have to be EMC tested under CE/UKCA marking. All CE marking is largely bollocks, it relies on people getting injured or worse before anything is done about it. Then you present your evidence of actually giving a shit.

Trading standards in the UK are so poorly funded that you might as well deregulate.

But rules are rules.

[peter-h]

Trading Standards are not the issue. It is customers asking for a CE / ENxxxxx Declaration of Conformity, and some of them (very few) will ask for a lab test report. But that is again EMC. Re safety, I am sure that if somebody gets injured, Trading Standards will not be your problem

[mikeselectricstuff]

Trading Standards are not the issue. It is customers asking for a CE / ENxxxxx Declaration of Conformity, and some of them (very few) will ask for a lab test report. But that is again EMC. Re safety, I am sure that if somebody gets injured, Trading Standards will not be your problem

The only time I've seen it be an issue is whete a product is sold through a distributor, the distributor wants paperwork to cover themselves.
If everyone went through all the regulatory hoops that are strictly required, half the products out there, especially from smaller companies, would not be viable, so some pragmatism is needed. The market will decide what matters to them.

[SteveThackery]

It seems from this very interesting discussion that Simon's question doesn't have a simple and clear answer.

Simon, if I might make a suggestion? Whilst you are waiting for the situation to become clear, I would sit down with a blank sheet of paper and then brainstorm all the possible ways that the software can affect the safety of the product. Basically, identifying all the places that are "touched" by the software, both literally - in terms of the electronics that interface with the software - and in terms of which features and functions are implemented or affected in any way by the software.

Then, with that list in hand, imagine all the ways in which the software can adversely affect the safe functioning of the system. This might include bad control signals to machinery, crashing when receiving bad data from sensors, failing to respond quickly enough to stimuli, and so on. Aim to list every possible failure mode.

You will have performed an FMEA (failure modes and effects analysis, for any newbies watching).  Put all of this into a formal document which you will keep forever.

Then your work splits two ways. Firstly you need to develop a comprehensive test strategy, the aim of which is to test all of the failure modes you have captured. Each failure mode is likely to require several specific tests to exercise all the ways that failure could occur. This Test Strategy needs to be another formal document, stored forever.

Once you have performed all the tests, bug-fixing as you go, you should record the results in a Test Results document, again, stored safely.

Going through this exercise will definitely improve the reliability and safety of the system. But just as importantly, you will have a fully documented process from FMEA through to Test Results (the final version of which will show a 100% pass rate). This will stand you in great stead if some kind of event should happen, but also if a regulatory body asks for evidence of the efforts you have made to assure the safety of your product.

Most developers stop there. However, I advocate another activity: the second of the ones I alluded to above. Try to engineer your software so that it polices itself, looking for possible errors and stopping them before they propagate. This involves things like sanity checking (range checking) of all the data sources - including manual inputs. Also, sanity checking internal data structures. Watchdog protection, as recently discussed in another thread, and so on. You can decide the best response to issues that get identified: either repair, recover and continue; or a system halt. And most valuable of all is a log file containing all the information arising from the software resilience activities, usually implemented as a circular buffer.

I'm sorry if this is teaching you to suck eggs! 😄  As an ex-Reliability Engineer this stuff floats my boat. There is something immensely satisfying in making a system which is virtually bulletproof - high reliability, high resilience, high availability. But especially, it impresses compliance auditors and - hopefully not in your case - defence lawyers.

[Simon]

Most developers stop there. However, I advocate another activity: the second of the ones I alluded to above. Try to engineer your software so that it polices itself, looking for possible errors and stopping them before they propagate. This involves things like sanity checking (range checking) of all the data sources - including manual inputs. Also, sanity checking internal data structures. Watchdog protection, as recently discussed in another thread, and so on. You can decide the best response to issues that get identified: either repair, recover and continue; or a system halt. And most valuable of all is a log file containing all the information arising from the software resilience activities, usually implemented as a circular buffer.

I have no formal training in this stuff. What you suggest is what I am trying to do as I write my reusable code. I don't know what the requirements are, even the basic ones.

There is then the hardware itself. I am using  PIC32CX-SG family chips for my future projects. I knew it was safety rated but or course that is only the start, if you write stupid code, no one can help you. I have noticed that it has ECC capable RAM although this means having half the RAM as the other half is used for the ECC. I don't know if this impacts performance but it sounds like a good idea to use it. Yes watchdog, I tend to use that.

Ultimately I need to answer the question for example: If the user hit an engine stop button, how certain are you that the micro will always respond to it and that there will be an output that turns a transistor on or off that turns a relay on or off. Even if I go the fail safe way of energising the relay to allow the engine to start so that a failure of the basic hardware would stop the engine from starting in the first place. At what point can I say this is safe?

Are certain types of micro controller mandated?

As Mike said any proper approval is going to cost more than it is worth for single machine orders so I guess it is really not something I should worry about other than choosing safety rated COTS items that meet the requirements.

[eutectique]

What you suggest is what I am trying to do as I write my reusable code. I don't know what the requirements are, even the basic ones.

Sounds like a wide and sun-lit highway to disaster, on many levels. From personal (not knowing what are you being paid for), to design (miscommunication with stakeholders, moving goal posts, creating something more-or-less-good-enough-that-kinda-works), to legal (who is paying if someone got hurt?).

Good luck.

[eutectique]

Nice overall advice.

However, I advocate another activity: the second of the ones I alluded to above. [...] This involves things like sanity checking (range checking) of all the data sources - including manual inputs.

I would certainly implement these is round one, with proper requirements specification, test specification, and unit tests. Ariane 5 and USS Yorktown being good reminders.

[mikeselectricstuff]

Ultimately I need to answer the question for example: If the user hit an engine stop button, how certain are you that the micro will always respond to it and that there will be an output that turns a transistor on or off that turns a relay on or off. Even if I go the fail safe way of energising the relay to allow the engine to start so that a failure of the basic hardware would stop the engine from starting in the first place. At what point can I say this is safe?

If your emergency stop button is a software input, you're doing it wrong. This sort of thing needs to be  hardware, ensuring safety regardless of what the software does. By all means have software try to limit damage to the device, report diagnostics etc., but it should have no role in the actual protection of people.

Read the many reports on the Therac-25 radiotherapy machine disaster for a great example of why this should be the case.

[SteveThackery]

At what point can I say this is safe?

You can't, of course. All you can do is demonstrate due diligence, which will probably require you to do the stuff I said, and be able to prove it.

Having said that, I'm afraid I can't help with identifying the relevant safety standards. My input was purely about what you should be doing anyway, if safety is a critical factor.

[Simon]

Ultimately I need to answer the question for example: If the user hit an engine stop button, how certain are you that the micro will always respond to it and that there will be an output that turns a transistor on or off that turns a relay on or off. Even if I go the fail safe way of energising the relay to allow the engine to start so that a failure of the basic hardware would stop the engine from starting in the first place. At what point can I say this is safe?

If your emergency stop button is a software input, you're doing it wrong. This sort of thing needs to be  hardware, ensuring safety regardless of what the software does. By all means have software try to limit damage to the device, report diagnostics etc., but it should have no role in the actual protection of people.

Read the many reports on the Therac-25 radiotherapy machine disaster for a great example of why this should be the case.

And that is why we have estops that act directly on things. But not knowing what the standards say, I don't know what my options are should I want any more flexibility and need to choose between in house and bought in.

[Simon]

What you suggest is what I am trying to do as I write my reusable code. I don't know what the requirements are, even the basic ones.

Sounds like a wide and sun-lit highway to disaster, on many levels. From personal (not knowing what are you being paid for), to design (miscommunication with stakeholders, moving goal posts, creating something more-or-less-good-enough-that-kinda-works), to legal (who is paying if someone got hurt?).

Good luck.

I am referring to just any code I write that is not meant to control anything that is safety related.... I already try to write code safely just to save me the time fixing problems.

[uer166]

But not knowing what the standards say

Dude, you don't need advice of a bunch of forum randos, but from an actual compliance consulting individual or firm. This is where nobody here would have a clue, even those in the compliance world since they don't have the full context of what the heck is it that you're trying to do and what risks/laws/standards apply. The hard part is figuring out what standards apply, less so how to achieve and prove compliance. All the talk here about code and implementation is like step #36 in the pipeline, and you are at step #0.

[Simon]

I know. I am not interested in the legal side, that is not my problem. What I am trying to get to is, should I be told that I need to comply with regs, what would my general environment look like?

So I am bare metal coding, I don't use third party frameworks, is that even passable?
Will I have to use a safe RTOS? I assume that is I don't need an RTOS then it's a non question unless this is deemed safer than bare metal.

etc.

[uer166]

GENERALLY (not always), having less code means less questions, less documentation, and an easier time. When I was involved with UL991/UL1998 code, we had all the safety stuff run in ISRs below FreeRTOS, with enough provable separation to show that the RTOS crap could never interfere. If we insisted on putting those functions in RTOS tasks, then RTOS itself would have needed to be certified as a class B safety compliant library.

In your specific case, who knows, above is one data point from one project though (FWIW it was also certified in EU by 3rd party, not just a slapped on CE mark)

[paulca]

In my world the first question to ask is, "Who is the software for?"  In 99% of cases the answer for me is "Some company who will assume the legal responsibility for it through the contracts".  Therefore they will state the regulatory requirements/standards/compliance needed in the statement of work and contract.  They will be considered "deliverables" and "quality metrics" the customer can accept or reject.

An example, a customer of ours farmed their API out to a bank.  The bank said, "No thank you.  Not until you fix the database being unencrypted as our compliance requirements prevent us from using it otherwise.", so that same customer of ours contracted us to "fix it and encrypt the database".

The business aspect to understand here is this...   retro-actively modifying the code to use an encrypted database years after it was first written and while it is in production service with millions of dollars in renevue coming through it.... is a LOT more expensive than doing it in advance would have been.

This is the real secret to understanding what compliance and standards you might need.  It depends on your market more than anything and what they are willing to accept.

The fact your machine has an e-stop on it, suggests it was required to.  Why?  What specified it needed same?  Do the facilities and customers that use your machines have to adhere and comply to safety standards?  What are they?  Making your software compliant with those standards is a BIG beacon on trust for your customers and makes integration easier and nasty surprises later on less likely.

I haven't worked professionally with embedded software, but in all other software the basic standards include "All level testing".

* Dependency security scanning.
*Static code analysis - bug finders, typo finders, style and conventions etc.
*(optionally) Dynamic code analysis - Memory/CPU usage, memory allocation/stack allocation, etc.
* Unit testing - usually single function/method level testing.
* Integration testing - "component" level testing with test harnesses.
* Systems Integration testing - real components integreating in test
* End to End tests - real world scenario testing.
* User acceptance testing - Real world scenario testing with real users.

As I understand embedded the most of the work for most projects with none, will by in the integration and unit testing.  That' where you have to build the most involved test harnesses.

The outputs of such tests are recorded and can be provided on dashboards.  They can have defined limitations and allowances, targets etc.

A consultant or similar might cost you a week or more of daily rate to analyse your project, market, customers and surrounding industry and come back with a list of "MUST", "SHOULD", "COULD" standards that you could apply to.

Going out in a suit and asking your customers who probably did this already could be a "cheat" move or your customer could get the feeling, "Shouldn't our machine vendor not know this already?"

[Simon]

Our products so far have been basic. I am trying to introduce improvements where possible but the basic safety needs to remain the same. So really I am probably best making sure I am not in any safety loop.

[paulca]

I would... as a first step improvement... consider documenting the software status in as much detail as you can muster.  Basic software documentation with the usual heavy use of diagrams, figures and tables is obvious.  The immediate bonus is it makes "onboarding" easier if you need to later.  Advanced customers might ask for it, especially if they are regulated.   

Also "Issue tracking" and real time status.  You can use a free github for this and use their issue tracking, but the defacto JIRA is cheap for a minimal instance, while it is highly opinionated and extremely complex to setup right for various development processes.  You can also get self hosted variants of the same to remain local.

The idea ... or just to start with, open "issues" for the things you plan to do.  EG:  Quality and standards review.  and start opening more tickets for everything you discover might need "fixed", "improved", "documented", "removed" "deprecated", etc.  Pick / setup good categories.  When something has a deadline, specify it.  When a bunch of work groups around a "feature" you want to add, seek software engineering management techniques which will help, such as "git flow" strategies, Jira Epics.

Speaking of git.  Version control, off-site backups, etc are a must for any practical professional software.

When coupled with the issue tracking (especially if you host them in the same platform) it permits you to track the "What, why, where, when?" down to the lines of code changed... and then link them to issues (say raised in hardware tests).

All of these things come under the software _engineering_ side of definition, specification, and much more importantly, metrics and evidence.

When you want to know the status of your software, which bugs are open, how much tech debt there is, you should in theory be able to open a dash board and immediately report from it.

Automated tests are a good chunk of work to set up but considered essential in enterprise.  Basically you cannot submit any code without the full test suite being run against it.... and passing.  You need to put a significant amount of effort into this, often multiples of the effort to write/change the software, but if you can gain high confidence in the testing, it will speed up development and releases.

[peter-h]

Aren't there special RTOSs which are certified for safety critical stuff e.g. avionics?

I have been involved with avionics so I know this is crap anyway, but it helps to tick boxes.

[eutectique]

There is SafeRTOS (https://www.freertos.org/Partners/Software/SafeRTOS) -- a redesign of FreeRTOS.

[uer166]

Aren't there special RTOSs which are certified for safety critical stuff e.g. avionics?

I have been involved with avionics so I know this is crap anyway, but it helps to tick boxes.

For the very big stuff, VxWorks (airliners etc). QNX (Blackberry) is used in aerospace and automotive too. SafeRTOS has been mentioned but I don't know a real world use case.

Some use Linux kernels for safety stuff as well, despite not being "certified" whatever that means. For example, VxWorks may be DO-178c certified, but if you don't need that particular standard, it doesn't help your system design; this goes back to knowing what you need to comply with being non-trivial.

[peter-h]

There is SafeRTOS

It would be interesting to see a critique of FreeRTOS and which bits they fixed in SafeRTOS.

Some use Linux kernels for safety stuff as well,

Some avionics use Win NT Embedded; it became apparent when people saw the crash messages

[LM2023]

Some avionics use Win NT Embedded; it became apparent when people saw the crash messages

IIRC it is used only in non safety critical parts/functions.

[peter-h]

I would concur on tht one - it was an Avidyne PFD. A big hassle if it fails because you probably have only the one, but you "can" fly without it.

The worst product I know of - the KFC225 autopilot - does not AFAIK use an RTOS. It was "highly certified" but there are serious bugs in there which cause the servos to burn out, and these were never fixed because the coders left Honeywell c. 2002 so nobody left could even build it.

[Siwastaja]

GENERALLY (not always), having less code means less questions, less documentation, and an easier time. When I was involved with UL991/UL1998 code, we had all the safety stuff run in ISRs below FreeRTOS, with enough provable separation to show that the RTOS crap could never interfere. If we insisted on putting those functions in RTOS tasks, then RTOS itself would have needed to be certified as a class B safety compliant library.

And if you asked someone else, they would say RTOS is mandatory for your certification, or reduces amount of work, and they could equally correct.

First step is as you say to find the relevant standards. That is at least somewhat rigid - in some cases there could be different interpretations/opinions which apply, but 95% of the time, 95% of the experts would agree that exactly these 5 standards apply to your product. One expert might add one more, and other remove one.

But when it comes to interpretation of those standards on concrete terms of how to exactly implement the product, all bets are off. Standard is not going to say "RTOS". It is going to be, mostly, on a more generic level (with some weird specific requirements like this LED must blink for exactly 3 seconds). So different experts advice on very different implementations.

And implementing something that feels natural to others might not feel natural to you. Meaning, the one who writes the code should be totally reading the standards, and asking for advice for specifics, if necessary, but taking that advice with grain of salt.

Asking for generic advice, like on this thread, will lead to nothing, because that triggers people to say stuff like "you must use RTOS" or "you must not use RTOS", which is not helpful at all, while probably Simon should be using familiar design patterns to him, and check those against the standard, and only adjust specifics when absolutely necessary.