I will simulate events during design and implementation and maintenance of high-level FSMs.Yup, I classify FSM's in the same bin I do algorithms, and do simulate those as well; especially if I want to check for end-states (where the FSM would get "stuck") and state chains that are not accessible, or are accessible in undesired situations.
Granted, I use FSMs mostly for parsing structured data and for user interfaces. Again, I consider these simulations closer to unit tests, as I'm usually worrying about correctness here.
I wonder if I'm the only one who mumbles and waves their fingers in the air when working out particularly complex FSMs. I do like to use pen and paper sketches as a swap file, too; those "notes" become gibberish as soon as I move on, even for myself: I literally sketch something to represent a detail whose inner complexity I cannot hold in my mind while working on higher-level structure... I also like to walk in circles (increases blood flow) when thinking hard. These make me utterly incompatible with cubicle farms
There has been a lot of academic work on FSMs, attempting to formally prove/disprove useful properties such as being deadlock and livelock free.
QuoteGranted, I use FSMs mostly for parsing structured data and for user interfaces. Again, I consider these simulations closer to unit tests, as I'm usually worrying about correctness here.Ugh. Run away. Compilers and anything related have negative interest for me!
FSMs seem to be begging to be expressed as hierarchical diagrams. Harel Statecharts are a good start, but that "language" has always struck me as having unbeneficial complexity.
I cannot even count the number of times I've woken up with a better solution in mind than the one I worked on the previous day, simply because my subconscious reorganizes things when I sleep.
Quote from: Nominal AnimalI cannot even count the number of times I've woken up with a better solution in mind than the one I worked on the previous day, simply because my subconscious reorganizes things when I sleep.
I don't think it's the subconscious, IMO you quite consciously work the problem, but the sleep-induced interruption allows you to abandon the safe paths your mind kept walking during the previous day and approach it from a surprising angle.
But problems requiring such interruptions, telling to the teddybear, the staring-to-the-code/walking cycles, are mostly the non-"unit-testable" or -splittable problems.
#ifdef TEST
#include "test/testsuite1.h"
//#include "test/testsuite2.h"
test();
#endif
I do believe it is my "subconscious", specifically related to what happens in REM sleep, where the mind organizes the memories; does the proverbial "housekeeping and cleaning" tasks. The discovery is then the effect of the mind removing the unnecessary clutter, if you will, with my conscious mind discovering the now easier to see solution.
The hardest part about unit testing is writing testable code.
The amount of effort you would need to go to, to write testable code in MCU world would probably make a "wide coverage" impractical.
Note. I say "testable code" with specific meaning, which is what unit testing is about. It says nothing, absolutely nothing about whether the code 'functions' as per requirements. The idea is just to test the code does what the developer intended it to do.
When you come to testing functionality, there is hardware involved, so it becomes an integration test. When everything is connected up together that is a system test. When real users are using it, it's a user acceptance test. Then alpha, beta, gamma etc. etc. etc. ad.inf.
Trying to pick a suitable MCU world example... say DMA'ing UART buffers for Tx and Rx. You do some detection, pattern matching and formulate some response and send it.
In most MCU projects people just start writing hardware UART code and then the "business logic" just grows like moss around it. Completely un-unittestable. At best you can integration test bits of it with a pyschical test harness maybe, but you are more likely to just skip all testing all the way through to full system test. This is fine until you have 2 or more engineers working on the thing, or even one developer with multiple change lists open. Then it ruins you day and your week.
In the Big Iron world, assuming we have no internet connection to go download a framework that does this already, already fully tested and in a full RELEASE stable state... assuming we don't have the memory either.
I/We would start by splitting it into layers.
* Code which does the analysis, pattern matching and produces the correct responses - this code knows nothing about buffering or UARTs.
* Code which manages buffers. Provides them, cleans them, queues them, blah, blah.
* Code which sends and receives buffers via UART.
Not only does these 3 layers allow you to test the top two in "code only test harness", aka unit test, but you can swap any layer component for any other which complies to the same contract (header file).
The bottom one can also be tested much lower level. You just need to feed it buffers, so a proper integration test of the UART hardware access layer is possible.
The "isolation of concern" applies not just to the code, but the programmer and which component he is working in. As long as it meets it's contract it is said to be good. A unit test that proves it meets it's contract makes it formally tested to do so. You are only testing that one bit of code, not the whole application. If the whole appliction is broken, verifying if a component is function correctly in isolation is very advantagous.
You do have to keep remembering where you are though, what memory footprint you can afford, how much of the extra reference hops and some unnecessary protections cost in terms of cycles where that is important.
The biggest, by far, advantage of unit tests, which is so often forgotten is not that they prove the code works. It's that in 6 months time, when someone just did a 3 week long refactoring... that the code STILL works.
Analogy with electronics.... your "units" are ICs. Except that you can make your own. So while scattering a bunch of passives and discretes all over the PCB might be required in RF stuff, most people would prefer to get an IC which encapsulates that functionality. If you were in a lab capable of making fab'd prototype ICs, you could see the appeal of pushing functionality into ICs where they can be tested in a repeatable way and then manufactured in a pre-tested, quality form and re-use in other prjects. Your ICs are your units and the test hardness for them the unit tests. Having those ICs run with the minimum amount of external support is the art of writing unit testable code.
How do I write a Unit test for that?
This is exactly the scenario where unit testing makes no sense.
What you are describing sounds like there is trivial amount of processing done. There is no point in unit testing that, you will waste a lot more time because this is not where errors in the code like this come from. They come from integration and running on the real hardware asynchronously.
The way to test things like this is to do functional testing using real hardware. You will need a test setup that can generate external inputs on command from the PC and read back the state of the firmware. Then PC runs a series of tests where it generates valid and invalid sequences and then looks for the reported results.
This is where you get the most out of your tests. You can leave it running for days just generating random noise on the pins and checking if something crashed or misbehaved.
Well it's part of a bigger system that will get much more complicated later.
It's audio equipment with front panel buttons. I don't have anything to simulate pressing the front panel buttons. Plus it makes noise.
It's audio equipment with front panel buttons. I don't have anything to simulate pressing the front panel buttons. Plus it makes noise - it's an untuned radio station.
But testing trivial units on their own will not improve overall system that much.
Unit test make sense if there is a piece of code that is hardware-independent and has fixed inputs and outputs. A good example would be if you have an ADC reading a buffer of data and then some DSP algorithm runs on this buffer. In this case you would unit test just the algorithm. You can generate test buffers and expected outputs. You don't simulate the ADC itself.
It is hard to tell without looking into the details of the exact equipment,
You will need button pushers to test the hardware handler.
EDIT: Consider, as I am, right now, that SPI, I2C and UART are all options for MCU<->MCU communications.
It is hard to tell without looking into the details of the exact equipment, but unit tests for things with user input like this are rarely useful.
I have seen people put asserts in their code like this, in enterprise in big iron.... I want to slap them. "Thou shall not abort the VM from a library! You are servant, not master, you shall NOT control my runtime!"