Comments

[tggzzz]

There is absolutely no way to reliably infer developer intent from the code itself.  The code is the result, not the design; and any particular feature –– or even the overall approach –– may be an unintended error, or more commonly, a misunderstanding or a "thinko": a logic bug/error similar to a typo.  These happen to every single human programmer.

Yes, and it isn't confined to software.

I was once given the job of converting a TTL design to the new-fangled PALs. That's very difficult if you don't know which bits of functionality a 7492+3*7474+gates are actually using. It is much easier if you know it is a swallow counter, or whatever.

Basically not knowing what something is supposed to be doing leads to "reimplement it all including the bugs".

The true purpose of comments is to provide the information the code does not provide, and that is intent. 

I like to point out that good comments enable you to accurately ignore 99.9% of the code, and quickly focus on the 0.1% that needs attention.

[Nominal Animal]

There is absolutely no way to reliably infer developer intent from the code itself.  The code is the result, not the design; and any particular feature –– or even the overall approach –– may be an unintended error, or more commonly, a misunderstanding or a "thinko": a logic bug/error similar to a typo.  These happen to every single human programmer.

Yes, and it isn't confined to software.

Very true.  Even when just optimizing some workflow, the key is to find the intent/purpose of the whole workflow, instead of trying to micro-optimize the already chosen approach.

I like to point out that good comments enable you to accurately ignore 99.9% of the code, and quickly focus on the 0.1% that needs attention.

I have to (try to) write such comments because of exactly this.  It reduces my cognitive load, and lets me spend my meager brain resources on the important stuff.

It is one of my biggest regrets, that I learned the usefulness of good comments so late in my programming career.  It is damned difficult to try and learn later on; you cannot just "tack" that on top of an existing programming knowledge, it has to be "built in" to work well and come out naturally.  I've found that for languages I learned afterwards, it is much, much easier to learn to write good comments as you learn the ins and outs of the language itself.

For example, consider the following Python Fibonacci sequence generator:

Code: [Select]

def fibonacci(n1, n2, count=None):
    """Generates the Fibonacci sequence starting with 'n1' and 'n2',
       optionally limited to 'count' first elements (minimum 2)."""
    yield n1
    yield n2
    if count is None:
        while True:
            yield n1+n2
            n1, n2 = n2, n1+n2
    else:
        for i in range(2, count):
            yield n1+n2
            n1, n2 = n2, n1+n2
If you save that as say fibo.py, and run pydoc3 fibo, you'll see
    FUNCTIONS
        fibonacci(n1, n2, count=None)
            Generates the Fibonacci sequence starting with 'n1' and 'n2',
            optionally limited to 'count' first elements (minimum 2).

To print the first 1000 Fibonacci numbers, one per line, you could use
    for n in fibonacci(0, 1, 1000):
        print(n)

This is the kind of "intent" I'd like to see in code comments.  The wording may feel odd when reading the docstring in the source code, but I like to use pydoc3 myself to see a quick synopsis of the interfaces a Python file provides, and "tune" the wording to read "best" that way.
As to how to best name such generator functions in Python, whether one should use type hints, et cetera: those are language-specific details best discussed elsewhere.

[tggzzz]

There is absolutely no way to reliably infer developer intent from the code itself.  The code is the result, not the design; and any particular feature –– or even the overall approach –– may be an unintended error, or more commonly, a misunderstanding or a "thinko": a logic bug/error similar to a typo.  These happen to every single human programmer.

Yes, and it isn't confined to software.

Very true.  Even when just optimizing some workflow, the key is to find the intent/purpose of the whole workflow, instead of trying to micro-optimize the already chosen approach.

Agreed.

Even if not taught it explicitly, any engineer ought to rapidly come to realise that they should listen to their customers stated desires in order to understand their needs. As I've put ir elsewhere... Don’t ask “Can you give me a lift into town?” Do ask “Can you give me a lift into town, so I can replace my broken frobnitz?”. The answer might be “There’s a spare frobnitz in the attic”, thus saving time, money, the environment – as well as making some space in the attic.

It also applies to salesmen and marketeers. In order to be able to effectively sell something to your customer, you have to understand what they give to their customers (and what they receive from their suppliers). That enables you to be able to say buy my X and it will help you persuade your customers to buy more of your stuff.

[SiliconWizard]

Yes, although one may question what is meant by "intent" exactly. Sometimes the developer's intent is not even completely known to themselves. While the intent is usually thought of as a willingness to implement some specififcation correctly, that may not necessarily be entirely the case, or, unfortunately way too often, the specififcation may itself not be accurate, or may not even exist at all. other than very informally.

So, there are even more reasons to consider that the code is definitely not proper documentation of what should be. But one may argue that it is the best possible documentation of what it *is*, which is kind of obvious, but often what people mean by documentation.

But comments? Yes, comment the what and why, and the how only if it's something particularly clever, otherwise refrain. Also, I recommend commenting when fixing a bug that was not an obvious bug, to document it. Main reason is to avoid anyone else looking at the code later on (or even yourself, after a while) reverting back to a buggy version that may look more obvious, but was the buggy approach to begin with. Relatively frequent when working on largish code bases in teams.

[bpiphany]

Of course the self explanatory code require a language and context that supports it. Low level C and the low level likes probably isn't that.

I'm not a low level programmer. But just an excerpt from my latest attempt. Found some new AVR aliases(?) for the register bits and things. Makes the names a bit more informative. And at the next abstraction layer up from that giving names and variables explanatory names should be perfectly possible.

Code: [Select]

#define BAUD_RATE 56700
#define USART0_BAUD_RATE (((float)F_CPU*64/(16*(float)BAUD_RATE))+0.5)
inline static void init_usart(void) {
  PORTMUX.CTRLB |= PORTMUX_USART0_bm;   /* Alternative pins for USART0 */
  PORTA.DIRSET   = PIN1_bm;             /* TX-pin output */
  PORTA.DIRCLR   = PIN2_bm;             /* RX-pin input  */
  USART0.CTRLA   = USART_RXCIE_bm;      /* Enable interrupt on recieve */
  USART0.BAUD    = (uint16_t)USART0_BAUD_RATE;
  USART0.CTRLC   = USART_CMODE_ASYNCHRONOUS_gc
                 | USART_PMODE_DISABLED_gc
                 | USART_CHSIZE_8BIT_gc;
  USART0.CTRLB   = USART_TXEN_bm
                 | USART_RXEN_bm;
}


[Nominal Animal]

Yes, although one may question what is meant by "intent" exactly. Sometimes the developer's intent is not even completely known to themselves.

I'm using the word 'intent' for intention, goal, plan, purpose; with emphasis on plan, as in overall design.
(The case where the developer doesn't have an overall plan or intent for the code, I call "throwing spaghetti code at the wall and seeing what sticks".)

While the intent is usually thought of as a willingness to implement some specification correctly

No, that's on a different level of complexity, something that belongs in the supporting documentation, in my opinion.

But that does bring up a really important point/question: What is the proper level of complexity for useful comments?

Ataradov wrote in reply #30 that they don't write comments, and that they often rip them out before analysing code.  I think that is most likely due to having had to work too much with code with useless and counterproductive comments, because they indeed do more harm than good.  My own preference differs, but I don't like fixed rules requiring comments either.  To me, they're annotations about what the developer is trying to achieve; important when used correctly, worse than useless when used incorrectly.

If you implement a command-line sort-like utility for some specific data you often use, and instead of reading the data into an array you use a self-organizing data structure (tree, heap) to minimize the human-observable latency (rather than minimizing CPU time used, because you cannot start sorting an array before it is completely read), this overall design is something that belongs to the documentation, and not somewhere in the comments.

That is, the overall purpose and approach of the code belongs to the documentation, and not in the comments.  So that puts a "ceiling" on the level of complexity.  And the code itself already tells us what it does, and repeating that in comments is definitely not useful, putting an even clearer "floor" on the level of complexity.

In my opinion, we need to assume understanding of (or capability to find out about) algorithms and known named approaches. For example, if you implement code that calculates Newtonian trajectories for particles in 2D or 3D using velocity Verlet integration, you should only need to name it in a comment; and not try to explain in the comments how velocity Verlet integration works.  Whether one describes velocity Verlet integration in the documentation depends on the overall purpose: I would only do that myself for scientific software (in a separate LibreOffice or LaTeX source file plus .pdf output file).

A majority of cases where I've found comments to be useful is when they describe expectations, especially unusual or not-easily-inferred ones.
Locking (threads, files, databases) is particularly common.  Another is typical parameter values/ranges, when an approach is used that is really fast and precise for almost all cases, and only falls back to a "slow" code path for unusual parameters; and this choice is based on profiling. ("Profiling shows X=Y in 99% of calls, so algorithm Z is used here.")

I also count "use [foo algorithm] to do [bar]" as an "expectation" in the above paragraph.  It is common for algorithm implementations to miss some corner cases.  A typical one is binary searches when the data contains duplicates: most implementations return one of the matching entries if found, and not necessarily a specific one (like the first one of the repeated values).

So, there are even more reasons to consider that the code is definitely not proper documentation of what should be. But one may argue that it is the best possible documentation of what it *is*, which is kind of obvious, but often what people mean by documentation.

True.  It is often this shift, switching from what the code does, to type a concise comment about any surprising/unexpected/un-inferrible choises made related to the purpose/intent/plan for the code, that makes writing good comments so hard.

(It is also why I believe it is easier to learn to write good comments when one learns that particular programming language or its general approach, rather than later on.  We humans are creatures of habits, and learning new habits is hard when one has to (even partially) un-learn a previous habit first.)

Also, I recommend commenting when fixing a bug that was not an obvious bug, to document it.

Most definitely.  If you find something unexpected or obscure about the code, and fix or verify it, add a comment about it!

I warmly recommend the common format of starting such comments with the ISO date (YYYY-MM-DD or YYYYMMDD) and username, say
    // 2023-12-31 nominalanimal: X is correct, because Y.
as it makes investigations using source code versioning systems (svn, git, etc) faster.

[coppice]

All of the above is why I tell learners to focus on writing comments that describe their intent, instead of describing the code.

That sounds like a great idea, but we need to deal with real human beings. I've never seen comments consistently try to describe intent. I've only rarely seen them say valuable things like "This bit might seem weird, but it deals with the following non-obvious situation - xxxxxxx". Having spent a good chunk of my career getting screwed up projects back on track, my first step became to use scripts to strip all comments from the code, except for the file headers. The comments rarely agree with the code. They rarely comment on the purpose of the code. They are just dead weight, making what is usually hard to read code, full of meaningless variable names, bulkier to read through. Unless you can find a way to change human nature, I don't see a way to consistently improve on that sad situation. The best I've managed is to get people to be more thoughtful about variable names, so at least those say something meaningful.

[Nominal Animal]

All of the above is why I tell learners to focus on writing comments that describe their intent, instead of describing the code.

That sounds like a great idea, but we need to deal with real human beings. I've never seen comments consistently try to describe intent.

Just because the world is full of shit now, does not mean everything has to be shit, or that the world has to be full of shit forever!

Yes, it is certain that the majority of software will be shit for the foreseeable future.  What I personally try to do, is help those who are interested in producing something other than shit; those who are trying to build something new and useful, rather than get paid to produce more shit to be sold to hapless victims.

There is zero chance of me (or us) changing the majority of programmers, or even a significant fraction of them, but we can help those who are interested in rising above the generic muck, and help make the best in the future better than the best we have now.  That may or may not be sufficient, but it is better than giving up.

The best I've managed is to get people to be more thoughtful about variable names, so at least those say something meaningful.

That's better than nothing, isn't it?  It's not like you've given up on all developers: you're just concentrating on those you can affect.

I mean, this is not something I nag people about in real life, really.  I don't even rant about it face-to-face: after successful projects or problem-solving sessions, I tell anecdotal stories of why I wish so much I'd have learned to write better comments as I was learning to program in general, as it would have saved so much effort and made code maintenance and long-term development so much easier.

And, when discussed, I describe the reasoning for (here, experience leading to) my own opinions.  (My opinion does not matter, but the reasons for my opinions might be useful to others.)


New programming languages are often discussed here and elsewhere.  While C is the most widely used low-level embedded programming language, it is by no means perfect.  I've occasionally discussed what properties a better-than-C language for low-level embedded programming would have (although I've done much more systems programming using POSIX C myself), but I've also mentioned I do not intend to create my own "replacement C" at all: just experimenting with changing the standard C library with something better suited to today's needs suffices for me.  (The ISO C standard splits C into "freestanding" and "hosted" environments, with the standard C library only available in "hosted" environments, and "freestanding" having basically zero runtime and no standard library functions present (except for a few special functions like memcpy(), memmove() that the compiler itself may generate).  Replacing the standard library with your own functions is quite easy, and typically only needs a few compiler and linker flags to be set.  If running under an OS, you do need to implement a syscall interface for each specific hardware architecture and OS using e.g. inline assembly, but it's all quite straightforward.)

That is the exact same thing: not an effort to change everything or everybody at once, but make it possible for the future to be made better than it is now by those who are interested in it.  Help and advice, not demand and require; and in small enough bites to be practical.

[coppice]

New programming languages are often discussed here and elsewhere.  While C is the most widely used low-level embedded programming language, it is by no means perfect.  I've occasionally discussed what properties a better-than-C language for low-level embedded programming would have (although I've done much more systems programming using POSIX C myself), but I've also mentioned I do not intend to create my own "replacement C" at all: just experimenting with changing the standard C library with something better suited to today's needs suffices for me.

That is the exact same thing: not an effort to change everything or everybody at once, but make it possible for the future to be made better than it is now by those who are interested in it.  Help and advice, not demand and require; and in small enough bites to be practical.

An interesting thing about most new languages is they keep following the syntax of C, which is horrible. Before C a lot of languages had learned the benefits of things like "endif". "endwhile", etc. reducing the ambiguity about the scope of constructs. C didn't learn from that experience, and most people follow the mistake blindly. If you've had to fix broken projects, you'll know how many would have been less broken if the syntax forced greater clarity on the page.

[Nominal Animal]

New programming languages are often discussed here and elsewhere.  While C is the most widely used low-level embedded programming language, it is by no means perfect.  I've occasionally discussed what properties a better-than-C language for low-level embedded programming would have (although I've done much more systems programming using POSIX C myself), but I've also mentioned I do not intend to create my own "replacement C" at all: just experimenting with changing the standard C library with something better suited to today's needs suffices for me.

That is the exact same thing: not an effort to change everything or everybody at once, but make it possible for the future to be made better than it is now by those who are interested in it.  Help and advice, not demand and require; and in small enough bites to be practical.

An interesting thing about most new languages is they keep following the syntax of C, which is horrible. Before C a lot of languages had learned the benefits of things like "endif". "endwhile", etc. reducing the ambiguity about the scope of constructs. C didn't learn from that experience, and most people follow the mistake blindly. If you've had to fix broken projects, you'll know how many would have been less broken if the syntax forced greater clarity on the page.

Sure.  Point is, I don't have the resources or charisma to convince others to switch from C to something else, much less a new programming language of my own design regardless of how good it would be technically.  Very few others are experimenting with just replacing the standard library part with a different, incompatible library, one more suited to current needs.  So, instead of trying to maximize the breadth of change I might affect, I'm concentrating on the small part where I might make the most positive impact, no matter how small in breadth.  A small step at a time.

The situation with code commenting is similar.  Quantitatively, we can only affect a tiny fraction of developers, at best.  However, because that tiny fraction is exactly those who are interested in their skill and art enough to try and do better, they are disproportionately more likely to write code that matters (that may even impact our future selves), so trying to help them in useful-sized bites –– like advice on commenting when suitable –– is worthwhile.

It's all about picking your battles with true long-term in mind.

[nctnico]

If you document interface functions properly in the .h file (for example), then I can, in an IDE or code editor, see this documentation automagically as I write code. I have never seen an interface function being documented too well. The opposite problem, too little, is usual. Once properly documented in code comments, automated tools such as doxygen can be used to create the separate documentation for those who prefer to have it, so it's a win-win.

Not really as doxygen can't show how everything fits together in the big picture. A long time ago I had to take a project over from a co-worker who got a burnout (he stayed sick for over a year). Needless to say there was no handover and I had no knowledge about the project at all. It litterally got dumped onto my lap. The single thing that helped me most to get going with this project was a diagram that showed how all the pieces work together and how the data flowed between them. Doxygen can't generate that kind of information. Personally I loath programmers who present doxygen generated crap as the project documentation as it doesn't tell me anything. Loading the project into an modern IDE is much more worthwhile as the IDE typically can show things like call hierarchies and where functions and variables are declared.

[PlainName]

Not really as doxygen can't show how everything fits together in the big picture.

I think it can if you set it up appropriately.

[SiliconWizard]

Yes, of course Doxygen will show you call graphs. That's one of its basic features. Not that I find Doxygen particularly good either (or rather, the way it's often used), but if people are going to judge it, they may as well know what it can or cannot do.

[tggzzz]

Not really as doxygen can't show how everything fits together in the big picture.

I think it can if you set it up appropriately.

... and then you invoke a function indirectly through an entry in a table of function pointers. In other words, one of the classic ways of implementing FSMs. Or callbacks. Or vtables. Or a generic test algorithm passed to a generic library function (e.g. a sort()  ). Or...

[coppice]

Not really as doxygen can't show how everything fits together in the big picture.

I think it can if you set it up appropriately.

... and then you invoke a function indirectly through an entry in a table of function pointers. In other words, one of the classic ways of implementing FSMs. Or callbacks. Or vtables. Or a generic test algorithm passed to a generic library function (e.g. a sort()  ). Or...

Even something as generally thorough as Ghidra still struggles to highlight all those situations, even though it should be able to see things like tables with a bunch of addresses matching the entry points of a bunch of routines.

[SiliconWizard]

Note that some guidelines such as MISRA prevent the use of function pointers, IIRC. Not saying that I agree, but just saying that these are often thought of as a bit too slippery for safety-critical stuff.

[tggzzz]

Not really as doxygen can't show how everything fits together in the big picture.

I think it can if you set it up appropriately.

... and then you invoke a function indirectly through an entry in a table of function pointers. In other words, one of the classic ways of implementing FSMs. Or callbacks. Or vtables. Or a generic test algorithm passed to a generic library function (e.g. a sort()  ). Or...

Even something as generally thorough as Ghidra still struggles to highlight all those situations, even though it should be able to see things like tables with a bunch of addresses matching the entry points of a bunch of routines.

Void* and equivalents mean all bets are off; anything can happen. Especially if something cannot be determined statically as it is only instantiated at runtime.

Languages which are strongly and statically typed go a long way to helping IDEs analyse codebases. People that only know C (or C++) rarely understand just how powerful and accurate "cntl-space" auto-completion can be at determining the complete set of legal possibilities at any specific point in the code.

[coppice]

Note that some guidelines such as MISRA prevent the use of function pointers, IIRC. Not saying that I agree, but just saying that these are often thought of as a bit too slippery for safety-critical stuff.

MISRA is short for MISRAble. Its a horrible mix of good and bad practices. I wonder if the net effect contributes to safety or danger?

[nctnico]

Note that some guidelines such as MISRA prevent the use of function pointers, IIRC. Not saying that I agree, but just saying that these are often thought of as a bit too slippery for safety-critical stuff.

From the last time I checked, I recall that constant function pointers are allowed. In most cases the interrupt vector table is just that... But also think about state machines and command line interpreters where having function pointers greatly simplifies the code.

[PlainName]

where having function pointers greatly simplifies the code

It can simplify the code, but make it harder to figure out what's going on when what's going on isn't the expected going on.

[madires]

... and that can be explained in comments.

[PlainName]

 

[Siwastaja]

where having function pointers greatly simplifies the code

It can simplify the code, but make it harder to figure out what's going on when what's going on isn't the expected going on.

Yeah. I never understood the fear of using a function pointers instead of enumerations. With a variable and, say, switch-case, you can check for some illegal values, but not if the wrong value is from the legal set, and that's the most realistic bug. The fear that the function pointer corrupts to any random value pointing in random place in memory is of course valid, but if you have memory corruption to begin with, then it's equally likely to do similar hard-to-understand damage (e.g., corrupt a return address) even if you don't use function pointers. Replacing function pointers with something else adds memory safety by maybe 0.001%. Instead, proving absence of overindexing, zero termination mistakes etc., which can be done, would easily catch 99% of memory-corrupting bugs.

If you want fair level of protection (say, >90%?) against the rest, or uncontrollable memory corruption (e.g., due to radiation), that is going to require significant effort everywhere in the code base. Avoiding function pointers would be just the tip of the iceberg.

I recall that constant function pointers are allowed.

If true, this sounds to me that these guys had absolutely no idea what they are doing. Because function pointers being constant, but still in memory, doesn't help against memory corruption caused by bugs (say, serious overindexing) or external sources at all. The only thing it does, it helps against the bugs where a programmer accidentally writes an insane value to the function pointer; now the compiler would prevent this write. But that is an extremely weird and remote scenario to begin with; it would require typecasting and is obvious even in mediocre code review. And if we go this far, what prevents the programmer to cast away the constness anyway?

The bug which actually happens, is the programmer enters a valid, but wrong state; and for this bug type, const function pointers or enumerated state values are exactly the same.

[nctnico]

By constant I (and the designers of MISRA) mean a variable sits in read-only memory. Not RAM. Typically everything you declare const in C on a microcontroller ends up in flash which is read-only.

From my experience the biggest cause of memory corruption are buffer overruns. So allowing function pointers in read-only memory only, is a sensible compromise IMHO. At least you won't get the situation that code is executed from a random address.

[jfiresto]

... If true, this sounds to me that these guys had absolutely no idea what they are doing. Because function pointers being constant, but still in memory, [only] ... helps against the bugs where a programmer accidentally writes an insane value to the function pointer; now the compiler would prevent this write....

Allowing function pointers in a variable makes it hard to catch certain bugs through static analysis. I recently had to do something like that, for a dead simple, three-line Python class, to stop a static code checker (pylint) from misinterpreting the Abstract Syntax Tree and complaining about calling arguments for a method the caller was not calling.