The problem here is the "buffer overflow"
Yup; and deliberate overwrite of the return address is a known
exploit scheme.
Separating the return addresses into a separate stack that is not easily accidentally overwritten is definitely something I'd like to see (but with the constraints I mentioned above).
Using a stack that grows upwards (increasing addresses) would help a lot here, because then the return address is at a lower address than the temporary variables and arrays; buffer overrun then does not overwrite the return address, but the unused area of the stack (and whatever follows the stack).
A big reason for buffer overruns being so common in C code is because a lot of the C standard library functions have idiotic interfaces. Hell, standard C still doesn't have
getline(),
strdup(),
strndup(), etc.
An embedded RTOS, or any code running in a freestanding environment (no standard C library available, only features described by <float.h>, <iso646.h>, <limits.h>, <stdalign.h>, <stdarg.h>, <stdbool.h>, <stddef.h>, <stdint.h>, and <stdnoreturn.h>; and compiler-provided C extensions), like say the Arduino environment for example, is free to implement its own.
For specific things like serial input processing, a coroutine consuming input byte by byte, and parsing and processing it as it comes in, would work fine on AVRs. But, the C model really doesn't provide a good interface for those. So, buffers are usually used instead.
(If you want, try writing a C function to parse input, with the function called once whenever a new byte is available, with that byte as a function parameter. So, basically like a finite state machine, where each call provides the new input value, so you can do a state transition, and optionally produce output – say, in the form of calling a function. It can be done, sure; but it's not something you'd expect a C learner to be able to do correctly.)
If you want mutable strings or data buffers in C, without risking buffer overruns, I like to use
struct area {
size_t size;
size_t used;
unsigned char data[];
};
where
data[] contains the actual data in a C99 flexible array member;
size is the size of that array; and
used is the number of bytes of data in the initial part of the array. This is very similar to Pascal strings, where the length of the string is just before the data; except I have size, length, and then data.
For I/O buffers, I like to use
struct iobuffer1 {
size_t size;
size_t head;
size_t tail;
unsigned char data[];
};
struct iobuffer2 {
size_t size;
unsigned char *head;
unsigned char *tail;
unsigned char *data;
};
where
head is the index or pointer to the first byte in the buffer,
tail is the index or pointer just past the last byte in the buffer (with
head == tail indicating an empty buffer). With these, the only buffer issues are off-by-one errors, especially when parsing e.g. escape codes and the escape mark is the last byte buffered. Those I avoid by comparing the current parsing position to the buffer tail, with as many "extra" bytes allocated to
data[] as the longest escape sequence is, and being able to cancel parsing and retry when there is more data available.
It can happens in kernel space as well as in user-space. I have here a micro-controller (without any MMU/TLB) running uc/OS2 as kernel, and a tiny Web-server as main task. Well, it's not robust code but rather weak, so technically it can for sure experiment a malicious-exploit when a function that handles an HTTP request doesn't check parameters and this is a serious vulnerability when the request is made just to deliberately cause a buffer overflow in order to cause the RA overwritten by something that will be used as Return Address when the function returns.
Have you read any Apache or APR sources? Instead of
heap, it uses
allocation pools. There is one "global" pool, but each request gets their own pool, and any memory that needs to be allocated, is allocated from that pool. (It's not a pool in the sense that it has limits; it's more in the sense that all allocations within that pool can be freed with a single call.) This does not help with buffer overruns, though? Well, it does, because no array-like data ever lives on stack; they're always allocated from the pool instead.
Similarly, we don't really need
a stack for function-local data. It is just one of the possible ways of managing the memory needed by functions for temporary or more longer-duration data. It's just that a stack is an obvious choice when you consider the complexity of implementing an allocator.
Let's say you had a very fast memory allocator, and the stack is only used for storing scalar data, no arrays.
If all arrays had to be dynamically allocated, then using something like
struct iobuffer2 above would require four pointers (three pointers and one unsigned size), and "standard library" functions could check for array boundaries.
Thus far, such dynamic allocators tend to be considered too "slow" for general purpose stuff.
(Yet, it is exactly the direction e.g. the Linux kernel is going.)
See how it comes round to memory management schemes? Well, at least for me. This is why I'm looking into segmented memory models, i.e. where one has some sort of special base pointer (maybe just aligned in memory), and offsets and sizes with respect to that base pointer; and the
hardware can detect buffer overruns (by simply making sure that the effective offset with respect to a given base pointer is within its size). Return address stack would be only one particular use case, and if it was only accessible to CALL/RET instructions, ROP exploits and buffer overruns affecting function return address would be impossible. It's just that passing three or four pointer- or pointer-size variables per address/variable is a lot of overhead; there must be a pattern that reduces the overhead, especially since typical code only juggles between a small number of separate "objects" (or variable scopes). If the segment is hardcoded to the assembly instruction, like on x86 and x86-64, you'd need a separate function for each combination of reference segments. If, however, the segment was coded to the "base pointer", then one function would suffice.