Several quanstions about Linux and memory management

[golden_labels]

See mmap(2) for details, fragments mentioning MAP_ANONYMOUS. tl;dr it’s a page mapping that has no explicit backing storage associated. It’s what is created using VirtualAllocEx and friends in Windows. Windows API has, mostly for historical reasons, API separation between anonymous pages and mapped pages.

malloc is a beast living on a different level and doesn’t provide memory. It’s an abstraction of the C language, which primarily creates objects inside the program. It is provided by the C user-space library and the operating system is not even aware of its existence. malloc interfaces with the operating system to obtain memory pages only if needed, but that’s its only association with memory. While it would be weird to see it not use anonymus pages under Linux, there is nothing that requires it to work this way.

[nctnico]

nctnico: please, update your knowledge. It’s no longer 1990s. Under normal conditions⁽¹⁾ swap is not “free memory” for people, who can’t afford actual RAM. It can’t be used that way and is not used like that in Linux for a very long time. Windows is moving away from that archaic model too since at least version 6.0 of the kernel. Simplifying things a bit: nowadays swap is used as an ad-hoc backing storage for anonymous pages, so the kernel can remove them from RAM and use that memory for something useful, like storing file cache. This is a performance increasing use, unlike “free RAM” which is greatly decreasing performance. A secondary use is suspend-to-disk. That is why swap is still defended by some against being completely removed from modern systems.

If you want some introduction to the situation from a more technical point of view, see a summary on Linux swap use from Chris Down, a kernel dev.

You are repeating exactly what I wrote but in different words. In the end the effect counts and from what I've seen from (modern!) memory constrained Linux systems is that having swap helps to avoid the OOM killer to kick in due to some processes claiming a wild amount of memory. Ofcourse you can get into a very technical discussion but to me the end result counts. Or put differently: how would you prevent the OOM killer to kick in on a system with a constrained (=enough to run the applications but not enough to support peak memory usage) amount of memory? AFAIK the answer is to have swap.

[golden_labels]

No, I do not. If I had any doubts earlier, you are pretty clear that it’s about providing more memory to programs. Which it is not. Please re-read what I have wrote and — if you still think that I agree with you statement — read and understand the linked article.

If an OOM condition occurs, swap is often only delaying OOM killer. In other words: it prevents OOM killer from fixing that swiftly and, in the process, also degrades performance. If a runaway leak is present, the presence of swap is making the situation so bad, many argue that swap should be abolished completely (I am not among them).

The way to prevent OOM killer from kicking in is to have RAM adequate for the workload. Of course that doesn’t help with buggy software: in that case you want the OOM killer. If you want to protect some services from being accidentally killed, you may adjust their OOM score or put potentially offending programs in a cgroup.

[nctnico]

No, I do not. If I had any doubts earlier, you are pretty clear that it’s about providing more memory to programs. Which it is not. Please re-read what I have wrote and — if you still think that I agree with you statement — read and understand the linked article.

If an OOM condition occurs, swap is often only delaying OOM killer. In other words: it prevents OOM killer from fixing that swiftly and, in the process, also degrades performance. If a runaway leak is present, the presence of swap is making the situation so bad, many argue that swap should be abolished completely (I am not among them).

The way to prevent OOM killer from kicking in is to have RAM adequate for the workload. Of course that doesn’t help with buggy software: in that case you want the OOM killer. If you want to protect some services from being accidentally killed, you may adjust their OOM score or put potentially offending programs in a cgroup.

Read carefully what I wrote: peak load and overcommitting. There is no actual shortage of memory for the task at hand and no leaky / buggy software.

[golden_labels]

Yes, this is exactly what I understood from your previous posts. And this is what swap is not for. It is not to provide ability to allocate more memory by the processes. Peak or otherwise.