What happens if you really run out of physical memory (including swap) after overcommitting? Does your process get a signal, or will OOM killer just run without notifying the process that triggered the condition?
Ig you try to allocate more than the system is willing to overcommit (e.g. a huge block all at once), malloc will fail. But if phyaical memory gets exhausted by accessing previously allocated pages, the OOM killer will evebtuslly come around and kill processes without signalling. Signal handlers could still make thenprocess (unknowingly) request more memory, so there is 0 guarantee that a handler could even run successfully.
Oh yeah, I didn't think of that. I wonder if you could write a signal handler carefully to not allocate any memory, stack or otherwise, or is some return address or an internal structure being allocated transparently...
Just trying to allocate stack space for the signal handler may cause the stack to spill into a new page. That is absolutely out of anybody's control. And if that new page cannot be provided, it's game over.
Makes sense. I was thinking that maybe signal handlers could use the regular stack of the thread, but that would of course make everything fall down if the "real" code would write to the stack before updating the stack pointer.
