I encountered an issue where hibernation on a freshly booted system works, but after a while it failed and the system just came up again. I checked dmegs and found the following messages.

[Di Jul 11 06:07:02 2023] PM: hibernation: hibernation entry
[Di Jul 11 06:07:02 2023] Filesystems sync: 0.011 seconds
[Di Jul 11 06:07:02 2023] Freezing user space processes
[Di Jul 11 06:07:02 2023] Freezing user space processes completed (elapsed 0.003 seconds)
[Di Jul 11 06:07:02 2023] OOM killer disabled.
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x00000000-0x00000fff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x0009e000-0x000fffff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x40000000-0x403fffff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x4652b000-0x4652cfff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x5b72c000-0x5b743fff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x5e4e2000-0x5fffefff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Marking nosave pages: [mem 0x60000000-0xffffffff]
[Di Jul 11 06:07:02 2023] PM: hibernation: Basic memory bitmaps created
[Di Jul 11 06:07:02 2023] PM: hibernation: Preallocating image memory
[Di Jul 11 06:07:03 2023] PM: hibernation: Image allocation is 97054 pages short
[Di Jul 11 06:07:03 2023] PM: hibernation: Basic memory bitmaps freed
[Di Jul 11 06:07:03 2023] OOM killer enabled.
[Di Jul 11 06:07:03 2023] Restarting tasks ... done.
[Di Jul 11 06:07:03 2023] PM: hibernation: hibernation exit

A similar situation got described in the Gentoo Forum where a comment references the Linux Documentation which states:

The freezing of devices is carried out after enough memory has been freed (at the time of this writing the image creation requires at least 50% of system RAM to be free)

So I checked with my system and yes, more than 50% of RAM got allocated. I wondered why this doesn't work anymore, because in a previous Linux installation hibernation just worked fine, even with full RAM.

Because all of my systems have enough RAM for their intended use, I tuned swap allocation to a minimum and have set:

sysctl vm.swappiness=0

Which is literally a „don't dare using swap“ option. It is okay, because hibernate can still use the swap as a storage, but it does not get used to out source stuff that usually belongs into RAM. When there is enough free RAM for hibernation, then everything is fine. When there is not, hibernate usually tries to free some memory in RAM and temporarily moves it to swap to it can pack it's resume image. But unfortunately this is not allowed due to swappiness being set to 0.

As a result, setting swappiness to 1 fixes it, because we still deny usage of swap, but allow it for cases where absolutely no other option exist.

sysctl vm.swappiness=1