Understand OOM Killer: Read Linux oom_kill.c Source

Topic Overview

When Memory Runs Out

The OOM killer chooses a process to terminate when the kernel cannot reclaim enough pages. It is a last resort after reclaim, swap, and compaction fail. Server operators know it from sudden Killed messages in logs.

How Linux Picks Victims

oom_kill.c scores tasks:

• oom_score_adj: user-tunable bias (-1000 to +1000)
• RSS and runtime: large, short-lived jobs often lose
• child tree: killing a parent may free a subtree

For example, a chrome renderer with high RSS and low oom_score_adj protection dies before sshd with negative adjustment.

Why This Matters in Production

Production teams running Postgres or Redis set /proc/PID/oom_score_adj to -1000 specifically to survive the badness-score algorithm in Linux's mm/oom_kill.c that you're reading here. Misjudging how CAP_SYS_ADMIN or oom_score_adj shifts that score is exactly why the wrong process , sometimes the database instead of the runaway script , gets SIGKILLed during real production incidents.

Debugging Checklist

Before you call the implementation done, walk failure modes on purpose. Test empty structures, single-element edge cases, maximum concurrency, and errno paths that must not crash the program. OS code usually fails in production when happy-path tests pass but invariants break under contention or memory pressure.

Implementation Notes

Keep structures small and name fields after kernel counterparts when possible. That lets you read man pages and kernel source side by side while you work. Print observable events during development; remove noisy logs once tests pass reliably.

For This Exercise

You will read mm/oom_kill.c (or a trimmed excerpt) and summarize the selection heuristic. This exercise requires correlating oom_score in /proc with which process your simulated pressure would target.

Your Task

Implement the Linux OOM killer's victim selection (mm/oom_kill.c). When memory is exhausted, the kernel scores every process with oom_badness(), which is roughly "how many pages would we get back". It adjusts the score by the process's oom_score_adj and kills the highest scorer. Reproduce the scoring, the /proc/PID/oom_score value that userspace sees, and the kernel's kill message.

Input

cCopy
Loading…

oom_badness(p)

1. Unkillable: kernel threads (kthread), PID 1, and any process with adj = -1000: skipped entirely (score prints them as unkillable).
2. points = rss + swap + pgtables / 4096 (integer division).
3. points += adj × total / 1000 (adj is from −1000 to 1000; use integer arithmetic truncating toward zero).
4. If points ≤ 0, it becomes 1. A killable process always has a positive score.

/proc/PID/oom_score is points × 1000 / total (integer division), capped at 2000.

Selection: the killable process with the highest points. On ties, the lowest PID wins. The victim is removed from the process list.

Output

score prints one line per process in PID order:

cCopy
Loading…

(the pgtables term is shown already divided by 4096.)

oom prints:

cCopy
Loading…

with sizes in kB (pages × 4), or oom: no killable process (a kernel panic in real life).

Example

Input:

cCopy
Loading…

Output:

cCopy
Loading…

Requirements

1. One oom_badness() returning the points or "unkillable"; score and oom both use it.
2. Use 64-bit integers: adj × total overflows 32 bits on large machines.

Hidden tests cover successive oom calls (each removes its victim), a tie broken by PID, a negative adj that clamps to 1, adj = 1000 making a small process the victim, a machine with only unkillable processes, and a total large enough to need 64-bit arithmetic.