Debug a Multithreaded Race Condition

Topic Overview

Debugging Multithreaded Programs

Multithreaded programs are notoriously difficult to debug because bugs often depend on exact timing and thread interleaving. A race condition that crashes in production may disappear entirely when you single-step through the code.

Essential GDB Thread Commands

• info threads: List all threads with IDs and states
• thread 2: Switch GDB's context to thread #2
• thread apply all bt: Backtrace all threads simultaneously
• set scheduler-locking on: Prevent other threads from running while stepping
• break foo thread 2: Breakpoint that fires only in thread 2

For example, if two threads both execute shared_counter++ without a mutex, the final count will be less than expected because the read-modify-write is not atomic.

For This Exercise

You will create pthreads that race on a shared counter without synchronization and document the GDB commands to detect the race. This mirrors the pattern that ThreadSanitizer flags and that engineers triage with thread apply all bt in production deadlock investigations.

Reproducing Race Conditions

Set set scheduler-locking step to hold other threads while you inspect shared state. Use watch shared_counter to break the instant any thread writes to the variable. Compare the backtraces from thread apply all bt when the counter has an unexpected value to see which threads were in the critical section simultaneously. ThreadSanitizer flags the race, but GDB confirms the exact interleaving.

Write a C program that replays a deterministic thread schedule (GDB scheduler-locking semantics) and reports the resulting program state.

Requirements:

• Read n, then n events: "lock T", "unlock T", or "inc T" executed in schedule order (T is a thread id)
• lock: acquires a free mutex; a thread re-locking a mutex it already holds succeeds recursively; otherwise the thread blocks, is recorded in FIFO arrival order, and the attempt is counted
• unlock: releases one level of a mutex the thread holds; on full release with waiters present, ownership is handed to the lowest-numbered waiting thread; unlocking a mutex the thread does not hold is counted as a bad unlock
• inc: increments the shared counter (unsynchronized, as in the racy original)
• Print exactly six lines: "counter: N", "owner: T" (0 when free), "blocked_attempts: K", "hung: H" (waiters left at the end when a thread still owns the mutex), "bad_unlocks: B", "threads_seen: M" (highest thread id seen)