Trace a Binary's Execution with strace, ltrace, and perf

Topic Overview

Tracing Execution with strace, ltrace, and perf

Observing what a binary does at runtime is often faster than reading thousands of lines of disassembly. Linux provides three tools for this: strace, ltrace, and perf.

Tool Overview

strace records system calls (kernel interface):

cCopy
Loading…

Key observations: open reveals file paths, connect reveals network activity, execve reveals subprocess creation.

ltrace records library calls (above syscalls):

cCopy
Loading…

perf uses hardware counters to find hot paths and encryption routines by CPU time.

For This Exercise

You will document how strace, ltrace, and perf reveal program behavior at runtime. For example, strace output showing open("/etc/passwd", O_RDONLY) immediately tells you the binary reads the password file without reading a single instruction.

Combining Traces

Run strace -f to follow child processes created by fork or execve. Filter noisy syscalls with strace -e trace=file to focus on file access. Pair strace with ltrace -S to see both library and system calls in one trace. For performance-heavy analysis, perf record followed by perf report shows which functions consumed the most CPU time during encryption or compression routines. Malware analysts routinely combine all three tools when triaging unknown samples in sandboxed environments.

Write a C program that acts as the call/ret consistency monitor a tracer (or a shadow-stack/ROP detector) runs over an execution trace.

Input:

• Line 1: integer N, the number of events
• Next N lines: one event each, either call <target hex> (e.g. "call 401000") ret <return hex> (e.g. "ret 401005")

Monitor rules (the shadow stack):

• call pushes the target address
• ret pops the top and compares: a match is silent; a mismatch is reported but the pop still happens, and analysis continues
• ret on an empty shadow stack is always a mismatch

For every mismatch, print exactly: event <i>: mismatch: expected top 0x<hex> but ret 0x<hex> event <i>: mismatch: ret 0x<hex> with empty shadow stack (event i is 1-based, addresses lowercase hex as given, formatted with %llx)

After all events, print exactly one summary line: summary: max_depth=<d> mismatches=<m> (max_depth is the largest shadow-stack size reached)

Success Criteria:

• Code compiles with gcc -Wall -Wextra
• Nested calls, mismatched returns, and underflow are all handled per the rules