Benchmark Programs Using time, perf stat, and hyperfine

Topic Overview

Measure Before You Optimize

Brendan Gregg's production performance work at Netflix and Sun starts with simple wall-clock timing, then perf stat, then deeper tools. Guessing where time goes wastes weeks. A three-line time run, a perf stat pass for cycles and cache misses, and hyperfine for repeated comparisons form the baseline toolkit every systems engineer keeps on hand.

What Each Tool Answers

time reports elapsed, user, and system seconds for one run. perf stat samples hardware counters without modifying your binary. hyperfine warms up, runs many iterations, and prints mean plus standard deviation so you can see noise.

bashCopy
Loading…

• Wall clock alone cannot separate CPU work from I/O wait
• Counter events explain why a loop is slow, not just that it is
• Always run release builds with the same CPU governor when comparing
• Save baseline numbers before changing code

Working Through the Exercise

Keep the relevant documentation open while you implement. When your output disagrees with the reference, trace one failing case by hand before changing random lines.

• Trace first: walk one failing input step by step
• Read the manual: skim the official doc for the mechanism named in this task
• Compare baselines: save measurements from the prior step so speedups are honest

Why for this exercise

You will benchmark a small C program with time, perf stat, and hyperfine, then print a short comparison table. This exercise asks you to treat measurement as the first optimization step, not an afterthought.

Your Task

time, perf stat and hyperfine answer different questions: where the wall-clock time went (user, system, or waiting), how efficiently the CPU ran (clock speed, instructions per cycle, branch misses), and how stable repeated measurements are. Real runs are never repeatable, so this task starts from recorded measurements and does the analysis these tools print, in exact integer arithmetic.

Input

cCopy
Loading…

Rules

• Durations print as N us below 1 ms, N.N ms below 1 s, and N.NN s otherwise, truncated (not rounded).
• time: CPU utilisation = (USER + SYS) / REAL, 1 decimal, rounded half up. Add (more than one core busy) if USER + SYS > REAL, or (mostly waiting: I/O or sleep) if USER + SYS < REAL / 2.
• perf: GHz = CYCLES / TASK_CLOCK_US / 1000, 2 decimals, rounded half up. IPC = INSTRUCTIONS / CYCLES, 2 decimals. Branch miss rate as a percentage, 1 decimal.
• bench: mean (rounded half up to the microsecond), sample standard deviation (with n − 1; the integer square root, rounded down, of the integer variance sum((t − mean)^2) / (n − 1); 0 for one run), min and max. Warn when sd × 10 > mean.
• summary: the benchmark with the smallest mean (the first one on ties) "ran" N times faster than each other one, where N = other mean / best mean, 2 decimals, rounded half up.
• Errors: time: expected REAL USER SYS (us), perf: expected TASK_CLOCK_US CYCLES INSTRUCTIONS BRANCHES BRANCH_MISSES, bench NAME: no positive run times, and summary: need at least two benchmarks.

Output

cCopy
Loading…

Example

Input:

cCopy
Loading…

Output:

cCopy
Loading…

Requirements

1. Use integer arithmetic throughout, including an integer square root, so the output is identical everywhere.
2. Report spread (sd, min/max) next to every mean. A single number hides noise.

Hidden tests cover multi-threaded and I/O-bound time lines, low-IPC and high-miss perf counters, a single-run benchmark, sub-millisecond runs, a summary across three benchmarks, and malformed lines.