Nathan Bronson describes how OpenAI engineers traced recurring Rockset crashes to two independent causes: silent CPU corruption on a single Azure physical host, and a long‑standing race condition in GNU libunwind. The first was a hardware error affecting VMs scheduled on that host; the second is a narrow, instruction‑level race during C++ exception unwinding that can produce NULL or otherwise corrupted return addresses.
What happened and why it matters
Rockset is a cloud‑native search and real‑time analytics system used by OpenAI for many internal workflows (Rockset was acquired by OpenAI in 2024). Its execution layer is written in C++, which gives low‑level CPU access for performance but permits invalid memory accesses and segfaults when bugs occur.
A few months prior to the investigation the team observed multiple crashes. Core dumps suggested that a normal C++ function would complete and then transfer control to an invalid address (e.g., NULL), or that the stack pointer (%rsp) was misaligned by 8 bytes before a return — behaviors that were not consistent with ordinary application bugs and resisted initial hypotheses.
From a single‑case to a population approach
The team initially performed deep, case‑by‑case debugging: inspect a handful of cores, form hypotheses, and rule them out. That approach failed because stack‑corruption makes logs and traces themselves unreliable, and manual core analysis does not scale.
They switched to an epidemiological approach: automatically analyze the entire population of cores. They built a pipeline (initially scaffolded with ChatGPT) to download prefixes of each core file, extract registers, filter false positives using logs, and classify crashes as return‑to‑null, misaligned‑stack, or other. Running this analysis over a year of production Rockset core dumps immediately revealed distinct clusters.
The hardware case: one bad physical host
The misaligned‑stack crashes all came from a single region, had a clear start date, and never originated from long‑running nodes. Tracing Kubernetes nodes and timestamps pointed to a single physical host; once that host was denylisted and removed from service, the misaligned‑stack crashes disappeared. The team was unable to reproduce the corruption in controlled stress tests on that machine, but operational mitigations were implemented: the fatal signal handler was enhanced to include register state so recurrences can be detected from logs, VM reuse was favored over recycling to ease bad‑node detection, and runbooks were updated.
The libunwind race and exception unwinding
After removing the bad‑host cluster, the remaining return‑to‑null crashes were analyzed and found to occur during exception unwinding. When C++ throws, runtime helpers examine the stack, consult metadata, and transfer control to cleanup handlers or catch blocks; that transfer involves restoring callee‑save registers and stack registers (%rbp and %rsp), and behaves like a setcontext/longjmp rather than a normal call/return.
OpenAI’s binary dynamically linked against GNU libunwind rather than libgcc. GNU libunwind builds a ucontext_t on the stack, fills in desired register state, then calls an internal assembly routine, _Ux86_64_setcontext, to apply the state. In the library version they used, _Ux86_64_setcontext first updated %rsp to the new stack base, and then executed a sequence of mov instructions that read registers out of the ucontext_t (pointed to by %rdi). Because the synthesized ucontext_t lives on the stack frame that _Ux86_64_setcontext itself is unwinding, updating %rsp can make the struct no longer part of the active stack or red zone.
If a signal (Rockset uses SIGUSR2 frequently for its coarse_thread_cputime_clock) is delivered in the narrow interval after %rsp is changed but before the next mov loads UC_MCONTEXT_GREGS_RIP(%rdi), the kernel may build the signal frame at %rsp‑128 and overwrite the memory pointed to by %rdi. If that occurs before the code reads the intended return address, the restored instruction pointer can be corrupted — in observed cases it became NULL.
What appeared in cores as “a function returned to NULL” was actually “the unwinder synthesized a target instruction pointer on the stack, but that target was clobbered by a signal before the control transfer completed.”
Why a very narrow race can still be frequent
The vulnerable window is extremely small (effectively one instruction width), but the overall failure probability scales with how often unwinding occurs and how often signals are delivered. Rockset’s design increased all three contributors: high exception rates for ingest backpressure, frequent SIGUSR2 deliveries from coarse_thread_cputime_clock, and increased signal‑handler stack usage after adding timer_getoverrun.
Using back‑of‑envelope estimates, if the vulnerable window is on the order of 10^‑10 seconds and SIGUSR2 arrives every 10^‑2 seconds of CPU time, each unwinding has roughly a 10^‑8 chance of being hit. If an overloaded host throws on the order of 10^4 exceptions per second, that corresponds to a mean time between failures on the order of 10^4 seconds (a few hours) per host; at fleet scale this explains the observed daily crash counts.
The libunwind bug is old — present in the initial x86_64 implementation that supported C++ exception unwinding — and its visible impact depends on the product of exception rate, signal rate, and handler stack usage.
Mitigations and upstream fixes
Immediate mitigations included switching from GNU libunwind to libgcc’s unwinder (also beneficial for lock contention and scalability). The engineers upstreamed a self‑contained reproducer and a fix to GNU libunwind and verified other unwinder implementations did not exhibit the issue.
Main lessons
Beyond the low‑level lessons about dynamic linking, DWARF unwind metadata, Linux signal delivery, the System V ABI, and C++ exception machinery, the central operational lesson was simple: build a high‑quality, population‑level data set. Prior to that, mixed evidence from two distinct failure modes prevented coherent reasoning. Once the core dumps were automatically analyzed and clustered, the problem split cleanly into a bad host and an old libunwind race, and fixes and mitigations became straightforward.
For infrastructure systems like Rockset, the investigation reinforced commitment to deep instrumentation, automated investigations, and operational tooling that converts apparently inexplicable failures into diagnosable and solvable issues.
— Nathan Bronson, Member of Technical Staff
