RunTasks keeps its RunState - counter, mutex, condition variable - on its own
stack frame and hands a pointer to the pool. The last worker to finish
decremented the counter OUTSIDE the mutex and only then took it to notify, while
the waiter's predicate was the counter itself. So the waiter could see zero the
instant the decrement landed, find its predicate already true, never block, and
return from RunTasks - popping the frame. The worker then locked a mutex and
signalled a condition variable that no longer existed, writing pthread state into
a frame the submitting thread had already reused.
pthread_mutex_unlock writes owner and nusers as eight contiguous zero bytes. Land
those on a live pointer and the next read of a member at offset 8 faults: the
observed crash was fmt's buffer<char>::append with this == nullptr, in the log
call immediately after a parallel pass, which is why it always appeared after the
resolution-filter line - that line was simply the next thing to use the frame.
Give the waiter a flag set under the same lock as the notify. Completion cannot
then be observed until the notifier has released the mutex, i.e. after its last
touch of the state. The counter keeps its lock-free fast path and decides only
who notifies, so there is still exactly one lock per pass.
Found independently by two investigations: a widened-window reproducer (2 crashes
in 38 unfixed, 0 in 60 fixed; glibc's own "__owner == 0" assertion caught in the
pool worker) and an isolated one that clobbered a freshly filled stack frame 732
times in 60000 and never after the fix. Growing RunState by eight bytes, changing
nothing else, took the rate from 0/90 to 3 hard failures in 30.
The dataset that failed about one run in twelve: 0 of 40. Space group and
merged output unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
leonarski_fandClaude Opus 5
bd2179b08f