Five files under t/ call BAIL_OUT at eleven places: a missing command, a
manifest section that is not there, an extraction that stopped matching,
a run_bounded that never returned. prove stops every remaining file on a
bail-out, not only the file that called it, so one of these hides the
results of every test that would have run after it. die is just as loud
and costs only its own file.
The header of genesis_native_deb.t also retold how an EL image reached an
Ubuntu node. The fallback and its effect are one sentence.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The probe starts a descendant, kills the leader with SIGKILL, and asserts the
descendant is gone when the call returns. It fails against the previous
run_bounded, where the descendant survived.
The assertions cover a normal exit, the highest exit code, and death by TERM,
KILL and a core-dumping signal. run_bounded is driven through the same helper,
so the decoding and its caller cannot disagree.
The probe sends itself a signal while the mask is held and asserts it arrives
only once the mask is restored, which is what makes the fork and the pid
registration a single uninterruptible step.
The race itself is not reproducible without a hook in the production path, so
the assertions pin the invariants the fix establishes: the build starts with the
handled signals unblocked, and the caller keeps them unblocked afterwards.
Two real processes stand in for a worker and the build it runs. The probe fails
when the handler forwards to nobody, which is what the orchestrators did.
The command records its own pid and sleeps well inside the budget, so the
bound cannot be what ends it; the wrapper is then terminated and the pid
probed. Without the forwarding the build survives.
A riscv64 goconserver `go build` sat 26 minutes with zero CPU ticks across a
20-second sample, both Go pids in futex_wait and no socket open. Nothing bounds
a build step, so the cell did not fail -- it hung, and a hung run reads as
"still running" rather than as a defect.
t/build_timeout.t drives the bounded path with a command that hangs and asserts
that the call returns, reports a timeout, and prints the process tree, each
pid's wchan, the open socket count and a CPU-tick sample. A second case drives a
spinning command and asserts the report calls it slow, not deadlocked, so the
sample means something.
Each call under test runs in a forked child whose stdio is detached to a file,
and the parent bounds that child. An unbounded run must fail this test, not
block prove.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão