mirror of
https://github.com/xcat2/xcat-dep.git
synced 2026-09-12 12:36:23 +00:00
Daniel Hilst
ca051e46c4
Addresses the three blocking issues from the PR #63 review. 1. THE TWO ARCH JOBS COULD BOTH REWRITE THE PUBLISHED REPOSITORY. The amd64 and ppc64el runs execute concurrently on their two hosts against the same --apt-dir. The run lock does not protect the repository -- it is per-arch by design, precisely so the two arches CAN build in parallel -- so both were free to wipe and repopulate the same pool/, dists/, Release, InRelease and tarball at the same time and interleave into a corrupt but green repository. Split by role, so architecture jobs produce staging artifacts and one locked finalization step publishes atomically: * A run that builds no longer publishes. It fills staging/<codename>/<arch>/ and stops, saying so and printing the finalize command. Publishing happens only with --publish, or implicitly on a run that builds nothing (--skip-build), which IS the finalization step; --skip-createrepo still forces it off. * publish_repo() takes ONE GLOBAL publish lock (.sbuild-all.publish.lock), not the per-arch build lock, so even a cron run racing a manual one on the same host serializes instead of interleaving. * It publishes atomically. assemble_into() builds and signs the complete tree in a side directory (<apt-dir>.publish-<run-id>.<pid>), the gate runs against THAT tree, and swap_into_place() then renames it onto --apt-dir. A reader -- the deploy rsync, an apt client on a served tree -- sees either the previous complete repo or the new complete repo, never a half-wiped pool or an index that disagrees with its Release. A failure anywhere before the swap leaves the published tree untouched and removes the side tree. The side tree is seeded from the current published one, so codenames outside --dists survive. * The tarball moved inside publish, under the same lock: previously every per-arch build tarred the shared apt tree while the other arch rewrote it. 2. build_deb_in_chroot() WAS NOT FAIL-HARD, AND THE ENVIRONMENT WAS NOT CLEAN. The common build-tooling install ended in `|| true`; a failed Build-Depends installation only warned; and the Build-Depends came from a sed pipeline over debian/control that stripped version constraints `(>= 12)`, could not express alternatives `a | b`, and mangled arch qualifiers. Worse, the per-codename chroots are long-lived and shared by all seven packages, and nothing asserted that a schroot session is throwaway -- so on a chroot without a union mount, package N's build-dependencies stayed installed for package N+1 and a package whose debian/control forgets a Build-Depends would build green on a sibling's leftovers. Making dependency setup fatal is only half the fix; it means nothing if a stale environment can satisfy an undeclared dependency. Both halves are addressed: * The in-chroot program is now generated by the pure, unit-tested BuildUtils::chroot_build_script() and runs under `set -euo pipefail`. apt-get update and the common tooling go through an apt_retry helper that retries a transient mirror hiccup and then FAILS the build. Build-Depends are installed with mk-build-deps (devscripts + equivs), which hands debian/control's relationships to apt verbatim, and a failure is fatal. The mk-build-deps dummy package is excluded from deb collection alongside dbgsym. * ensure_disposable_chroot() repairs a chroot.d entry that lacks union-type=overlay, and build_deb_in_chroot() re-reads `schroot --config` and hard-fails on a chroot that is still not disposable, naming the fix. BuildUtils::chroot_is_disposable() is the pure predicate. 3. THE REPOSITORY GATE STILL HAD FALSE PASSES. * Standalone --verify-repo skipped signature verification whenever --gpg-home was not passed, so the common `--verify-repo <dir>` invocation silently checked completeness only. Signatures are now verified BY DEFAULT there; --no-verify-signature is the explicit opt-out. (The automatic pre-swap gate still requires a signature iff --gpg-sign was used, so an intentionally unsigned tree does not false-fail -- and publishing unsigned now warns.) * The expected architecture set was inferred from what happened to be present, so an entirely missing secondary architecture read as "this run did not build it" and passed. It is now always a CLAIM: --expect-arch (repeatable, accepts a space/comma list) if given, else the staged arch set when publishing, else each codename's own Release "Architectures:" line when verifying standalone (BuildUtils::parse_release_architectures). The pure BuildUtils::verify_repo_arches then reports an expected arch with no NATIVE package as MISSING-ARCH and natives for an unexpected arch as UNEXPECTED-ARCH (a stale architecture). Native detection matters because the Architecture:all packages ride into every arch's index, so a non-empty binary-<arch>/Packages is not evidence that arch was built. * Release now advertises exactly the expected arch set and only those binary-<arch> indices are written, so an amd64-only build no longer advertises a ppc64el it cannot serve. * An expected cell with no manifest section used to be skipped with a note -- a third free pass. It is now a hard NO-MANIFEST error. TESTS t/sbuild-all.t gains verify_repo_arches (both directions), the parse_release_architectures parser, chroot_is_disposable, regression guards on the generated in-chroot script (no `|| true`, no warn-only build-deps, no sed extraction, mk-build-deps present, every apt-get behind the fatal helper), and the build_deb_in_chroot disposability guard driven through a stub schroot. t/verify-repo.t is new: it drives the real `sbuild-all.pl --verify-repo` against fixture apt trees and asserts each former false pass now fails -- missing secondary architecture (with and without --expect-arch), arch:all-only index, unsigned repo with no --gpg-home, missing manifest section -- and that an honest single-arch repo still passes. Validated on the build hosts: a full focal+noble amd64 build of all seven packages through the new fatal dependency path (mk-build-deps installs the declared Build-Depends inside the oldest, focal, chroot), and the publish path end to end -- a staging-only run leaves the repo untouched; publish assembles aside, gates, and swaps; an unrelated codename survives the swap; stale debs are dropped; a failed gate leaves the published repo byte-identical and cleans up the side tree. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>