The build wrote its debs straight into the directory a publish assembles from,
and the smoke ran afterwards. A cancellation during the smoke, which reaches the
builder as a signal, left an unverified deb there with no cleanup, and the
publish gate gets no further than names and versions.
Each package now builds into a private directory under the result directory and
its debs are moved out only after every check passes. A leftover directory from
an interrupted run is removed before the next build of that package.
The debs land in staging before the smoke runs, and the publish gate checks
names and versions only. A deb whose binary could not run therefore stayed in
staging and was eligible for publication, which is the case the smoke exists to
catch. The failure now removes the debs it rejected and says so.
A cross-built deb links against the target's loader and libraries, neither of
which exists on the build host, so a binary that cannot run still produces a
green build. The rpm side installs the package into the mock chroot and runs -V
there; the deb side had no smoke at all.
build_deb_in_chroot now takes an optional smoke: it installs the produced deb
with apt-get inside the chroot that built it, runs the named command and matches
its output. apt-get rather than dpkg -i, so a wrong or missing Depends fails
here instead of on a node. --skip-install drops the smoke.
--install-deps named no qemu-user-static and no binfmt-support, so on a host
without them the mode reported success and the next command still refused to
create the riscv64 chroot. The message tells the operator to install those two
packages, which the mode meant to do.
The build timeout helper was imported at compile time, and it pulls in
XCAT::BuildUtils, which needs File::Slurper. The Ubuntu build hosts do not
all carry that module, so every caller began to depend on it -- including
--install-deps, whose whole job is to install it on a host that lacks it.
The helper is now loaded where it is used.
Every per-package build ran through a bare system() call with no wall-clock
bound. Under qemu-user a build can deadlock -- a riscv64 goconserver `go build`
held both Go pids in futex_wait for 26 minutes with no CPU ticks and no open
socket -- and the step then never returns. The pipeline does not go red; it
stops, and a stopped run reads as "still running".
XCAT::BuildUtils::run_bounded runs the command in its own process group, kills
that group when a budget expires, and prints first what the manual
investigation had to collect by hand: the process tree, each pid's kernel wchan
and stack, its open socket count, and the CPU ticks the group used across a
20-second sample. Zero ticks names a deadlock; ticks name a build that is only
slow. BuildUtils::build_deb_in_chroot bounds every Ubuntu package build, and
mockbuild-all.pl bounds the dep and perl steps of a forcearch target.
The budget is 900 seconds for a native build and ten times that for a foreign
architecture, because qemu-user under TCG runs at roughly a tenth of native
speed. Both sit about five times above the slowest build measured on
xcat-master-ub: 3 minutes native, and 26 minutes for riscv64 ipmitool-xcat on
resolute with four codenames building at once. Native mock steps stay unbounded
-- no measurement of them exists, and a guessed budget would turn a trusted
cell red. sbuild-all.pl --build-timeout and mockbuild-all.pl --build-timeout
override the default; 0 removes the bound.
t/build_timeout.t fails without this change: run_bounded never returns and the
test reports the hang instead of blocking.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The 2.19 release needs riscv64 debs, and sbuild-all.pl accepted only amd64 and
ppc64el, so no target produced them.
The supported architecture set moves into BuildUtils as one source of truth
(supported_arches/is_supported_arch), which --arch, --target and --expect-arch now
consult; the mirror rule becomes "anything but amd64 is on ubuntu-ports", which is
what ppc64el already needed and riscv64 needs too; and the genesis control remap
stops naming ppc64el.
ipmitool-xcat also could not build there. Its debian/control names architectures
explicitly and omitted riscv64, so debhelper reported "No packages to build.
Possible architecture mismatch" and the build died at ./configure. conserver and
goconserver say Architecture: any and needed nothing.
debs-manifest.conf gains a [<codename>-riscv64] section for each codename. It
lists neither the x86 boot loaders -- a riscv64 node netboots UEFI grub2 -- nor
xcat-genesis-base, whose riscv64 flavour is the OpenEmbedded one in the shared
pool.
t/sbuild-all.t covers the control-file defect: it fails on ipmitool without this
change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A CD run died at compile time inside XCAT::BuildUtils because xcat-master-ub was
missing File::Slurper: "Can't locate File/Slurper.pm in @INC", in the middle of a
build. It was fixed by hand, so the next unprovisioned host fails the same way
and the documented install line can drift from what the code actually loads.
--install-deps installs this host's prerequisites and exits: the sbuild/schroot
toolchain plus the modules. It then LOADS each module and fails naming any that
is still missing, rather than trusting apt's exit code.
That probe earned its place immediately: the first list named libipc-cmd-perl,
which does not exist on Ubuntu -- IPC::Cmd is core there -- and apt failed the
whole install over it. The package is gone from the list and the module is
asserted by loading instead, with a test that pins both halves of that reasoning.
The list and the command are pure functions in BuildUtils, so the decision is
unit-tested and the side effect stays in the caller. Run on xcat-master-ub and
xcat-master-ub-ppc: both report every module present.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A CI run of this branch died on resolute/ppc64el with a 404 fetching
libssl-dev_3.5.5-1ubuntu3.4_ppc64el.deb: a development suite rolled openssl and
dropped that version from the pool while the chroot's index still named it.
mk-build-deps was the one apt operation in the in-chroot script NOT wrapped in
apt_retry, so a single transient mirror inconsistency failed the package -- and,
with the matrix running failFast, took the other architecture's in-flight builds
down with it.
Retry it the same way the rest of the script retries apt, refreshing the index
between attempts, since a stale index is precisely what produces this. It stays
FATAL once the attempts are spent: a package must never build against whatever
the chroot happens to carry. The refresh goes through apt_retry, so every
apt-get in the script still runs under the fatal helper.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
PR #64 landed on master and added --genesis-release to build-apt-repo.sh -- the
very script this branch deletes, having absorbed the apt assembly + signing phase
into sbuild-all.pl. A plain merge would either resurrect the shell publisher or
silently drop the OpenEmbedded Genesis release from every apt suite, so the
feature is ported to where apt publication now lives.
sbuild-all.pl --genesis-release <dir>:
- The release is validated once at startup, before any build or publish, with the
same checksum-verify-checksum sequence mockbuild-all.pl uses on the rpm side, so
a release rewritten together with its SHA256SUMS while the verifier runs is
rejected. It must be complete (every supported architecture) and carry debs.
- During assemble_into, each release deb is copied into the codename's pool and the
flat per-version directory and re-checked against the verified checksums. That
happens with the publish lock held, between the pool wipe and apt-ftparchive, so
the bytes that are indexed and signed are the bytes that were verified -- the
separate re-verification pass build-apt-repo.sh ran before indexing has no
window left to cover here.
- Copies are plain copies, never link(): a pool file sharing an inode with the
release would let a write through either path change what the other holds.
- Anything staged under the OpenEmbedded Genesis package name is dropped when the
option is given; the verified release is the only source of those packages.
- XCAT::GenesisRelease is loaded on demand rather than imported at compile time. It
pulls in XCAT::BuildUtils, which needs File::Slurper, and xcat-master-ub does not
carry it: a compile-time import made every apt build -- including the ones that
never pass --genesis-release -- die with "Can't locate File/Slurper.pm".
Also here:
- --publish-lock-wait <seconds> makes the 1800s publish-lock wait settable, so a
caller that would rather fail fast than queue can, and so the lock is testable.
- t/genesis_openembedded_consumer.t: the four APT consumer tests now drive
sbuild-all.pl's real publish path (staging tree, publish lock, atomic swap)
instead of build-apt-repo.sh, including the new flock-based lock behaviour.
- The workflow compiles sbuild-all.pl and BuildUtils.pm instead of shellchecking
the removed script; BUILD.md and genesis-openembedded/README.md document the apt
invocation.
Full suite green on both build hosts: 345 tests on xcat-master-ub (Ubuntu 24.04,
where the APT and RPM consumer tests actually run) and 341 on xcat-master.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The manifest pinned only the UPSTREAM version: deb_version() and
resolve_present_names() both reduced the version with deb_upstream_version()
before comparing, dropping the epoch and the debian_revision. A package can
carry the right upstream version and still be the wrong package:
* the debian_revision is the PACKAGING revision -- elilo-xcat 3.14-5 and
3.14-6 are different builds of the same upstream 3.14;
* the epoch overrides version comparison outright -- an un-epoched 2.19.0
sorts BELOW 2:2.18.0.
That matters here because xCAT's own debian/control declares versioned
dependencies on these packages -- goconserver (>= 0.3.3-snap...), ipmitool-xcat
(>= 1.8.17-1), grub2-xcat (>= 2.02-...) in xCAT, xCATsn and xCAT-server -- so an
upstream-only pin can accept a deb that the gate calls good and that
`apt install xCAT` then refuses.
The pin is now matched against the FULL [epoch:]upstream[-revision], exactly as
it appears in the built .deb and in the published Packages index. No new
operator was needed: the existing exact/glob/'*' grammar simply applies to the
whole version, which is stricter than a >= floor because it also rejects an
epoch the pin does not name. Six pins become fully exact; two stay globbed
because their revision is not owned by the checkout:
* goconserver=0.3.3-snap* -- upstream exact; the revision is the CD stamp
snap<SOURCE_DATE_EPOCH>, which changes every run. The glob still REQUIRES a
snap-stamped revision, so an unstamped deb is rejected.
* xcat-genesis-base=2.* -- its version is not owned by xcat-dep; it walks
with whatever xcat-core the genesis was built from (XCAT_CORE_REF), and the
two arches legitimately differ since each is converted from its own rpm.
The gate still enforces exactly one genesis FOR THIS ARCH.
deb_version() gained an optional $arch for that logical name: both arch-suffixed
genesis debs are staged on the amd64 host (the cross-arch ppc one for #7610) and
carry different revisions, so without it the pair would look like a version
conflict. This mirrors resolve_present_names, which already refuses to borrow
another arch's genesis.
Comparing full versions also exposed a pre-existing defect it had been masking:
the collector treated PREBUILT .deb files checked into the source tree as build
output. elilo/ ships elilo-xcat_3.14-5_all.deb, elilo-xcat_3.14-6_all.deb and
gnu-efi_3.0v-5_amd64.deb; the first and third were being published into the apt
repo under this run's name, and the stale 3.14-5 collided with the freshly built
3.14-6 once revisions were compared. The build now snapshots the .debs present
before the build and subtracts them afterwards -- a file the build overwrites
changes size/mtime and still counts as output. Verified in the noble chroot: only
elilo-xcat_3.14-6_all.deb is collected; gnu-efi and 3.14-5 are reported and left
behind.
deb_upstream_version() has no remaining callers and is removed.
Tests: version_matches over whole versions (stale revision rejected, unnamed
epoch rejected, both globs still enforcing what they should); deb_version
returning the full version, dying on two revisions of one upstream version (the
elilo case), and selecting the per-arch genesis; resolve_present_names handing
the full version to the comparator; and two end-to-end cases in t/verify-repo.t
asserting the gate now fails a stale packaging revision and an unnamed epoch.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Addresses the three blocking issues from the PR #63 review.
1. THE TWO ARCH JOBS COULD BOTH REWRITE THE PUBLISHED REPOSITORY.
The amd64 and ppc64el runs execute concurrently on their two hosts against the
same --apt-dir. The run lock does not protect the repository -- it is per-arch by
design, precisely so the two arches CAN build in parallel -- so both were free to
wipe and repopulate the same pool/, dists/, Release, InRelease and tarball at the
same time and interleave into a corrupt but green repository.
Split by role, so architecture jobs produce staging artifacts and one locked
finalization step publishes atomically:
* A run that builds no longer publishes. It fills staging/<codename>/<arch>/
and stops, saying so and printing the finalize command. Publishing happens
only with --publish, or implicitly on a run that builds nothing
(--skip-build), which IS the finalization step; --skip-createrepo still
forces it off.
* publish_repo() takes ONE GLOBAL publish lock (.sbuild-all.publish.lock), not
the per-arch build lock, so even a cron run racing a manual one on the same
host serializes instead of interleaving.
* It publishes atomically. assemble_into() builds and signs the complete tree
in a side directory (<apt-dir>.publish-<run-id>.<pid>), the gate runs against
THAT tree, and swap_into_place() then renames it onto --apt-dir. A reader --
the deploy rsync, an apt client on a served tree -- sees either the previous
complete repo or the new complete repo, never a half-wiped pool or an index
that disagrees with its Release. A failure anywhere before the swap leaves
the published tree untouched and removes the side tree. The side tree is
seeded from the current published one, so codenames outside --dists survive.
* The tarball moved inside publish, under the same lock: previously every
per-arch build tarred the shared apt tree while the other arch rewrote it.
2. build_deb_in_chroot() WAS NOT FAIL-HARD, AND THE ENVIRONMENT WAS NOT CLEAN.
The common build-tooling install ended in `|| true`; a failed Build-Depends
installation only warned; and the Build-Depends came from a sed pipeline over
debian/control that stripped version constraints `(>= 12)`, could not express
alternatives `a | b`, and mangled arch qualifiers. Worse, the per-codename
chroots are long-lived and shared by all seven packages, and nothing asserted
that a schroot session is throwaway -- so on a chroot without a union mount,
package N's build-dependencies stayed installed for package N+1 and a package
whose debian/control forgets a Build-Depends would build green on a sibling's
leftovers.
Making dependency setup fatal is only half the fix; it means nothing if a stale
environment can satisfy an undeclared dependency. Both halves are addressed:
* The in-chroot program is now generated by the pure, unit-tested
BuildUtils::chroot_build_script() and runs under `set -euo pipefail`.
apt-get update and the common tooling go through an apt_retry helper that
retries a transient mirror hiccup and then FAILS the build. Build-Depends are
installed with mk-build-deps (devscripts + equivs), which hands
debian/control's relationships to apt verbatim, and a failure is fatal. The
mk-build-deps dummy package is excluded from deb collection alongside dbgsym.
* ensure_disposable_chroot() repairs a chroot.d entry that lacks
union-type=overlay, and build_deb_in_chroot() re-reads `schroot --config` and
hard-fails on a chroot that is still not disposable, naming the fix.
BuildUtils::chroot_is_disposable() is the pure predicate.
3. THE REPOSITORY GATE STILL HAD FALSE PASSES.
* Standalone --verify-repo skipped signature verification whenever --gpg-home
was not passed, so the common `--verify-repo <dir>` invocation silently
checked completeness only. Signatures are now verified BY DEFAULT there;
--no-verify-signature is the explicit opt-out. (The automatic pre-swap gate
still requires a signature iff --gpg-sign was used, so an intentionally
unsigned tree does not false-fail -- and publishing unsigned now warns.)
* The expected architecture set was inferred from what happened to be present,
so an entirely missing secondary architecture read as "this run did not build
it" and passed. It is now always a CLAIM: --expect-arch (repeatable, accepts
a space/comma list) if given, else the staged arch set when publishing, else
each codename's own Release "Architectures:" line when verifying standalone
(BuildUtils::parse_release_architectures). The pure
BuildUtils::verify_repo_arches then reports an expected arch with no NATIVE
package as MISSING-ARCH and natives for an unexpected arch as
UNEXPECTED-ARCH (a stale architecture). Native detection matters because the
Architecture:all packages ride into every arch's index, so a non-empty
binary-<arch>/Packages is not evidence that arch was built.
* Release now advertises exactly the expected arch set and only those
binary-<arch> indices are written, so an amd64-only build no longer
advertises a ppc64el it cannot serve.
* An expected cell with no manifest section used to be skipped with a note --
a third free pass. It is now a hard NO-MANIFEST error.
TESTS
t/sbuild-all.t gains verify_repo_arches (both directions), the
parse_release_architectures parser, chroot_is_disposable, regression guards on
the generated in-chroot script (no `|| true`, no warn-only build-deps, no sed
extraction, mk-build-deps present, every apt-get behind the fatal helper), and
the build_deb_in_chroot disposability guard driven through a stub schroot.
t/verify-repo.t is new: it drives the real `sbuild-all.pl --verify-repo` against
fixture apt trees and asserts each former false pass now fails -- missing
secondary architecture (with and without --expect-arch), arch:all-only index,
unsigned repo with no --gpg-home, missing manifest section -- and that an honest
single-arch repo still passes.
Validated on the build hosts: a full focal+noble amd64 build of all seven
packages through the new fatal dependency path (mk-build-deps installs the
declared Build-Depends inside the oldest, focal, chroot), and the publish path
end to end -- a staging-only run leaves the repo untouched; publish assembles
aside, gates, and swaps; an unrelated codename survives the swap; stale debs are
dropped; a failed gate leaves the published repo byte-identical and cleans up the
side tree.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The previous commit added the arch:all boot tools (syslinux-xcat/grub2-xcat/
elilo-xcat/xnba-undi) to the ppc64el manifest and taught build_one_codename to
skip BUILDING them on non-amd64 -- but validate_manifest, which runs per-arch on
every non-dry-run invocation and is NOT gated by --skip-createrepo, still
demanded them. On the ppc64el build stage (--arch ppc64el, no --skip-build) the
builder skipped the four (correct -- their source is x86-only) so they were never
staged, then validate_manifest reported them MISSING and aborted the whole ppc
build before assembly: every BUILD_PPC=true run would have failed.
Fix the drift structurally: extract the skip rule into one pure, tested decider
BuildUtils::skip_arch_all_on() behind a shared sbuild-all.pl helper
pkg_skip_on_arch(), consulted by BOTH build_one_codename and validate_manifest,
so a package the build skips is never demanded by the per-arch validation. The
arch:all debs' presence on ppc is still verified later against the published
index by verify_assembled_repo. Also make control_binary_arch return the full
Architecture value (not just the first token of a multi-arch list) and add a
regression test for the shared skip rule.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The ppc64el manifest sections listed only the natively-built deps
(ipmitool-xcat, conserver-xcat, goconserver) + genesis, omitting the four
noarch boot components syslinux-xcat, grub2-xcat, elilo-xcat and xnba-undi. A
ppc management node needs these for netboot -- the EL manifest and the
historical 2.16 ppc dep repo both ship them (grub2-xcat + syslinux-xcat
especially; yaboot-xcat was dropped in 2.18 as obsolete). They ARE published
to the ppc index today (Architecture:all rides into every binary-<arch>
index), but the completeness gate never asserted their presence for ppc.
List them as required-present on every ppc64el target so the gate verifies the
ppc repo carries them. They remain SINGLE-PRODUCER: their source is x86-only
(syslinux compiles with nasm/gcc-multilib), so build_one_codename now skips an
Architecture:all package on any non-amd64 arch -- detected via a new pure,
unit-tested control_binary_arch() helper -- so listing them for ppc64el drives
verification, not a second (failing) build.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The prior commit scoped the apt verify gate to '--arch floor UNION present
arches', but detected presence with a non-empty binary-<arch>/Packages. That
is wrong: every binary-<arch> index carries the Architecture:all debs
(grub2-xcat, genesis), so a BUILD_PPC=false run has a non-empty binary-ppc64el
index built purely from arch:all debs -- and the gate would still demand the
native ppc compiled deps (ipmitool-xcat, conserver-xcat, goconserver) it never
built, the exact false-fail the change was meant to remove.
Detect a genuine per-arch build via a new pure helper index_has_native_arch(),
which is true only when the index has a stanza with Architecture == that arch
(not merely Architecture:all). Unit-tested happy + the arch:all-only sad case.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Two review follow-ups to the Ubuntu matrix build:
1. verify_assembled_repo hardcoded {amd64, ppc64el}, so a single-arch run
(BUILD_PPC=false, or --verify-repo on an amd64-only tree) false-failed
demanding a ppc index it never built. The gate now verifies the --arch
set as a required FLOOR unioned with any arch that actually published a
non-empty index: the multi-arch assemble (invoked --arch amd64) still
verifies the ppc64el debs it carries, while a genuine single-arch run no
longer demands the absent arch. No Jenkinsfile/invocation change, so
master's shared inline job stays compatible.
2. standard_options advertised finalize-xcat-dep!/force-unlock!, which
sbuild-all.pl never wires (--force-unlock would even error as unknown).
Drop them.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Align the apt gate with the EL gate so both AGREE on success/failure:
- Duplicate: parse_packages_index now DIES loudly on a package with two DISTINCT versions
(stale .deb not cleaned), mirroring EL's rpm_version -- no more silent keep-highest.
Removed the dpkg keep-highest oracle; added a happy/sad test.
- Signature: sig_observed_key returns the signer fingerprint or undef (no presence-only
fallback); the gate hard-fails (SIGKEY) if --gpg-key-id doesn't resolve to a fingerprint,
so it always confirms the repo was signed by EXACTLY the CLI key. Signature is required
only when --gpg-sign was used.
- Align the MISSING message with EL (undef pin -> '*'). Document the gate + idiosyncrasies
in BUILD.md.
prove t/sbuild-all.t: 98/98.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up on the apt-repo gate:
- FALSE-PASS (concern #7610): genesis name-resolution picked xcat-genesis-base-amd64
alphabetically for BOTH cells (both genesis debs are Architecture:all and appear in every
arch index), so the ppc64el cell never verified its NATIVE xcat-genesis-base-ppc64el -- a
dropped ppc genesis passed. Extract a PURE, unit-tested resolve_present_names that resolves
the arch-suffixed genesis to THIS cell's arch only (never a different arch), with a test that
reproduces the masked-genesis case.
- signature: reject EXPKEYSIG/REVKEYSIG/EXPSIG (expired/revoked keys emit VALIDSIG too); drop
the short-GOODSIG-keyid fallback (could never equal the 40-hex expected fpr -> spurious WRONGKEY).
- FALSE-FAIL: the post-assembly auto-run now requires a signature only when --gpg-sign was
actually used (an unsigned-by-choice repo no longer dies UNSIGNED); standalone --verify-repo
keeps checking whenever a gpg key/home is configured.
- cosmetic: de-duplicate the MISSING-INDEX message.
prove t/sbuild-all.t: pure gate tests incl. the new genesis-resolution cases.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Adds a real gate on the ASSEMBLED apt repo, per codename x arch, using
debs-manifest.conf as the single source of truth, layered pure/testable:
- BuildUtils: verify_repo_packages(\%expected,\%present) (MISSING/VERSION),
verify_repo_signature(\%expected,\%observed) (UNSIGNED/WRONGKEY), and
parse_packages_index($text) -- all PURE and unit-tested (happy+sad, no dpkg-deb).
- sbuild-all.pl does the IO via one sub verify_assembled_repo: parses each published
binary-<arch>/Packages (resolving arch-suffixed names like xcat-genesis-base-<arch>,
reducing to upstream via deb_upstream_version to compare against the manifest pin),
runs gpg --verify on each dists/<cn>/InRelease and extracts the signer fingerprint,
then delegates to the two pure deciders and dies listing every [<cn>/<arch>] problem.
- Runs AUTOMATICALLY at the end of assemble_apt (once Packages + signed Release exist);
suppressible with --no-verify-repo; skipped under --dry-run. Also a standalone,
lock-free, build-free '--verify-repo=<apt_dir>' mode using the script's --manifest/
--dists/--gpg-key-id/--gpg-home. Replaces the coarse pool-global hard-coded check.
prove t/sbuild-all.t: 90/90 (was 71). Smoke-tested: complete tree passes; dropped pkg
-> MISSING; wrong version -> VERSION; missing index -> MISSING-INDEX; all die nonzero.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up for the Ubuntu sbuild matrix:
- Add a fail-fast exclusive flock over the whole run (<output-root>/.sbuild-all.lock,
file-scoped handle) so two overlapping runs can't corrupt the shared staging/apt
tree -- this is the root of the observed 'remove_tree .../staging/<cn>/<arch>:
Directory not empty' (an NFS silly-rename from a concurrent run).
- wipe_tree(): remove_tree that captures {error} and dies loud, so an ENOTEMPTY no
longer carps-and-continues leaving stale debs; used for all staging/pool/dists wipes.
- Wire the tested, hash-based cross_copy_genesis_deb into build_genesis (was a naive
glob+copy, so the unit-tested stale-dropping copier was dead code); remove the
genuinely-unused deb_snap_version/rewrite_changelog_top helpers + their subtests
(compiled deps intentionally ship their tracked changelog version).
- Dedupe assemble_apt on binary Package+Architecture (keep highest via
dpkg --compare-versions) so a double-produced genesis can't land two versions in
the pool, independent of the --skip-genesis contract.
- Derive Release Architectures from the arches actually staged (non-empty
binary-<arch>/Packages), not a hard-coded 'amd64 ppc64el'.
- goconserver: guard 'go mod init' when a go.mod exists (+ TODO to commit go.sum for
the pinned SHA). Accept-and-ignore the unused per-package --log-dir/--build-number/
--skip-install flags (documented). Remove orphaned make_deb.sh dispatchers
(build-debs-all, build.sh, ipmitool/build.sh) + update the READMEs.
perl -c clean; prove t/sbuild-all.t: 71/71.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The per-package <dep>/sbuild.pl were generic wrappers that shelled out to each
package's make_deb.sh, so a package's build brain was split across two files and
package-specific fixes (e.g. goconserver's Go toolchain) landed in make_deb.sh
instead of the builder -- inconsistent with the EL side, where <dep>/mockbuild.pl
IS the per-package builder.
Make each <dep>/sbuild.pl own its build (mirroring <dep>/mockbuild.pl), absorbing
its make_deb.sh (source prep, patches, toolchain, dpkg-buildpackage), and remove
all seven make_deb.sh. The common chroot orchestration -- ephemeral schroot session,
apt update, build-dep install, out-of-tree copy, SOURCE_DATE_EPOCH, deb collection +
host-side verification -- moves into BuildUtils::build_deb_in_chroot; each builder
supplies only its package-specific recipe (passed base64-encoded to avoid quoting
interplay through schroot).
Also make the older Ubuntu codenames buildable:
- goconserver: install a pinned modern Go (1.25.12) in the build -- focal/jammy ship
a Go too old to even auto-switch toolchains, and goconserver's pinned deps need
Go >= 1.25 (it is a static CGO-free binary, so the pinned toolchain is portable and
reproducible across codenames).
- ipmitool + goconserver: lower debian/compat 13 -> 12 and Build-Depends debhelper
(>= 12), since Ubuntu 20.04 (focal) ships debhelper 12; compat 12 also builds cleanly
on newer codenames.
Validated: all recipe shapes build green through the new path -- tarball
(ipmitool@focal), git-clone+pinned-Go (goconserver@focal), tarball+patches
(syslinux@noble), in-place (grub2-xcat), in-place+custom-rules (xnba); 80/80 unit
tests still pass.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu/Debian dependency build shipped as three bash scripts
(build-dep-debs.sh, build-apt-repo.sh, mk-dep-chroots.sh) whose review
(PR #63) surfaced correctness problems: partial/stale output could be
published, the rpm->deb genesis conversion dropped the maintained package
semantics (Depends/Breaks/Replaces + maintainer scripts), the arch matrix
was invalid (x86-only syslinux/elilo/xnba treated as ppc64el packages, and
Architecture:all packages with no single producer), several required
failures exited zero, and the build/repo scripts disagreed on their staging
path and codename set (focal missing from the assembler).
Rewrite it as proper, unit-tested Perl mirroring the EL side
(mockbuild-all.pl / MockBuildUtils.pm / <dep>/mockbuild.pl / t/*.t /
packages-manifest.conf), sharing one CLI vocabulary:
- BuildUtils.pm: shared, testable helpers + the canonical CLI spec, plus the
Debian-specific helpers (out-of-tree changelog stamping, genesis control
preservation, deb inspection, cross-arch genesis provisioning).
- sbuild-all.pl: the orchestrator, absorbing all three shell scripts. Builds
+ validates into a fresh per-arch staging tree and only (re)assembles the
published apt repo from validated staging -- so partial/failed output never
ships and stale debs never accumulate. Auto-initializes the per-codename
sbuild chroots on first run. Fails the whole run non-zero on any missing
chroot/package/artifact or version-pin mismatch.
- <dep>/sbuild.pl x7: per-package builders that drive each package's
MAINTAINED debian/ in the matching chroot (never re-implemented), so the
converted/built packages keep their control metadata and maintainer scripts.
- debs-manifest.conf: per-[<codename>-<arch>] required set + version pins,
encoding the per-arch package sets (x86 boot components built once on amd64
as the single producer; ppc64el builds only the arch-specific compiled deps).
- t/sbuild-all.t: fixture tests for every pure helper.
- goconserver/make_deb.sh: pin the upstream SHA instead of cloning a moving
branch, so every matrix cell builds the same source (reproducible).
Codename set unified across build, assembly, chroots and docs (focal IS
supported). BUILD.md documents the new flow.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Vinícius Ferrão
Daniel Hilst