Align the apt gate with the EL gate so both AGREE on success/failure:
- Duplicate: parse_packages_index now DIES loudly on a package with two DISTINCT versions
(stale .deb not cleaned), mirroring EL's rpm_version -- no more silent keep-highest.
Removed the dpkg keep-highest oracle; added a happy/sad test.
- Signature: sig_observed_key returns the signer fingerprint or undef (no presence-only
fallback); the gate hard-fails (SIGKEY) if --gpg-key-id doesn't resolve to a fingerprint,
so it always confirms the repo was signed by EXACTLY the CLI key. Signature is required
only when --gpg-sign was used.
- Align the MISSING message with EL (undef pin -> '*'). Document the gate + idiosyncrasies
in BUILD.md.
prove t/sbuild-all.t: 98/98.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up on the apt-repo gate:
- FALSE-PASS (concern #7610): genesis name-resolution picked xcat-genesis-base-amd64
alphabetically for BOTH cells (both genesis debs are Architecture:all and appear in every
arch index), so the ppc64el cell never verified its NATIVE xcat-genesis-base-ppc64el -- a
dropped ppc genesis passed. Extract a PURE, unit-tested resolve_present_names that resolves
the arch-suffixed genesis to THIS cell's arch only (never a different arch), with a test that
reproduces the masked-genesis case.
- signature: reject EXPKEYSIG/REVKEYSIG/EXPSIG (expired/revoked keys emit VALIDSIG too); drop
the short-GOODSIG-keyid fallback (could never equal the 40-hex expected fpr -> spurious WRONGKEY).
- FALSE-FAIL: the post-assembly auto-run now requires a signature only when --gpg-sign was
actually used (an unsigned-by-choice repo no longer dies UNSIGNED); standalone --verify-repo
keeps checking whenever a gpg key/home is configured.
- cosmetic: de-duplicate the MISSING-INDEX message.
prove t/sbuild-all.t: pure gate tests incl. the new genesis-resolution cases.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Adds a real gate on the ASSEMBLED apt repo, per codename x arch, using
debs-manifest.conf as the single source of truth, layered pure/testable:
- BuildUtils: verify_repo_packages(\%expected,\%present) (MISSING/VERSION),
verify_repo_signature(\%expected,\%observed) (UNSIGNED/WRONGKEY), and
parse_packages_index($text) -- all PURE and unit-tested (happy+sad, no dpkg-deb).
- sbuild-all.pl does the IO via one sub verify_assembled_repo: parses each published
binary-<arch>/Packages (resolving arch-suffixed names like xcat-genesis-base-<arch>,
reducing to upstream via deb_upstream_version to compare against the manifest pin),
runs gpg --verify on each dists/<cn>/InRelease and extracts the signer fingerprint,
then delegates to the two pure deciders and dies listing every [<cn>/<arch>] problem.
- Runs AUTOMATICALLY at the end of assemble_apt (once Packages + signed Release exist);
suppressible with --no-verify-repo; skipped under --dry-run. Also a standalone,
lock-free, build-free '--verify-repo=<apt_dir>' mode using the script's --manifest/
--dists/--gpg-key-id/--gpg-home. Replaces the coarse pool-global hard-coded check.
prove t/sbuild-all.t: 90/90 (was 71). Smoke-tested: complete tree passes; dropped pkg
-> MISSING; wrong version -> VERSION; missing index -> MISSING-INDEX; all die nonzero.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up for the Ubuntu sbuild matrix:
- Add a fail-fast exclusive flock over the whole run (<output-root>/.sbuild-all.lock,
file-scoped handle) so two overlapping runs can't corrupt the shared staging/apt
tree -- this is the root of the observed 'remove_tree .../staging/<cn>/<arch>:
Directory not empty' (an NFS silly-rename from a concurrent run).
- wipe_tree(): remove_tree that captures {error} and dies loud, so an ENOTEMPTY no
longer carps-and-continues leaving stale debs; used for all staging/pool/dists wipes.
- Wire the tested, hash-based cross_copy_genesis_deb into build_genesis (was a naive
glob+copy, so the unit-tested stale-dropping copier was dead code); remove the
genuinely-unused deb_snap_version/rewrite_changelog_top helpers + their subtests
(compiled deps intentionally ship their tracked changelog version).
- Dedupe assemble_apt on binary Package+Architecture (keep highest via
dpkg --compare-versions) so a double-produced genesis can't land two versions in
the pool, independent of the --skip-genesis contract.
- Derive Release Architectures from the arches actually staged (non-empty
binary-<arch>/Packages), not a hard-coded 'amd64 ppc64el'.
- goconserver: guard 'go mod init' when a go.mod exists (+ TODO to commit go.sum for
the pinned SHA). Accept-and-ignore the unused per-package --log-dir/--build-number/
--skip-install flags (documented). Remove orphaned make_deb.sh dispatchers
(build-debs-all, build.sh, ipmitool/build.sh) + update the READMEs.
perl -c clean; prove t/sbuild-all.t: 71/71.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu/Debian dependency build shipped as three bash scripts
(build-dep-debs.sh, build-apt-repo.sh, mk-dep-chroots.sh) whose review
(PR #63) surfaced correctness problems: partial/stale output could be
published, the rpm->deb genesis conversion dropped the maintained package
semantics (Depends/Breaks/Replaces + maintainer scripts), the arch matrix
was invalid (x86-only syslinux/elilo/xnba treated as ppc64el packages, and
Architecture:all packages with no single producer), several required
failures exited zero, and the build/repo scripts disagreed on their staging
path and codename set (focal missing from the assembler).
Rewrite it as proper, unit-tested Perl mirroring the EL side
(mockbuild-all.pl / MockBuildUtils.pm / <dep>/mockbuild.pl / t/*.t /
packages-manifest.conf), sharing one CLI vocabulary:
- BuildUtils.pm: shared, testable helpers + the canonical CLI spec, plus the
Debian-specific helpers (out-of-tree changelog stamping, genesis control
preservation, deb inspection, cross-arch genesis provisioning).
- sbuild-all.pl: the orchestrator, absorbing all three shell scripts. Builds
+ validates into a fresh per-arch staging tree and only (re)assembles the
published apt repo from validated staging -- so partial/failed output never
ships and stale debs never accumulate. Auto-initializes the per-codename
sbuild chroots on first run. Fails the whole run non-zero on any missing
chroot/package/artifact or version-pin mismatch.
- <dep>/sbuild.pl x7: per-package builders that drive each package's
MAINTAINED debian/ in the matching chroot (never re-implemented), so the
converted/built packages keep their control metadata and maintainer scripts.
- debs-manifest.conf: per-[<codename>-<arch>] required set + version pins,
encoding the per-arch package sets (x86 boot components built once on amd64
as the single producer; ppc64el builds only the arch-specific compiled deps).
- t/sbuild-all.t: fixture tests for every pure helper.
- goconserver/make_deb.sh: pin the upstream SHA instead of cloning a moving
branch, so every matrix cell builds the same source (reproducible).
Codename set unified across build, assembly, chroots and docs (focal IS
supported). BUILD.md documents the new flow.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst