The build guides published with --expect-arch "amd64 ppc64el". An explicit
set bypasses staged-architecture discovery, and the assembly writes the
Packages indexes and Release metadata only for that set, so following them
produced a repository with no riscv64 index even when the build staged it.
PR #65 changed where the OpenEmbedded Genesis packages live. On the RPM side
they are published once under xcat-dep/common; on the APT side, once under
pool/main/xcat-genesis-openembedded, with every suite's Packages index pointing
at that one copy instead of each suite carrying its own. It made that change in
build-apt-repo.sh -- the script this branch deletes, having absorbed apt assembly
and signing into sbuild-all.pl -- so the layout is ported here rather than lost.
sbuild-all.pl:
- --genesis-release now rebuilds a single shared pool inside the side tree and
indexes it into every suite, so a release is stored once rather than once per
codename. The debs are Architecture:all and identical everywhere; the previous
per-suite copy multiplied hundreds of megabytes by the number of suites.
- Because every suite's index points into that pool, publishing a release must
cover every suite: a run whose --dists omits one is refused, instead of leaving
that suite indexing files the new release retired.
- OpenEmbedded Genesis debs are dropped from suite pools unconditionally now,
not only when a release is being published -- they belong to the shared pool.
- Published files get an explicit mode 0644: they are served by a web server
running as another user, and inheriting the builder's umask is how that breaks.
master's transaction machinery (per-file backups, --force-unlock recovery of an
interrupted publisher) is deliberately NOT ported: it exists because the shell
publisher writes into the live repository. sbuild-all.pl assembles a complete
side tree, gates it, and renames it into place under one global publish lock, so
a failed or killed run leaves the published repository untouched and there is no
half-written state to recover. The tests assert that guarantee directly.
The APT consumer tests now drive sbuild-all.pl's publish path with master's new
expectations: the shared pool holds one complete release, suite pools hold none
of it, every suite/arch index carries the shared Filename, a later single-suite
rebuild keeps using the pool, a partial-suite release is refused, and a
publication that cannot be signed leaves the packages, indexes and key exactly
as they were.
Full suite green: 416 tests on xcat-master (rome01, EL10) and 414 on
xcat-master-ub, where the APT cases actually run.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
PR #64 landed on master and added --genesis-release to build-apt-repo.sh -- the
very script this branch deletes, having absorbed the apt assembly + signing phase
into sbuild-all.pl. A plain merge would either resurrect the shell publisher or
silently drop the OpenEmbedded Genesis release from every apt suite, so the
feature is ported to where apt publication now lives.
sbuild-all.pl --genesis-release <dir>:
- The release is validated once at startup, before any build or publish, with the
same checksum-verify-checksum sequence mockbuild-all.pl uses on the rpm side, so
a release rewritten together with its SHA256SUMS while the verifier runs is
rejected. It must be complete (every supported architecture) and carry debs.
- During assemble_into, each release deb is copied into the codename's pool and the
flat per-version directory and re-checked against the verified checksums. That
happens with the publish lock held, between the pool wipe and apt-ftparchive, so
the bytes that are indexed and signed are the bytes that were verified -- the
separate re-verification pass build-apt-repo.sh ran before indexing has no
window left to cover here.
- Copies are plain copies, never link(): a pool file sharing an inode with the
release would let a write through either path change what the other holds.
- Anything staged under the OpenEmbedded Genesis package name is dropped when the
option is given; the verified release is the only source of those packages.
- XCAT::GenesisRelease is loaded on demand rather than imported at compile time. It
pulls in XCAT::BuildUtils, which needs File::Slurper, and xcat-master-ub does not
carry it: a compile-time import made every apt build -- including the ones that
never pass --genesis-release -- die with "Can't locate File/Slurper.pm".
Also here:
- --publish-lock-wait <seconds> makes the 1800s publish-lock wait settable, so a
caller that would rather fail fast than queue can, and so the lock is testable.
- t/genesis_openembedded_consumer.t: the four APT consumer tests now drive
sbuild-all.pl's real publish path (staging tree, publish lock, atomic swap)
instead of build-apt-repo.sh, including the new flock-based lock behaviour.
- The workflow compiles sbuild-all.pl and BuildUtils.pm instead of shellchecking
the removed script; BUILD.md and genesis-openembedded/README.md document the apt
invocation.
Full suite green on both build hosts: 345 tests on xcat-master-ub (Ubuntu 24.04,
where the APT and RPM consumer tests actually run) and 341 on xcat-master.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The two checksum passes around the verifier read as a duplicated walk; say what
the second one is for. Record which rpm versions reproduce the package identity
verify-release requires, so a build host outside that set is a known question
rather than a guess.
%_buildtime is not an rpm macro: built with it alone and no SOURCE_DATE_EPOCH,
rpm 4.14.3, 4.16.1.3 and 4.19.1.1 all stamped the current time. The build time
and build host the release verifier checks come from SOURCE_DATE_EPOCH with
use_source_date_epoch_as_buildtime, and from _buildhost -- both honoured on
every one of those versions and on rpm 6.0.2, where the package also builds
and verifies unchanged.
Vinícius Ferrão
Daniel Hilst