Daniel Hilst
2a4eb248b4
PR #65 changed where the OpenEmbedded Genesis packages live. On the RPM side they are published once under xcat-dep/common; on the APT side, once under pool/main/xcat-genesis-openembedded, with every suite's Packages index pointing at that one copy instead of each suite carrying its own. It made that change in build-apt-repo.sh -- the script this branch deletes, having absorbed apt assembly and signing into sbuild-all.pl -- so the layout is ported here rather than lost. sbuild-all.pl: - --genesis-release now rebuilds a single shared pool inside the side tree and indexes it into every suite, so a release is stored once rather than once per codename. The debs are Architecture:all and identical everywhere; the previous per-suite copy multiplied hundreds of megabytes by the number of suites. - Because every suite's index points into that pool, publishing a release must cover every suite: a run whose --dists omits one is refused, instead of leaving that suite indexing files the new release retired. - OpenEmbedded Genesis debs are dropped from suite pools unconditionally now, not only when a release is being published -- they belong to the shared pool. - Published files get an explicit mode 0644: they are served by a web server running as another user, and inheriting the builder's umask is how that breaks. master's transaction machinery (per-file backups, --force-unlock recovery of an interrupted publisher) is deliberately NOT ported: it exists because the shell publisher writes into the live repository. sbuild-all.pl assembles a complete side tree, gates it, and renames it into place under one global publish lock, so a failed or killed run leaves the published repository untouched and there is no half-written state to recover. The tests assert that guarantee directly. The APT consumer tests now drive sbuild-all.pl's publish path with master's new expectations: the shared pool holds one complete release, suite pools hold none of it, every suite/arch index carries the shared Filename, a later single-suite rebuild keeps using the pool, a partial-suite release is refused, and a publication that cannot be signed leaves the packages, indexes and key exactly as they were. Full suite green: 416 tests on xcat-master (rome01, EL10) and 414 on xcat-master-ub, where the APT cases actually run. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
4.9 KiB
OpenEmbedded Genesis packages
xCAT builds the Genesis image in xcat-core. This directory turns those build
outputs into packages and feeds them into the xcat-dep repositories.
Run the release builder on a host that can build the OpenEmbedded layer. The
xcat-core checkout must be clean. --xcat-ref checks that the checkout points
to the intended commit; it does not change the checkout for you.
The packaging scripts use File::Slurper and IPC::Cmd. Install
libfile-slurper-perl on Ubuntu. On EL, install perl-File-Slurper and
perl-IPC-Cmd from EPEL and AppStream.
./genesis-openembedded/build \
--xcat-source /path/to/xcat-core \
--xcat-ref <tag-or-commit> \
--all \
--work-dir /path/to/oe-work \
--output-dir /path/to/xcat-genesis-release
The default format is all, which produces RPM, SRPM, and DEB packages. Use
--format rpm or --format deb when only one package family is needed. The
supported image architectures are x86, x86_64, ppc64, ppc64le,
armv7hf, aarch64, and riscv64.
Use --architecture for development builds. Repository publication requires a
complete release built with --all.
Each package installs one exact-architecture export under
/opt/xcat/share/xcat/netboot/genesis-openembedded/<architecture>/. The package
and install namespaces are separate from the old Genesis packages, so both
generations can be published and installed without replacing one another. The
packages are noarch or all because they are installed on the management
node, not run on the target node.
--work-dir keeps the OpenEmbedded downloads and build state between release
builds. Without it, the builder uses a temporary directory and removes it when
the command finishes.
The release directory contains:
rpm/,srpm/, anddeb/package directoriesrelease.manifest, including the xcat-core commitSHA256SUMS
Validate the directory before publishing it:
./genesis-openembedded/verify-release --complete /path/to/xcat-genesis-release
The checksum file detects incomplete or changed output. It does not authenticate the release, so only accept a directory produced by a trusted build host.
verify-release also checks the identity of each package, including a fixed
build host and a build time taken from the source epoch. Those are reproduced
by rpm 4.14.3, 4.16.1.3, 4.19.1.1, and 6.0.2. An EL8 or later builder, or a
current Fedora builder, produces a release the verifier accepts.
Pass that same directory to the repository builders:
perl ./mockbuild-all.pl \
--genesis-release /path/to/xcat-genesis-release \
[other build options]
perl ./sbuild-all.pl \
--genesis-release /path/to/xcat-genesis-release \
--publish --expect-arch "amd64 ppc64el" \
[other build options]
The RPM builder publishes the binary packages once under xcat-dep/common.
The per-EL repositories keep the old Genesis packages and contain no copies of
the OpenEmbedded packages. Source RPMs stay in the verified release directory.
The publisher locks the shared repository and replaces common only after the
new package set, metadata, signatures, and local setup files are ready.
The APT builder publishes the DEBs once under
pool/main/xcat-genesis-openembedded. Every suite indexes those same files.
Genesis publication updates all suites together, so a run whose --dists omits
one is refused. Later suite rebuilds keep using the shared pool. Pass a new
release only when replacing the Genesis packages.
APT metadata is assembled in a side tree and swapped onto the published repository with a single rename, under one global publish lock, so a failed or interrupted publication leaves the previous repository exactly as it was.
Both consumers require all seven architectures and verify package identities and checksums before publication. A management node can install an image for a different target architecture.
Without --genesis-release, both builders keep their existing behavior. The
new packages do not provide, replace, or obsolete the old package names.
xcat-core selects the OpenEmbedded install namespace when an image package is
present and falls back to the old Genesis image otherwise. Package installation
or update runs mknb for that package's architecture on nodes with local TFTP
storage. With site.sharedtftp=0, update service nodes before the management
node so they can serve the exact architecture name immediately. Removing an
image package retires its published files and rebuilds any legacy fallback that
is still installed. Generated images and packages belong in release storage,
not in Git.
The per-target RPM tarballs do not include xcat-dep/common. Mirror that
repository separately, including its metadata, when preparing an offline
installation.
Run the package tests on a Linux builder with RPM, DEB, and repository tools:
prove t/build_utils.t
prove -It/lib t/genesis_openembedded_release.t
sudo -E prove -It/lib t/genesis_openembedded_consumer.t