EL10 x86_64/ppc64le take these perl deps of xCAT from EPEL; riscv64 has no EPEL
and Rocky Linux 10 riscv64 BaseOS/AppStream/CRB do not carry them, so xcat-dep
builds them for that arch. Add them to mockbuild-perl-packages.pl's package
table, and a --epel-gap switch that appends them to the build list (the
default list6 build is unchanged):
perl-Crypt-Blowfish EPEL 10 src.rpm (2.14-25.el10_0, vendored now; the spec
in the dir is a SUSE one); its optional tests need
Crypt::CBC, so it 'needs' our perl-Crypt-CBC
perl-Crypt-CBC perl-Crypt-CBC.spec + Crypt-CBC-2.33.tar.gz: the spec
is now the Fedora one of the vendored fc29 src.rpm
(release 21) with the BuildRequires an EL10 buildroot
lacks (make, perl-interpreter); it replaces the SUSE
spec (perl-macros) that could not build on EL (noarch)
perl-Crypt-Rijndael perl-Crypt-Rijndael-1.13-10.fc29.src.rpm
perl-Digest-SHA1 perl-Digest-SHA1-2.13-23.fc28.src.rpm
perl-Expect perl-Expect-1.35-6.fc29.src.rpm (noarch)
perl-Mail-Sender perl-Mail-Sender-0.903-7.fc29.src.rpm (noarch)
perl-Net-DNS Net-DNS.spec + Net-DNS-0.80.tar.gz, now built with
--noxs as noarch (the cpan2rpm spec hard-coded
buildarch x86_64 for the XS dn_expand) and with the
BuildRequires cpan2rpm specs never carry; release 2
perl-Net-IP perl-Net-IP-1.26-30.el10_0.src.rpm (noarch)
perl-Path-Class new dir, EPEL 10 src.rpm (0.37-24.el10_0, noarch): only
a build dep -- Crypt-SSLeay's Makefile.PL needs it --
so perl-Crypt-SSLeay 'needs' it; on x86_64 EPEL still
provides it and nothing changes there
Two EPEL-only deps of xCAT are deliberately not built: perl-SOAP-Lite
(1.27-3.fc29 src.rpm is in the table for completeness, but its BuildRequires
IO::SessionData, MIME::Lite, XML::Parser::Lite and Test::XML are EPEL-only as
well, so it can neither be built nor installed without EPEL; xCAT uses it for
HP blade/VirtualBox support only) and perl-DB_File (needs libdb, which EL10
dropped and Rocky Linux 10 riscv64 does not have at all; only xCAT-server's
Confluent client uses it and xCAT-server merely recommends the package).
The per-package builders take the arch of what they build from 'uname -m', which
is wrong for a forcearch mock config such as rocky-10-riscv64-xcat built on an
x86_64 host: the chroot produces riscv64 rpms and the scripts then reject them
("Unexpected RPM arch"). Give every arch-producing builder a --target-arch
option (default: uname -m, so nothing changes for native builds):
- ipmitool/mockbuild.pl, conserver/mockbuild.pl: look for and verify
<target-arch> rpms. ipmitool's install smoke test cannot install a foreign
rpm on the host, so for a cross build it installs the rpm into the build
chroot with mock --install and runs ipmitool-xcat -V there (conserver already
smoke-tests in the chroot).
- goconserver/mockbuild.pl: the binaries are built on the host, so cross-compile
with GOARCH (x86_64 amd64, aarch64 arm64, riscv64 riscv64, ...) and package
with rpmbuild --target <arch>; rpm refuses 'BuildArch: <foreign arch>' on
this host ("No compatible architectures found for build"), so that line is
only emitted for native builds. A cross build cannot install its rpm on the
host either, so the smoke test unpacks it and runs goconserver and congo
through the binfmt handler (qemu-user-static) the forcearch mock builds of
the other deps need anyway.
- mockbuild-perl-packages.pl: --target-arch for the 'native' rpm check and the
default result/log dirs; the 'native' (XS) packages of a cross build are
smoke-tested inside the build chroot (mock --install, then perl -M<module>
there) instead of on the host; --noarch-mock-cfg to build the noarch
packages in a native chroot of the same release instead of the emulated one
(the rpms are identical for every arch, and an emulated perl build is an
order of magnitude slower); and a 'needs' key in the package table: a
package that needs others is built after them, in waves, with their rpms
installed into its chroot via mock --additional-package. Needs outside the
selected set are ignored, so the default list6 build is unchanged (the
chroot provides the module, e.g. from EPEL). This is what lets an EPEL-free
chroot build perl-Crypt-Blowfish on top of our own perl-Crypt-CBC, or
perl-Crypt-SSLeay with perl-Path-Class.
xCAT::NetworkUtils uses Net::IP, and perl-Net-IP is EPEL-only on EL10, so an
architecture without EPEL (riscv64) has to get it from xcat-dep. Vendor the
EPEL 10 source rpm, as the other per-package dirs do with their Fedora
src.rpms:
perl-Net-IP/perl-Net-IP-1.26-30.el10_0.src.rpm (noarch)
and register it in mockbuild-perl-packages.pl's package table in 'srpm' mode
(not in the default build list: EL10 x86_64/ppc64le keep taking it from EPEL).
xcat-dep has no riscv64 build host, and mock-core-configs' rocky-10-riscv64.cfg
cannot be used from the x86_64 hosts that build the repos: it only admits a
riscv64 host (legal_host_arches) and names its chroot 'rocky-10-x86_64'.
Ship mock-configs/rocky-10-riscv64-xcat.cfg: the stock templates/rocky-10.tpl
(BaseOS/AppStream/CRB/extras for $basearch=riscv64) with root
'rocky-10-riscv64-xcat', target_arch riscv64, legal_host_arches x86_64+riscv64
and forcearch riscv64, so mock runs the Rocky Linux 10 riscv64 chroot through
user-mode QEMU and the packages are built by the chroot's own riscv64
toolchain. There is no EPEL for riscv64, so nothing EPEL-only is reachable from
this chroot (the build scripts deal with that in the following commits).
Document in BUILD.md what the host needs before this config works: a static
qemu-riscv64 registered in binfmt_misc with the F flag (EL10 has no
qemu-user-static-riscv package; take the binary from a Fedora container), the
/usr/bin/qemu-riscv64-static file mock insists on, podman for the bootstrap
image, golang for the goconserver cross-compile, the install of the config
into /etc/mock/ and the commands that validate the setup. Verified on an
x86_64 EL10 host with mock 6.7: chroot init in about 3.5 minutes, 'uname -m'
inside the chroot reports riscv64.
riscv64 nodes boot through UEFI firmware and grub2 only, and xCAT hands
DHCP client architecture 27 (0x001b) the boot file
boot/grub2/grub2.riscv64. Nothing shipped that file.
Add the unmodified grub2 UEFI image from the Rocky Linux 10 riscv64
BaseOS tree (EFI/BOOT/grubriscv64.efi, the copy used for installation
media and network boot, built from grub2-2.12-46.el10_2.rocky.0.1.src.rpm,
GPL-3.0-or-later, provenance and checksum recorded in the readme) as
Source2, install it under /tftpboot/boot/grub2/riscv64-efi/ and let the
rpm and deb scripts copy it to /tftpboot/boot/grub2/grub2.riscv64 the
same way grub2.ppc is made from the POWER core image. mockbuild.pl
verifies the image type (it reads the PE header itself: file(1) only
learned the RISC-V machine types in 5.38, so an EL8 build host would
reject a good image), the rpm payload and the installed file (byte for
byte the source image); the deb installs the same tree.
Pin that a locked APT directory is refused with a message naming it, that
--force-unlock takes over and the lock is released at the end, and that a run
which skips building keeps the build results it exists to collect.
Everything from the pool wipe to the signature is one transaction over a shared
tree, and the Genesis packages are verified inside it, so a second writer between
that verification and apt-ftparchive would be indexed and signed unchecked --
or, arriving later, would leave clients with checksum failures against metadata
this run signed.
Take the tree for the duration with an atomic mkdir lock, the same NFS-safe
scheme mockbuild-all.pl uses for its output, released on exit and overridable
with --force-unlock when a killed run left one behind.
The staging repositories are cleared per invocation, but the builder results are
not: a reused --run-id leaves the previous run's packages under build-results,
where a step that fails this time is collected from the last time it succeeded.
Individual failures are tolerated by design, so this can mix two invocations in
one signed repository.
Start the result tree empty whenever the run builds. --skip-build collects the
repository-level build-output tree instead and keeps what is there.
Pin the rule that decides whether an invocation produced anything: every
attempted step failing is a total failure, one survivor is not, and a run with
no steps to attempt is unaffected.
Tolerating individual dep-builder failures is deliberate: some packages are el-
or arch-pinned and are expected to fail on some targets. Tolerating all of them
is not -- it means the builder is unusable, the invocation produced nothing, and
whatever the run publishes came from somewhere other than this build.
Count the failures on both the serial and the parallel path and stop when they
account for every attempted step, before collection can take an earlier run's
artifacts for this one's. The rule itself lives in BuildUtils, where it can be
exercised without a builder.
A package left in the staging repository by an earlier run is invisible to
collection but visible to createrepo and deploy. Pin that the run clears it, so
the empty-collection guard cannot be satisfied by an earlier invocation's output.
The collection guard counts what this run copied, but the staging repositories
were never cleared: packages left by an earlier run with the same --run-id sat
there unseen by collection, were indexed by createrepo and published by
deploy_target, where the name-only dependency assertion accepted them. A run
whose builders all failed could therefore ship a previous run's packages.
Empty the staging repositories before collection, so everything they hold
afterwards comes from this invocation.
Pin that every pooled Genesis package is checked against the release manifest
again before apt-ftparchive reads the pool, so the check cannot be dropped
without a failing test.
apt-ftparchive indexes and the signature covers whatever is in the pool at that
moment, while the packages were checked when they were copied. Verify them again
against the release manifest immediately before the indexes are generated, so a
package that changed in between cannot be published as a verified one.
The pooled package carries the release bytes, which the existing digest check
covers, but it must not be the release file itself; assert the identity is
distinct so a link cannot come back unnoticed.
Every selected suite receives the whole release, so publication copied one image
per architecture per codename -- hundreds of megabytes each, several gigabytes on
the build host and again in the published tree.
Use --reflink=auto: filesystems that can share extents copy-on-write stop paying
for the duplication, and the pool entry stays a file of its own everywhere, which
a hardlink would not -- that would leave the published package and the verified
release sharing an inode, where a write through either path changes the other.
Vinícius Ferrão
Daniel Hilst