Building a SUSE 16 image from SLE media failed every package with
"key ID fec28eaf09d9ea69: NOKEY". Leap publishes that key as
gpg-pubkey-*.asc, which the existing glob picks up; SLE publishes the
same key only as repodata/repomd.xml.key, so nothing was imported.
15 media carries both spellings, so this changes nothing there.
udev's kmod builtin dlopens libkmod, and dracut installs it from an
inst_libdir_file line in a module-setup.sh rather than by following
NEEDED. Which module carries that line moved: the dracut on SLE 16
media declares it only in 00systemd, which the diskless module set
never loads, so the image came up with no libkmod, udev autoloaded
nothing, and the guest reached the network scan with only loopback.
Leap's newer dracut also declares it in 95udev-rules, which base
depends on, which is why Leap was unaffected.
OpenSSH 10 splits each connection into sshd-session and that into
sshd-auth, so the initramfs sshd on 2222 could not serve a single
session. el10 added sshd-session for the same reason.
SLES 16 ships no ISC dhclient and nothing provides dhcp-client, so the
image could not be built from SLES media at all. Leap carries dhcpcd
too, so one client covers both. el10 made the same move when RHEL
dropped dhclient.
A trailing comma made args.cmd a tuple holding the argv list, so
fancy_chroot called startswith on a list and the child died before exec.
permissions.local was written but never applied, leaving ssh-keysign
0755 and hostbased auth inoperative on SUSE diskless images.
- the aarch64 osdeploy spec builds the stateful suse16 addons but its
diskless loop was never extended, so the aarch64 rpm shipped suse16
without suse16-diskless and a packed image got a dangling addons.cpio.
- imgutil's builddeb keeps its own copy of the directory list that
confluent_imgutil.spec.tmpl has, and it had learned about neither suse16
nor el10.
- gather_bootloader gained a /usr/share/efi fallback for shim on both
architectures but only for x86_64 on grub, so an aarch64 root found a
shim and then died copying grub.
Finally, rewriting repos.d file by file rather than copying the tree meant
a subdirectory or a file that is not valid UTF-8 aborted the build before
any package was installed, which also regressed SUSE 15. Pass anything
that is not a plain text repo definition through untouched and restore the
modes on the ones that are rewritten.
SuseHandler refused anything but 15.x. What 16 needed beyond widening it:
- its repo urls are written in terms of ${releasever}, which zypper
resolves from the target root's os-release, a file that does not exist
yet when the first packages go in
- its repos name a zypper service backed by a package-provided directory
the target root does not have, so zypper discarded every one of them
as an orphan
- there is no mkinitrd to work out which kernel to build for, and bare
dracut would build for the build host's running kernel
- the efi payloads moved out of /usr/lib64/efi, arping out of /usr/sbin,
nsswitch.conf and protocols under /usr/etc, and the presets enable
sshd already
- the module list predated virtio, so an image built for a KVM guest had
no network at all, and dm-crypt could not allocate a transform for the
encrypted image without the aes-xts modules
- urlmount still links libpthread, an empty stub since glibc 2.34 that
nothing else in the initramfs pulls in
Hand written rather than autofixed, since three of the four need the
surrounding code read to be sure they are equivalent:
- confetty: `powerstate == None` -> `is None`.
- nodeconfig: `setmode != True` / `!= False` -> `not setmode` / `setmode`.
Safe because setmode only ever holds None, True or False, and the two
lines above each test normalise None away first.
- pam: split two `if cond: stmt` one-liners.
- imgutil: `from shutil import copytree as copytree`, an alias that renames
nothing. Not a re-export marker, this is a script.
imageboot falls back to cp when unsquashfs is unavailable, but the build
side did not: a bare dracut_install/copy_exec aborts initramfs generation
when the binary is absent, and the capture prerequisite check refused to
capture the image at all.
Mark the initramfs copies optional and report the missing package as an
advisory rather than a hard prerequisite, so such images still build and
capture, just without the faster extraction path. Widen the EL check to
every release past el8 so future ones inherit it.
Release tags do not live on master: 3.15.2 through 3.15.6 were tagged on branch
3.15, so git describe reaches only 3.15.1 and dev builds were stamped
3.15.2.dev<n>. Besides being confusing, rpm and dpkg both rank the released
3.15.6 above that, so a dev package will not install over a released one.
Add a top-level VERSION file naming the release the branch is working toward
(4.0.0 on master) and a mkversion helper that stamps packages from it, keeping
the tag-derived value as a floor so a forgotten bump cannot go backwards.
mkversion also replaces the block copy-pasted into seven build scripts, and
makesetup no longer writes a per-package VERSION file, so the stale checked-in
confluent_common/VERSION goes with it.
confluent_imgutil does not depend on confluent_server, and the yaml
import is optional too, yet both capture and pack dereference osimage
and yaml unconditionally when writing manifest.yaml. With
confluent_osdeploy present but the server absent that raises rather
than producing a profile.
Guard the manifest on both being importable and say so, since rebase
is what the manifest exists for. The yaml fallback now binds None
instead of leaving the name undefined.
The two call sites carried the manifest write verbatim in both, so fold
them into one function rather than duplicate the guard as well.
Nothing unwound the loop device and dm-crypt mapping when the copy loop
raised, so interrupting a pack stranded both, still holding the profile's
rootimg.sfs.
Tear them down from a finally. The retry loop moves with them, so also
honour its tries counter, as unpack_image already does; spinning forever
inside a finally would hang the interrupt it is meant to clean up after.
A bounded retry loop can also give up, and the detach that follows would
then fail with EBUSY and, raising from a finally, replace the exception
that brought us here. Warn and leave both in place instead.
Both functions became coroutines solely to await one get_hashes call,
but their bodies are long stretches of blocking work: mksquashfs, the
encrypt_image copy loop, rsync, ssh and osdeploy.
From Python 3.11 on, asyncio.run installs a SIGINT handler that cancels
the main task and returns rather than raising, so an interrupt is only
noticed at the next await. Interrupting a pack during mksquashfs
surfaced as a CalledProcessError from the dying child instead of a
KeyboardInterrupt, and with a base profile, where nothing is ever
awaited, pack carried on and published the profile before exiting.
Run the loop only around the call that needs it.
capture and pack hash the whole profile directory, which by that point
holds rootimg.sfs, the kernel and the distribution initramfs. rebase
only ever looks up entries that came from the profile source directory,
so the image blobs cost gigabytes of hashing for nothing.
Pass the source directory as the filter, as generate_stock_profiles
already does. Older manifests keep working, since rebase reads their
entries with a default.
`unsquashfs` can use multiple CPU cores during image extraction, significantly reducing boot time.
For example the whole boot time from PXE to shell on a 8-core VM, from approximately 45 seconds to 20 seconds.
This PR adds `squashfs-tools` as a dependency. Since the package is smaller than 1 MB, the additional image size is justified by the performance improvement.
For backward compatibility, the existing `cp`-based extraction method is used when `unsquashfs` is unavailable, such as with images built before this change.
The extraction logic has also been moved into the common functions and is now shared between EL9, EL10, and Ubuntu.
Both untethered `squashfs` images and `confluent_multisquash` images are supported.
Images must be rebuilt to include `unsquashfs` and benefit from the faster extraction path.
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.
Without this fix the onboot services failed to start on SELinux enabled
captured image.
Allow building EL and Ubuntu diskless images for a foreign architecture (e.g.
aarch64 on an x86_64 host) by leveraging qemu-user-static. The target
architecture is detected automatically from a -s source tree (for EL),
or may be requested explicitly with the new --arch option.
When the target differs from the host, dnf/debootstrap is invoked with
--forcearch/--arch and the presence of an enabled binfmt_misc handler
with the F (fix-binary) flag is verified up front, so emulation keeps working
inside the installroot chroot and a missing setup yields an actionable
error instead of a confusing exec failure mid-build.
The image architecture is recorded in confluentimg.buildinfo so that
pack selects the initramfs addons for the image architecture rather
than the build host, and exec of a foreign-arch root performs the same
binfmt check.
Copy Debian apt sources and keyrings into the target before apt runs. Run apt with DEBIAN_FRONTEND=noninteractive.
Return constrained child status to callers, and make pack fail clearly when no kernel was installed.
Jarrod Johnson
Markus Hilger
Jarrod Johnson
Vinícius Ferrão