imageboot falls back to cp when unsquashfs is unavailable, but the build
side did not: a bare dracut_install/copy_exec aborts initramfs generation
when the binary is absent, and the capture prerequisite check refused to
capture the image at all.
Mark the initramfs copies optional and report the missing package as an
advisory rather than a hard prerequisite, so such images still build and
capture, just without the faster extraction path. Widen the EL check to
every release past el8 so future ones inherit it.
Release tags do not live on master: 3.15.2 through 3.15.6 were tagged on branch
3.15, so git describe reaches only 3.15.1 and dev builds were stamped
3.15.2.dev<n>. Besides being confusing, rpm and dpkg both rank the released
3.15.6 above that, so a dev package will not install over a released one.
Add a top-level VERSION file naming the release the branch is working toward
(4.0.0 on master) and a mkversion helper that stamps packages from it, keeping
the tag-derived value as a floor so a forgotten bump cannot go backwards.
mkversion also replaces the block copy-pasted into seven build scripts, and
makesetup no longer writes a per-package VERSION file, so the stale checked-in
confluent_common/VERSION goes with it.
confluent_imgutil does not depend on confluent_server, and the yaml
import is optional too, yet both capture and pack dereference osimage
and yaml unconditionally when writing manifest.yaml. With
confluent_osdeploy present but the server absent that raises rather
than producing a profile.
Guard the manifest on both being importable and say so, since rebase
is what the manifest exists for. The yaml fallback now binds None
instead of leaving the name undefined.
The two call sites carried the manifest write verbatim in both, so fold
them into one function rather than duplicate the guard as well.
Nothing unwound the loop device and dm-crypt mapping when the copy loop
raised, so interrupting a pack stranded both, still holding the profile's
rootimg.sfs.
Tear them down from a finally. The retry loop moves with them, so also
honour its tries counter, as unpack_image already does; spinning forever
inside a finally would hang the interrupt it is meant to clean up after.
A bounded retry loop can also give up, and the detach that follows would
then fail with EBUSY and, raising from a finally, replace the exception
that brought us here. Warn and leave both in place instead.
Both functions became coroutines solely to await one get_hashes call,
but their bodies are long stretches of blocking work: mksquashfs, the
encrypt_image copy loop, rsync, ssh and osdeploy.
From Python 3.11 on, asyncio.run installs a SIGINT handler that cancels
the main task and returns rather than raising, so an interrupt is only
noticed at the next await. Interrupting a pack during mksquashfs
surfaced as a CalledProcessError from the dying child instead of a
KeyboardInterrupt, and with a base profile, where nothing is ever
awaited, pack carried on and published the profile before exiting.
Run the loop only around the call that needs it.
capture and pack hash the whole profile directory, which by that point
holds rootimg.sfs, the kernel and the distribution initramfs. rebase
only ever looks up entries that came from the profile source directory,
so the image blobs cost gigabytes of hashing for nothing.
Pass the source directory as the filter, as generate_stock_profiles
already does. Older manifests keep working, since rebase reads their
entries with a default.
`unsquashfs` can use multiple CPU cores during image extraction, significantly reducing boot time.
For example the whole boot time from PXE to shell on a 8-core VM, from approximately 45 seconds to 20 seconds.
This PR adds `squashfs-tools` as a dependency. Since the package is smaller than 1 MB, the additional image size is justified by the performance improvement.
For backward compatibility, the existing `cp`-based extraction method is used when `unsquashfs` is unavailable, such as with images built before this change.
The extraction logic has also been moved into the common functions and is now shared between EL9, EL10, and Ubuntu.
Both untethered `squashfs` images and `confluent_multisquash` images are supported.
Images must be rebuilt to include `unsquashfs` and benefit from the faster extraction path.
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.
Without this fix the onboot services failed to start on SELinux enabled
captured image.
Allow building EL and Ubuntu diskless images for a foreign architecture (e.g.
aarch64 on an x86_64 host) by leveraging qemu-user-static. The target
architecture is detected automatically from a -s source tree (for EL),
or may be requested explicitly with the new --arch option.
When the target differs from the host, dnf/debootstrap is invoked with
--forcearch/--arch and the presence of an enabled binfmt_misc handler
with the F (fix-binary) flag is verified up front, so emulation keeps working
inside the installroot chroot and a missing setup yields an actionable
error instead of a confusing exec failure mid-build.
The image architecture is recorded in confluentimg.buildinfo so that
pack selects the initramfs addons for the image architecture rather
than the build host, and exec of a foreign-arch root performs the same
binfmt check.
Copy Debian apt sources and keyrings into the target before apt runs. Run apt with DEBIAN_FRONTEND=noninteractive.
Return constrained child status to callers, and make pack fail clearly when no kernel was installed.
To handle amd64/arm64 profiles, have the pkglist allow for architecture specific qualifiers.
Additionally, soften failure to accomplish selinux changes.
The r8169 enjoys some popularity.
Also, we'd like to be able to try out diskless/cloning with
VMs, so vmxnet3 and virtio_net are handy to round that out.
Jarrod Johnson
Markus Hilger
Jarrod Johnson
Vinícius Ferrão