Since we have a Seccomp cBPF going anyway, add insmod/modprobe operations to block.
Also reframe things to be a bit more readable using enum and less magical numbers.
EL 9.8 ships OpenSSH 9.9, whose sshd runs each connection through
/usr/libexec/openssh/sshd-session. Without it the initramfs sshd refuses
to start, so debugssh does not work on EL 9.8 images. Install it when
present, older EL9 releases do not have it.
An image meant to build without network access must contain every
package build checks for, and that list differs by distribution, version
and architecture and changes between releases. build --containerfile
writes a Containerfile, or with --apptainer an Apptainer definition
file, generated from the same package lists, including -p, -a and
--selinux, so it cannot drift from what build expects. The OS is that of
the -s distribution or of the build system, as for a build. --arch names
the matching build platform and --from replaces the default base image.
-r is refused, the image installs from the repositories of its base
image.
The generated EL file installs full coreutils instead of the
coreutils-single of container bases, and the Ubuntu file cleans the apt
cache.
For sites that already build images elsewhere, for example in CI from a
Dockerfile, imgutil build -s also takes a container image: a SIF file,
an OCI or Docker archive, an OCI layout directory, or a podman reference
with its transport, such as docker:// or containers-storage:. Files and
directories are recognized by their content, and any other source is a
distribution as before. The image is extracted into the scratch
directory, with podman or, for a SIF file, from its squashfs partition.
It is then completed the way build does: the confluent initramfs
configuration, the default package list with the package manager and
repositories of the image, and an initramfs for every kernel. The result
works with exec and pack like any built tree, and the build info names
the image as its source.
When the image already has every package of the list, the package
manager is not run, so an image prepared in CI builds without network
access. Missing packages are named before installing them. The
architecture comes from the image, and a requested --arch is checked
before extracting it. -r is refused because the image brings its own
repositories. With --arch, the matching variant of an image for several
architectures is used. Other than a local image, podman pulls or loads
the image into a temporary storage that is removed afterwards, so the
images and tags of the system stay as they are.
Container base images are not bootable systems, so the build also
unmasks the units they mask (systemd-remount-fs, getty.target,
systemd-logind and others) and removes policy-rc.d, reporting each
change, drops the boot entries kernel-install writes (they carry the
command line of the build system, installing to disk writes its own),
and with --selinux labels the tree with full contexts. Ubuntu images get
ubuntu-minimal, which debootstrap installs but container bases lack.
EL9, Ubuntu 24.04 and SUSE 16 or later images are supported, as long as
imgutil has a package list for the release. Older releases and
architectures other than x86_64 and aarch64 are refused.
The default 'max' cpu of qemu-user emulates SVE and pointer
authentication, which aarch64 binaries use and which is expensive to
emulate. Default QEMU_CPU to neoverse-n1 for aarch64 builds and exec,
unless it is already set. neoverse-n1 needs qemu 7.1 or later, the
documentation says how to choose another cpu.
Building an EL10 image with mlnx-ofa_kernel-dkms and doca-ofed-userspace
on an EL10 x86_64 host with qemu 10.2:
DOCA build Total
aarch64, master 3270 s (54.5 min)
aarch64, branch 2399 s (40 min), −27%
x86_64, native 251 s (4 min)
Older qemu-user-static, such as qemu 8.2 of Ubuntu 24.04, emulates
pointer authentication much more slowly, so builds there can get up to
ten times faster.
suse does not allow group readable ssh keys, so
keysign must be setuid root for host based authentication.
Also, chkstat got renamed, try to use the new name if possible.
A diskless boot of a captured profile only gets the profile's kernelargs,
not the kernel arguments of the source. Sources relying on some, such as
net.ifnames=0 for their interface names, then came up with different
interface names than the source and an install to disk of the same
image. Say so in the generated profile.yaml and after the capture, and
end the file with a newline so appended settings do not run into the
kernelargs line.
Capture copies an initramfs-tools hook and conf.d/confluent.conf into the
source to build the deployment initramfs, but only ever removed the EL
dracut module afterwards. With MODULES=list left in place, the source's
next initramfs update was limited to the hook's driver list, and a source
captured again carried both files into the image, where rebuilding the
initramfs on install to disk shrank the clones' initramfs the same way.
Remove them after capture and leave them out of the image.
Dropping every NetworkManager profile and ifcfg file also removed
settings that confignet does not recreate from confluent attributes,
such as bonds, DHCP profiles, MTU or the InfiniBand transport mode. Only
drop the profiles that carry static addresses of the source.
dhcpcd, present on Ubuntu 24.04 and later and on EL10, keeps its leases,
DUID and secret in /var/lib/dhcpcd, and NetworkManager keeps a secret in
secret_key. Copied into an image, every clone would use the same DHCPv6
DUID and derive the same stable privacy IPv6 addresses. Both regenerate
them when missing.
dhclient keeps its leases in /var/lib/dhcp there, and on EL also as
*.leases next to the *.lease files already excluded. Keep dhcpd.leases
of a DHCP server, which lives in the same directory.
The image kept /etc/hostname of the source host, so a diskless node ran
under that name until onboot set its own, and early journal and syslog
entries carried the wrong name. With an empty /etc/hostname systemd
keeps the node name the deployment initramfs already set. Install to
disk writes /etc/hostname itself.
Captured images kept the NetworkManager connection profiles and ifcfg
files of the source host. A diskless boot kept them too, and confignet
only rewrites the profiles named like the interfaces it configures, so
every other interface came up with the source host's settings, such as
its static addresses. Install to disk already removes them, and capture
already masks netplan files for the same reason. Drop them from the
image, except ifcfg-lo.
mksquashfs matches excluded paths by inode, so excluding a hard linked
file also dropped its other links elsewhere in the filesystem. Keep such
files in the image. squashfs-tools 4.3 also splits exclude file lines on
whitespace, so pass paths with whitespace or backslashes with -e instead.
Capture copied everything on the local filesystems, including user
files and sockets in /tmp and /var/tmp, crash dumps, logs and the
persistent journal, sssd cached credentials, package caches, the
random seed, DHCP leases and root's shell history. All of it ended up
on every node deployed from the profile.
Drop the contents of scratch, crash, journal, mail spool, sssd cache
and package cache directories, keep only the directory tree under
/var/log and /var/cache/apt since daemons expect their subdirectories
there, and drop a short list of individual files. The directories
themselves are kept, so their mode, ownership and labels survive.
mksquashfs gets the paths as an exclude file, so this applies the same
way whether or not a directory is its own mount.
Hosts without lvm2 installed made dracut report a FAILED dracut-install
during imgutil capture. The initramfs was still built, just without lvm.
lvm is only used there to deactivate stale volume groups before the
install. Any LVM layout is created from inside the image, which carries
its own lvm2 when the source used LVM.
Mark lvm optional, the same way unsquashfs already is.
mksquashfs packs the image and the dracut module needs openssl. Without
them capture failed silently and only the later rsync of private.key
reported an error. Also stop when capturing a filesystem fails, unmount
the capture tmpfs on the target, and have run_constrained print the
exception it used to swallow. squashfs-tools is now required, so drop
the optional prerequisite reporting.
Captured images keep the policy of the source, which lacks the rules
images built with --selinux get. Install them on the source for the
capture and remove them again afterwards, unless the source already
had them.
imgutil build left the downloaded packages and repository metadata in
the image: 1.3 GB of .deb archives plus 145 MB of package lists for
Ubuntu 26.04, about 100 to 160 MB of dnf metadata for EL and 80 MB of
zypp metadata for Leap 16. Untethered nodes copy all of it into memory.
Remove them after building. Installing more packages into the image
then needs a repository refresh first, as usual for images.
setfiles without -F only corrects the type of a file and keeps its user,
role and range. Files of a new build tree get the SELinux user of the
process that created them, so about 1800 of them stayed unconfined_u
instead of system_u. Label the new tree with -F. Pack keeps relabeling
without it, so contexts set on purpose in a captured or edited tree stay.
urlmount is started by the initramfs before the policy is loaded, so it
keeps running as kernel_t. The EL10 targeted policy denies kernel_t
connecting to http_port_t, so after systemd loads the policy every read
of a tethered root image that needs a new connection fails and the node
freezes. imageboot.sh also runs dd from the initramfs later, which is
denied entrypoint. Install a small module with exactly these two rules
in images built with --selinux.
The kernel package writes a boot loader entry inside the build chroot.
Its linux and initrd paths point into the scratch directory of the
server (grub2-mkrelpath sees the bind mount) and its options are the
kernel command line of the server. Diskless boot never reads it, but
after installing to disk it sits next to the entry post.sh writes on
the installed system, so grub may pick the broken one. Remove the
entries when building, post.sh writes the right one when installing.
The _dbpath macro set for fingerprinting an image is process global. It
was only removed after a successful query, so an error left it set and
later fingerprinting in the same process, which fancy_chroot does on
every command, read the wrong database.
Without the rpm Python bindings, fingerprinting an image found its
system-release provider in the image database but then ran rpm -qi
against the host, so an image whose release package the host lacks was
not recognized. Use the same rooted rpm command for both queries.
DebHandler raised SystemExit on a failed apt-get, but prep_root runs
under run_constrained, which only catches Exception and otherwise exits
0. A failed package install was therefore reported as a successful
build. Raise an Exception instead, as ElHandler already does.
EL9 and older keep the rpm database in /var/lib/rpm, EL10 moved it to
/usr/lib/sysimage/rpm. fingerprint_host_el opened the image with the
default path of the host, so on an EL10 server an EL9 image was not
recognized (and an empty database was created in it). pack then wrote
no profile.yaml and no addons link, and import refused the image. Use
/var/lib/rpm when it is a real directory in the image. In EL10 it is a
relative link, so the default of the host still applies.
The SUSE kernel builds it as a module, so dhcpcd failed with "Address
family not supported" and a diskless node with ipv4_method dhcp never
got an address.
imgutil build --selinux installs the targeted policy and keeps SELinux enforcing.
imgutil pack labels images that enable SELinux with their own policy.
Building a SUSE 16 image from SLE media failed every package with
"key ID fec28eaf09d9ea69: NOKEY". Leap publishes that key as
gpg-pubkey-*.asc, which the existing glob picks up; SLE publishes the
same key only as repodata/repomd.xml.key, so nothing was imported.
15 media carries both spellings, so this changes nothing there.
udev's kmod builtin dlopens libkmod, and dracut installs it from an
inst_libdir_file line in a module-setup.sh rather than by following
NEEDED. Which module carries that line moved: the dracut on SLE 16
media declares it only in 00systemd, which the diskless module set
never loads, so the image came up with no libkmod, udev autoloaded
nothing, and the guest reached the network scan with only loopback.
Leap's newer dracut also declares it in 95udev-rules, which base
depends on, which is why Leap was unaffected.