Dropping every NetworkManager profile and ifcfg file also removed
settings that confignet does not recreate from confluent attributes,
such as bonds, DHCP profiles, MTU or the InfiniBand transport mode. Only
drop the profiles that carry static addresses of the source.
dhcpcd, present on Ubuntu 24.04 and later and on EL10, keeps its leases,
DUID and secret in /var/lib/dhcpcd, and NetworkManager keeps a secret in
secret_key. Copied into an image, every clone would use the same DHCPv6
DUID and derive the same stable privacy IPv6 addresses. Both regenerate
them when missing.
dhclient keeps its leases in /var/lib/dhcp there, and on EL also as
*.leases next to the *.lease files already excluded. Keep dhcpd.leases
of a DHCP server, which lives in the same directory.
The image kept /etc/hostname of the source host, so a diskless node ran
under that name until onboot set its own, and early journal and syslog
entries carried the wrong name. With an empty /etc/hostname systemd
keeps the node name the deployment initramfs already set. Install to
disk writes /etc/hostname itself.
Captured images kept the NetworkManager connection profiles and ifcfg
files of the source host. A diskless boot kept them too, and confignet
only rewrites the profiles named like the interfaces it configures, so
every other interface came up with the source host's settings, such as
its static addresses. Install to disk already removes them, and capture
already masks netplan files for the same reason. Drop them from the
image, except ifcfg-lo.
mksquashfs matches excluded paths by inode, so excluding a hard linked
file also dropped its other links elsewhere in the filesystem. Keep such
files in the image. squashfs-tools 4.3 also splits exclude file lines on
whitespace, so pass paths with whitespace or backslashes with -e instead.
Capture copied everything on the local filesystems, including user
files and sockets in /tmp and /var/tmp, crash dumps, logs and the
persistent journal, sssd cached credentials, package caches, the
random seed, DHCP leases and root's shell history. All of it ended up
on every node deployed from the profile.
Drop the contents of scratch, crash, journal, mail spool, sssd cache
and package cache directories, keep only the directory tree under
/var/log and /var/cache/apt since daemons expect their subdirectories
there, and drop a short list of individual files. The directories
themselves are kept, so their mode, ownership and labels survive.
mksquashfs gets the paths as an exclude file, so this applies the same
way whether or not a directory is its own mount.
Mark urlmount as an initramfs root storage daemon before entering FUSE so systemd-shutdown does not wait for or kill the process serving the root image.
imgutil build left the downloaded packages and repository metadata in
the image: 1.3 GB of .deb archives plus 145 MB of package lists for
Ubuntu 26.04, about 100 to 160 MB of dnf metadata for EL and 80 MB of
zypp metadata for Leap 16. Untethered nodes copy all of it into memory.
Remove them after building. Installing more packages into the image
then needs a repository refresh first, as usual for images.
setfiles without -F only corrects the type of a file and keeps its user,
role and range. Files of a new build tree get the SELinux user of the
process that created them, so about 1800 of them stayed unconfined_u
instead of system_u. Label the new tree with -F. Pack keeps relabeling
without it, so contexts set on purpose in a captured or edited tree stay.
imageboot.sh wrote /etc/netplan/10-<iface>-confluentcfg.yaml with the
default mode 0644, and netplan warns that its configuration should not
be accessible by others. confignet already writes its files 0600, but
it leaves an identical file in place, so the boot-time one stayed 0644.
urlmount is started by the initramfs before the policy is loaded, so it
keeps running as kernel_t. The EL10 targeted policy denies kernel_t
connecting to http_port_t, so after systemd loads the policy every read
of a tethered root image that needs a new connection fails and the node
freezes. imageboot.sh also runs dd from the initramfs later, which is
denied entrypoint. Install a small module with exactly these two rules
in images built with --selinux.
extract_untethered_rootimg reports failure, for example when the image
does not fit into the memory backed root, but imageboot.sh ignored it and
booted into the partly extracted root, which then hung later on. Stop
with a message saying why instead.
Without netplan, nmcli or wicked the manager variable was never set and
confignet died with a NameError. Say what is missing instead. Also pick
netplan over nmcli in the retry pass, as the first pass does, instead of
switching to NetworkManager on systems that have both.
The kernel package writes a boot loader entry inside the build chroot.
Its linux and initrd paths point into the scratch directory of the
server (grub2-mkrelpath sees the bind mount) and its options are the
kernel command line of the server. Diskless boot never reads it, but
after installing to disk it sits next to the entry post.sh writes on
the installed system, so grub may pick the broken one. Remove the
entries when building, post.sh writes the right one when installing.
The _dbpath macro set for fingerprinting an image is process global. It
was only removed after a successful query, so an error left it set and
later fingerprinting in the same process, which fancy_chroot does on
every command, read the wrong database.
Without the rpm Python bindings, fingerprinting an image found its
system-release provider in the image database but then ran rpm -qi
against the host, so an image whose release package the host lacks was
not recognized. Use the same rooted rpm command for both queries.
image2disk rewrote root= in the boot loader entries but kept their
linux and initrd paths. Those come from where the image was built: in a
diskless image from imgutil they carry the scratch directory of the
server (grub2-mkrelpath sees its bind mount), and a /boot/ path does not
match the separate /boot filesystem image2disk creates for such images.
grub then fails with "you need to load the kernel". Make the paths
relative to the boot filesystem of the target.
DebHandler raised SystemExit on a failed apt-get, but prep_root runs
under run_constrained, which only catches Exception and otherwise exits
0. A failed package install was therefore reported as a successful
build. Raise an Exception instead, as ElHandler already does.
f9dc92ec fixed this in client.py, but nodeattrib has used asynclient.py
since 76efea7c, so it still sends the underscored name (info_note
instead of info.note) as the attribute, and a missing variable raises a
KeyError instead of a usage error. Port the client.py logic over.
EL9 and older keep the rpm database in /var/lib/rpm, EL10 moved it to
/usr/lib/sysimage/rpm. fingerprint_host_el opened the image with the
default path of the host, so on an EL10 server an EL9 image was not
recognized (and an empty database was created in it). pack then wrote
no profile.yaml and no addons link, and import refused the image. Use
/var/lib/rpm when it is a real directory in the image. In EL10 it is a
relative link, so the default of the host still applies.
An untethered or uncompressed diskless boot downloaded the root image
only from the deployment server and retried it forever when that server
failed. Try the other managers that answered discovery as well, as the
tethered urlmount already does.
urlmount limits each small range request to 10 seconds, but the single
whole-image download cannot take a total limit. Give up on a connect
after 10 seconds and on a transfer that stalls for 30 seconds instead,
so a hung server moves on to the next one.
Write the nightly release notes only after every bundle is uploaded.
The notes carry the build key that prepare compares against, so a
failed upload used to record the new build anyway, and the following
nights skipped it while the release kept stale or missing bundles.
Publish only from the default branch. A manual run defaulted to
publishing, so testing the workflow on a topic branch replaced the
nightly with that branch.
Run the packaging build on pull requests that touch MANIFEST.in, which
decides what goes into the client and server packages, or anything
under confluent_vtbufferd, which is compiled here and carries the only
debian/ packaging in the tree.
Give every job a timeout instead of the six hour default, so a hung
container or download fails within minutes.
The initramfs writes the overlay upper layer before the SELinux policy
is loaded, so those files are unlabeled_t. On an enforcing image /
itself is unlabeled, and logind, getty and sshd fail.
Label exactly the upper layer paths like el9-diskless does. The root
is an overlay in untethered mode too, so this covers both. chcon comes
from the image, the initramfs has none.
Without it the server is only found by link-local discovery, so nodes
behind a DHCP relay never boot. Ported from el9-diskless. The initramfs
has dhcpcd from 24.04 on and only ipconfig before.
Markus Hilger
Jarrod Johnson