Nothing unwound the loop device and dm-crypt mapping when the copy loop
raised, so interrupting a pack stranded both, still holding the profile's
rootimg.sfs.
Tear them down from a finally. The retry loop moves with them, so also
honour its tries counter, as unpack_image already does; spinning forever
inside a finally would hang the interrupt it is meant to clean up after.
A bounded retry loop can also give up, and the detach that follows would
then fail with EBUSY and, raising from a finally, replace the exception
that brought us here. Warn and leave both in place instead.
Both functions became coroutines solely to await one get_hashes call,
but their bodies are long stretches of blocking work: mksquashfs, the
encrypt_image copy loop, rsync, ssh and osdeploy.
From Python 3.11 on, asyncio.run installs a SIGINT handler that cancels
the main task and returns rather than raising, so an interrupt is only
noticed at the next await. Interrupting a pack during mksquashfs
surfaced as a CalledProcessError from the dying child instead of a
KeyboardInterrupt, and with a base profile, where nothing is ever
awaited, pack carried on and published the profile before exiting.
Run the loop only around the call that needs it.
check_output unwraps a single list argument, check_call never did, so
callers passing a list hit a TypeError out of create_subprocess_exec.
Two callers do: the genisoimage run behind Windows profile imports,
where an except Exception swallows the failure and the boot.iso is
silently missing, and the nodeconfig run in discovery, which takes out
automatic node configuration on discovery outright.
capture and pack hash the whole profile directory, which by that point
holds rootimg.sfs, the kernel and the distribution initramfs. rebase
only ever looks up entries that came from the profile source directory,
so the image blobs cost gigabytes of hashing for nothing.
Pass the source directory as the filter, as generate_stock_profiles
already does. Older manifests keep working, since rebase reads their
entries with a default.
The asyncio port added an await between every 2048 byte read, which
roughly doubled the cost of hashing. imgutil runs entire packed images
through this, and the server pays it on rebase and media import.
Read a megabyte per iteration instead. That still yields hundreds of
times per gigabyte, so the event loop stays responsive, and sha512 is
independent of the read size, so existing manifests remain valid.
Unfortunately, the problem of urlmount's selinux context is left open.
urlmount starts before policy load, preventing transition.
However the policy blocks access urlmount needs when loaded.
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.
Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization). Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
BUILDSRC is only set if imgutil build is run with --source, otherwise
the build host repos are used. If --source is not used, there is no
distribution symlink and add_local_repositores failed with 404.
Check if BUILDSRC is set and skip add_local_repositories if this is the
case.
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.
Without this fix the onboot services failed to start on SELinux enabled
captured image.
If LLDP is uncooperative, maybe the mac was learned.
If no mac apparently learned, then we ping_everywhere in hopes of soliciting traffic, and then rescan the switches.
Then get all mac addresses, try to determine zone from generated mac, and print on success.
The following post install steps were missing on Ubuntu builds:
- Permission fixes
- sysctl load
- Service restart
- confluent PAM symlink to /etc/pam.d/sshd. It works without it on
Ubuntu because it falls back to other which allows login on Ubuntu,
but the behaviour should be the same on every OS. Furtheremore, an
admin might implement additional steps to sshd PAM and would like to
have this in Confluent, too
The CA database under /etc/confluent/tls/ca is typically created by a root context such as osdeploy initialize -t, but the confluent service runs as the owner of /etc/confluent, and openssl ca rewrites the database (index, serial) as the invoking user on every issuance. Certificate issuance through the running service (e.g. the /self/tlscert deployment API) then fails on the root-owned database until packaging happens to repair the ownership.
Run the CA creation (full CA and the currently unused simple CA variant) and the openssl ca invocation under normalize_uid, the convention already used when publishing the CA certificate. The issued certificate is staged through a temporary file since the destination may only be writable by the invoking user, e.g. the web server certificate paths.
Existing root-owned CA databases are repaired by packaging or manually via: chown -R --reference=/etc/confluent /etc/confluent/tls
Markus Hilger
Jarrod Johnson
Jarrod Johnson