osimport polls import progress from a coroutine, so a blocking sleep
between reads stalls the whole client loop. It was the only use of time
in the script, so the import goes with it.
Both loops that read the importer's output test for a percentage first,
so an ERROR: line whose text carries a % takes the percentage branch and
float() raises instead of the error being reported. The import target
name can carry one too, and that one is user supplied. importmedia runs
as a bare task, so the exception is swallowed and the client polls a
phase that never advances.
Test for ERROR: first and treat an unparsable percentage as no
percentage. Set percent on the error path of the second loop as well,
as the first already does.
The loop that drains the importer's remaining output reads a byte at a
time but clears currline on every iteration, one level out from where
the earlier loop clears it. currline is therefore never longer than a
single byte, so the percentage and ERROR: branches can never match and
the tail of an import is silently discarded.
Clear it only once a line has been consumed, as the earlier loop does.
scan_iso walks an entire ISO with blocking libarchive reads, yielding
only once per entry, and the header-sum branch of fingerprint reads the
whole file with no yield at all. Both run in the daemon, reached from
MediaImporter.init on every fingerprint and importing request.
The scan costs about 8us per entry and is indifferent to media size,
since libarchive seeks past file data rather than reading it: measured
at 80ms for 10k entries whether the image is 0.2 GB or 8.8 GB, and at
310ms for 40k. The header-sum branch is the one that scales with size,
reading a multi-gigabyte image end to end.
Make the pair plain functions and hand them to a thread instead.
check_output unwraps a single list argument, check_call never did, so
callers passing a list hit a TypeError out of create_subprocess_exec.
Two callers do: the genisoimage run behind Windows profile imports,
where an except Exception swallows the failure and the boot.iso is
silently missing, and the nodeconfig run in discovery, which takes out
automatic node configuration on discovery outright.
The asyncio port added an await between every 2048 byte read, which
roughly doubled the cost of hashing. imgutil runs entire packed images
through this, and the server pays it on rebase and media import.
Read a megabyte per iteration instead. That still yields hundreds of
times per gigabyte, so the event loop stays responsive, and sha512 is
independent of the read size, so existing manifests remain valid.
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.
Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization). Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
The following post install steps were missing on Ubuntu builds:
- Permission fixes
- sysctl load
- Service restart
- confluent PAM symlink to /etc/pam.d/sshd. It works without it on
Ubuntu because it falls back to other which allows login on Ubuntu,
but the behaviour should be the same on every OS. Furtheremore, an
admin might implement additional steps to sshd PAM and would like to
have this in Confluent, too
The CA database under /etc/confluent/tls/ca is typically created by a root context such as osdeploy initialize -t, but the confluent service runs as the owner of /etc/confluent, and openssl ca rewrites the database (index, serial) as the invoking user on every issuance. Certificate issuance through the running service (e.g. the /self/tlscert deployment API) then fails on the root-owned database until packaging happens to repair the ownership.
Run the CA creation (full CA and the currently unused simple CA variant) and the openssl ca invocation under normalize_uid, the convention already used when publishing the CA certificate. The issued certificate is staged through a temporary file since the destination may only be writable by the invoking user, e.g. the web server certificate paths.
Existing root-owned CA databases are repaired by packaging or manually via: chown -R --reference=/etc/confluent /etc/confluent/tls
Apply ruff's safe autofixes.
The changes are mechanical and behaviour-preserving. Issues fixed:
- F401: remove unused imports.
- F841: drop unused local variables and assignments, including discarded
await/return values, unused "except ... as e" bindings, and unused
"with ... as name" targets.
- F541: remove the f prefix from f-strings that contain no placeholders.
- E711: compare against None with "is"/"is not" instead of "=="/"!=".
- E712: test truthiness directly instead of comparing to True.
- E713: use "x not in y" instead of "not x in y".
- E714: use "is not" instead of "not ... is".
- E731: convert lambdas bound to a name into def statements.
- W291/W293: trim trailing whitespace on touched lines.
Before the async port the bmc var was used for a "host=bmc" parameter
which does not exist anymore.
Now add [] around IPv6 addresses with missing brackets but keep the scope zone like %eth0
as this is needed in current code.
The IPMI 1.5 session-challenge callback returned coroutine objects from error reporting and session activation instead of expressing an asynchronous callback contract directly. That made completion depend on the dispatch path noticing and awaiting the returned object.
Make the callback asynchronous and explicitly await both onlogon() and _activate_session(), ensuring failure notification and activation finish before callback dispatch continues.
Markus Hilger
Jarrod Johnson
Jarrod Johnson