gather propagates the first exception and leaves its siblings running,
so a transport level failure against one node ends the import with a
traceback while the rest of the batch is cancelled at loop shutdown.
The forked children used to contain such a failure to their own node.
assign_macs already reports an error response itself, so this is the
connection dropping rather than the server refusing the assignment.
Collect the exceptions instead, report each one and count it towards
the exit code. Schedule the assignments as tasks while doing so, since
the plain coroutines are left unawaited if defining a later node raises
before the gather is reached.
Replacing the forked children with a gather kept their fan-out: every
row of the import file gets a session of its own and they all start at
once, so a large file opens a local socket and a server side session
task per node simultaneously.
Hold a semaphore for the duration of each node's assignment instead, so
a finished node's session is dropped before the next one starts. Also
build that session once per node rather than once per MAC, and say why
the caller's session is not reused, which was self evident while this
ran in a forked child.
register_endpoint primes current but not total, so a first response
without a count field goes straight to
UnboundLocalError: local variable 'total' referenced before assignment
on the elif. Start at zero, which skips the progress line until the
server does report a count.
nodediscover's rescan poll and nodeconsole's screenshot refresh both slept
with time.sleep inside a coroutine. In nodeconsole --video that stops the
input handler and the VNC streaming tasks for the whole interval.
_connect_unix left the socket blocking, while _connect_tls sets a zero
timeout, so every loop.sock_recv and sock_sendall against the local
socket ran the blocking call inline and stalled the whole event loop.
nodeconsole --video showed this most clearly: a power action opens its
own session, so the tiles stopped refreshing and keystrokes went
unhandled until the BMC finished.
asyncio only enforces this in debug mode, where the client failed
outright with ValueError: the socket must be non-blocking. The
descriptor passing retries in asynctlvdata also assume a non-blocking
socket, since they wait for BlockingIOError.
simple_nodegroups_command awaited the async generators returned by read
and update, which raises
TypeError: 'async_generator' object can't be awaited
and printgroupattributes was left synchronous, iterating one of those
generators with plain for. Neither is reachable yet, since nodeattrib
only ever passes a noderange and nodegroupattrib still uses the
traditional client, but they are the paths nodegroupattrib will use once
it is ported.
When sendmsg() reports EAGAIN, _sendmsg rescheduled itself with
loop.add_reader(fd, _sendmsg, loop, fut, sock, fd)
which waits for the socket to become readable rather than writable, and
passes four of the six required arguments, so the callback raised
TypeError once it did fire. Wait for writability and pass the message
and descriptors through.
Also skip the work in _recvmsg if the future was cancelled while waiting
for data, as _sendmsg already does, so a cancelled read does not end in
InvalidStateError from set_result.
This module is imported by the server as well, so both paths are reached
by the daemon whenever a descriptor is passed over the local socket.
import_csv was left with several synchronous idioms:
- search_record is a coroutine function, but was called without await.
The returned coroutine is always truthy, so the rescan on incomplete
discovery data never happened, and iterating the result raised
TypeError: 'coroutine' object is not iterable
- the node creation loop iterated an async generator with plain for
- the per-node discovery assignment was forked off with os.fork() while
the event loop was running, and the child then built a fresh session
on the inherited selector
Assign discovery entries with asyncio.gather instead of a forked child,
which keeps the assignments concurrent and lets their exit codes
propagate. The forked child always ended in sys.exit(0), so its
accumulated errorcode was discarded.
register_endpoint and subscribe_discovery were left as plain functions
iterating the async client generators, so nodediscover register,
subscribe and unsubscribe all failed immediately with
TypeError: 'async_generator' object is not iterable
Unfortunately, the problem of urlmount's selinux context is left open.
urlmount starts before policy load, preventing transition.
However the policy blocks access urlmount needs when loaded.
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.
Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization). Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
BUILDSRC is only set if imgutil build is run with --source, otherwise
the build host repos are used. If --source is not used, there is no
distribution symlink and add_local_repositores failed with 404.
Check if BUILDSRC is set and skip add_local_repositories if this is the
case.
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.
Without this fix the onboot services failed to start on SELinux enabled
captured image.
If LLDP is uncooperative, maybe the mac was learned.
If no mac apparently learned, then we ping_everywhere in hopes of soliciting traffic, and then rescan the switches.
Then get all mac addresses, try to determine zone from generated mac, and print on success.
The following post install steps were missing on Ubuntu builds:
- Permission fixes
- sysctl load
- Service restart
- confluent PAM symlink to /etc/pam.d/sshd. It works without it on
Ubuntu because it falls back to other which allows login on Ubuntu,
but the behaviour should be the same on every OS. Furtheremore, an
admin might implement additional steps to sshd PAM and would like to
have this in Confluent, too
The CA database under /etc/confluent/tls/ca is typically created by a root context such as osdeploy initialize -t, but the confluent service runs as the owner of /etc/confluent, and openssl ca rewrites the database (index, serial) as the invoking user on every issuance. Certificate issuance through the running service (e.g. the /self/tlscert deployment API) then fails on the root-owned database until packaging happens to repair the ownership.
Run the CA creation (full CA and the currently unused simple CA variant) and the openssl ca invocation under normalize_uid, the convention already used when publishing the CA certificate. The issued certificate is staged through a temporary file since the destination may only be writable by the invoking user, e.g. the web server certificate paths.
Existing root-owned CA databases are repaired by packaging or manually via: chown -R --reference=/etc/confluent /etc/confluent/tls
Markus Hilger
Jarrod Johnson
Jarrod Johnson