Replacing the forked children with a gather kept their fan-out: every
row of the import file gets a session of its own and they all start at
once, so a large file opens a local socket and a server side session
task per node simultaneously.
Hold a semaphore for the duration of each node's assignment instead, so
a finished node's session is dropped before the next one starts. Also
build that session once per node rather than once per MAC, and say why
the caller's session is not reused, which was self evident while this
ran in a forked child.
register_endpoint primes current but not total, so a first response
without a count field goes straight to
UnboundLocalError: local variable 'total' referenced before assignment
on the elif. Start at zero, which skips the progress line until the
server does report a count.
nodediscover's rescan poll and nodeconsole's screenshot refresh both slept
with time.sleep inside a coroutine. In nodeconsole --video that stops the
input handler and the VNC streaming tasks for the whole interval.
_connect_unix left the socket blocking, while _connect_tls sets a zero
timeout, so every loop.sock_recv and sock_sendall against the local
socket ran the blocking call inline and stalled the whole event loop.
nodeconsole --video showed this most clearly: a power action opens its
own session, so the tiles stopped refreshing and keystrokes went
unhandled until the BMC finished.
asyncio only enforces this in debug mode, where the client failed
outright with ValueError: the socket must be non-blocking. The
descriptor passing retries in asynctlvdata also assume a non-blocking
socket, since they wait for BlockingIOError.
simple_nodegroups_command awaited the async generators returned by read
and update, which raises
TypeError: 'async_generator' object can't be awaited
and printgroupattributes was left synchronous, iterating one of those
generators with plain for. Neither is reachable yet, since nodeattrib
only ever passes a noderange and nodegroupattrib still uses the
traditional client, but they are the paths nodegroupattrib will use once
it is ported.
When sendmsg() reports EAGAIN, _sendmsg rescheduled itself with
loop.add_reader(fd, _sendmsg, loop, fut, sock, fd)
which waits for the socket to become readable rather than writable, and
passes four of the six required arguments, so the callback raised
TypeError once it did fire. Wait for writability and pass the message
and descriptors through.
Also skip the work in _recvmsg if the future was cancelled while waiting
for data, as _sendmsg already does, so a cancelled read does not end in
InvalidStateError from set_result.
This module is imported by the server as well, so both paths are reached
by the daemon whenever a descriptor is passed over the local socket.
import_csv was left with several synchronous idioms:
- search_record is a coroutine function, but was called without await.
The returned coroutine is always truthy, so the rescan on incomplete
discovery data never happened, and iterating the result raised
TypeError: 'coroutine' object is not iterable
- the node creation loop iterated an async generator with plain for
- the per-node discovery assignment was forked off with os.fork() while
the event loop was running, and the child then built a fresh session
on the inherited selector
Assign discovery entries with asyncio.gather instead of a forked child,
which keeps the assignments concurrent and lets their exit codes
propagate. The forked child always ended in sys.exit(0), so its
accumulated errorcode was discarded.
register_endpoint and subscribe_discovery were left as plain functions
iterating the async client generators, so nodediscover register,
subscribe and unsubscribe all failed immediately with
TypeError: 'async_generator' object is not iterable
Apply ruff's safe autofixes.
The changes are mechanical and behaviour-preserving. Issues fixed:
- F401: remove unused imports.
- F841: drop unused local variables and assignments, including discarded
await/return values, unused "except ... as e" bindings, and unused
"with ... as name" targets.
- F541: remove the f prefix from f-strings that contain no placeholders.
- E711: compare against None with "is"/"is not" instead of "=="/"!=".
- E712: test truthiness directly instead of comparing to True.
- E713: use "x not in y" instead of "not x in y".
- E714: use "is not" instead of "not ... is".
- E731: convert lambdas bound to a name into def statements.
- W291/W293: trim trailing whitespace on touched lines.
Fix SC2045 (error): Iterating over ls output is fragile. Use globs.
Add exception SC2068 exception for confluent_client/confluent_env.sh as this is intended.
SC2068 (error): Double quote array expansions to avoid re-splitting elements.
Allow arbitrary per-connection network settings, such as static routes
or a firewalld zone, to be specified as semicolon-delimited key=value
pairs on a net.*.extra_settings attribute. The keys are passed through
to the network backend of the deployed OS in its native syntax: nmcli
properties on NetworkManager systems, netplan YAML paths on netplan
systems, and ifcfg variables on wicked systems.
Neither tool detected when the attribute database produces conflicting
name/IP data, silently emitting the conflicts.
confluent2hosts now warns when the same hostname is generated for
multiple different addresses within one address family (dual-stack
IPv4+IPv6 pairs stay silent), which happens naturally in -a mode when a
node has several networks without distinct per-net hostnames.
confluent2dnsmasq now warns when generated reservations share a
hostname across different IPs, reserve the same IP more than once
(dnsmasq refuses to start on a duplicate dhcp-host IP), or reuse a MAC.
Allow building EL and Ubuntu diskless images for a foreign architecture (e.g.
aarch64 on an x86_64 host) by leveraging qemu-user-static. The target
architecture is detected automatically from a -s source tree (for EL),
or may be requested explicitly with the new --arch option.
When the target differs from the host, dnf/debootstrap is invoked with
--forcearch/--arch and the presence of an enabled binfmt_misc handler
with the F (fix-binary) flag is verified up front, so emulation keeps working
inside the installroot chroot and a missing setup yields an actionable
error instead of a confusing exec failure mid-build.
The image architecture is recorded in confluentimg.buildinfo so that
pack selects the initramfs addons for the image architecture rather
than the build host, and exec of a foreign-arch root performs the same
binfmt check.
Adds `confluentdbutil showattrib <noderange> <attribute>...` to print the
node attribute.
In contrast to nodeattrib it can shows secrets and crypted values with -u flag.
It's server-side only: reads the config store and master key directly, never over
the API.
It's read-only and works without confluentd running.
These commands ship in confluent_client/bin but had no .ronn man page, so
they did not appear in the generated documentation. Add man pages matching
the existing style, with synopsis and options taken from each command's
argument parser:
- confluent2ansible: export node inventory to an Ansible hosts file
- confluent2lxca: export nodes to a Lenovo XClarity Administrator bulk import CSV
- confluent2xcat: export nodes to an xCAT stanza definition (and optional macs.csv)
- dir2img: build a FAT image from a directory for nodemedia upload
- nodecertutil: manage BMC CA certificates and sign BMC certificates
- nodegrouprename: rename a node group
- noderename: rename nodes
Ensure all input order is preserved as it is processed.
Institute a 10ms delay after a key transmit. This is to prevent overwhelming QEMU console, and improve it for other consoles.
Tested by pasting a large volume of text and seeing that it was intact.
Markus Hilger
Jarrod Johnson
Jarrod Johnson
Laurence