The copy ran after the cd to the repo root, so it read ../LICENSE from
outside the checkout and never placed the file in confluent_osdeploy/. The
tarball went out without it and the spec's %install, which does
"cp LICENSE" after %setup cds into the unpacked directory, failed.
imgutil/buildrpm already copies before its cd; do the same here.
The aarch64 spec has its LICENSE lines commented out, so only the x86_64
build broke, but the copy was equally wrong in both scripts.
Release tags do not live on master: 3.15.2 through 3.15.6 were tagged on branch
3.15, so git describe reaches only 3.15.1 and dev builds were stamped
3.15.2.dev<n>. Besides being confusing, rpm and dpkg both rank the released
3.15.6 above that, so a dev package will not install over a released one.
Add a top-level VERSION file naming the release the branch is working toward
(4.0.0 on master) and a mkversion helper that stamps packages from it, keeping
the tag-derived value as a floor so a forgotten bump cannot go backwards.
mkversion also replaces the block copy-pasted into seven build scripts, and
makesetup no longer writes a per-package VERSION file, so the stale checked-in
confluent_common/VERSION goes with it.
gather propagates the first exception and leaves its siblings running,
so a transport level failure against one node ends the import with a
traceback while the rest of the batch is cancelled at loop shutdown.
The forked children used to contain such a failure to their own node.
assign_macs already reports an error response itself, so this is the
connection dropping rather than the server refusing the assignment.
Collect the exceptions instead, report each one and count it towards
the exit code. Schedule the assignments as tasks while doing so, since
the plain coroutines are left unawaited if defining a later node raises
before the gather is reached.
Replacing the forked children with a gather kept their fan-out: every
row of the import file gets a session of its own and they all start at
once, so a large file opens a local socket and a server side session
task per node simultaneously.
Hold a semaphore for the duration of each node's assignment instead, so
a finished node's session is dropped before the next one starts. Also
build that session once per node rather than once per MAC, and say why
the caller's session is not reused, which was self evident while this
ran in a forked child.
register_endpoint primes current but not total, so a first response
without a count field goes straight to
UnboundLocalError: local variable 'total' referenced before assignment
on the elif. Start at zero, which skips the progress line until the
server does report a count.
nodediscover's rescan poll and nodeconsole's screenshot refresh both slept
with time.sleep inside a coroutine. In nodeconsole --video that stops the
input handler and the VNC streaming tasks for the whole interval.
_connect_unix left the socket blocking, while _connect_tls sets a zero
timeout, so every loop.sock_recv and sock_sendall against the local
socket ran the blocking call inline and stalled the whole event loop.
nodeconsole --video showed this most clearly: a power action opens its
own session, so the tiles stopped refreshing and keystrokes went
unhandled until the BMC finished.
asyncio only enforces this in debug mode, where the client failed
outright with ValueError: the socket must be non-blocking. The
descriptor passing retries in asynctlvdata also assume a non-blocking
socket, since they wait for BlockingIOError.
simple_nodegroups_command awaited the async generators returned by read
and update, which raises
TypeError: 'async_generator' object can't be awaited
and printgroupattributes was left synchronous, iterating one of those
generators with plain for. Neither is reachable yet, since nodeattrib
only ever passes a noderange and nodegroupattrib still uses the
traditional client, but they are the paths nodegroupattrib will use once
it is ported.
When sendmsg() reports EAGAIN, _sendmsg rescheduled itself with
loop.add_reader(fd, _sendmsg, loop, fut, sock, fd)
which waits for the socket to become readable rather than writable, and
passes four of the six required arguments, so the callback raised
TypeError once it did fire. Wait for writability and pass the message
and descriptors through.
Also skip the work in _recvmsg if the future was cancelled while waiting
for data, as _sendmsg already does, so a cancelled read does not end in
InvalidStateError from set_result.
This module is imported by the server as well, so both paths are reached
by the daemon whenever a descriptor is passed over the local socket.
import_csv was left with several synchronous idioms:
- search_record is a coroutine function, but was called without await.
The returned coroutine is always truthy, so the rescan on incomplete
discovery data never happened, and iterating the result raised
TypeError: 'coroutine' object is not iterable
- the node creation loop iterated an async generator with plain for
- the per-node discovery assignment was forked off with os.fork() while
the event loop was running, and the child then built a fresh session
on the inherited selector
Assign discovery entries with asyncio.gather instead of a forked child,
which keeps the assignments concurrent and lets their exit codes
propagate. The forked child always ended in sys.exit(0), so its
accumulated errorcode was discarded.
register_endpoint and subscribe_discovery were left as plain functions
iterating the async client generators, so nodediscover register,
subscribe and unsubscribe all failed immediately with
TypeError: 'async_generator' object is not iterable
A node imported by a merge was assigned a free index and then had it
overwritten by the index carried in the backup, which may already belong
to a node in the target database. Keep the allocated index instead; a
full restore still honors the dumped index.
The -x/--exclude option drops matching node and node group attributes
from a dump, restore, or merge, so a backup can leave out dynamic state
such as deployment.state_last_updated or data that should not travel with it.
Patterns use shell-style wildcards, and a bare namespace such as net
excludes every attribute below it. The node "groups" and "id.index"
attributes and the node group "noderange" attribute are always retained
so that a restore can still reconstruct group membership and node index
assignments.
Unfortunately, the problem of urlmount's selinux context is left open.
urlmount starts before policy load, preventing transition.
However the policy blocks access urlmount needs when loaded.
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.
Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization). Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
BUILDSRC is only set if imgutil build is run with --source, otherwise
the build host repos are used. If --source is not used, there is no
distribution symlink and add_local_repositores failed with 404.
Check if BUILDSRC is set and skip add_local_repositories if this is the
case.
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.
Without this fix the onboot services failed to start on SELinux enabled
captured image.
Jarrod Johnson
Markus Hilger
Jarrod Johnson