2
0
mirror of https://github.com/xcat2/confluent.git synced 2026-08-03 16:07:00 +00:00
Commit Graph

3186 Commits

Author SHA1 Message Date
Markus Hilger 6e6cbce0a2 Do not report an interrupted firmware update as complete
The retry counter is there to ride out a few unanswered polls, but
exhausting it broke out of the loop with complete still unset and fell
through to the 'complete' return, so an SMM that stopped answering
part way through an apply was reported to the operator as updated.
Raise instead; a genuine finish still leaves the loop on the progress
reaching 100.
2026-07-27 04:56:32 +02:00
Markus Hilger 5135a6cd3d Make the SMM web session cache safe to share
Now that the expiry comparison actually caches a client, the session it
holds is shared, so tearing it down and replacing it needs the same care
the IMM handler already takes:

Dispose of an expired session with a logout instead of dropping the
reference, otherwise every refresh leaves an authenticated session
behind on an SMM that only has a handful of slots.  That logout has to
tolerate a session the SMM has already reaped, hence the except.

Guard the login itself, so two coroutines arriving at an empty or
expired cache do not both log in and orphan one of the two sessions.

Stamp the vintage once the login round trips are done rather than
before, so a slow SMM cannot hand back a client that is already expired.
2026-07-27 04:56:32 +02:00
Markus Hilger ca81907d25 Restore SMM web request semantics lost in the async port
The old WebConnection.request() added a
'Content-Type: application/x-www-form-urlencoded' header to any POST
carrying a body, but grab_response_with_status() only sets a content
type for dict payloads, so the SMM login and every /data form POST now
go out as text/plain.  This is not a fix for an observed failure: an SMM
running FPC variant 38 was measured accepting a text/plain login exactly
as readily as a urlencoded one.  It restores the header the synchronous
code always sent and that the TSM and IMM handlers still set explicitly,
rather than relying on every SMM firmware level being equally lax about
what it will parse.

Also stop hard failing on responses the synchronous code discarded on
purpose.  'set=securityrollback:1' is only understood by newer SMM2
firmware.  And /data/logout answers 401 once the session is gone, as
measured on that same SMM, so raising on a non-200 there turns a
completed hostname, domain or NTP operation into a spurious error.
2026-07-27 04:55:57 +02:00
Markus Hilger 8817ee6deb Await NextScale SMM settings operations
The SMM hostname, domain, and NTP helpers looked synchronous even though their web transport is asynchronous. Removing awaits in the Lenovo OEM handler therefore returned unresolved coroutine work instead of completed settings results.

Convert the SMM settings and logout helpers to the asynchronous web interface, validate HTTP status responses, and await each operation from the OEM handler so callers only observe completed results.
2026-07-27 04:54:44 +02:00
Markus Hilger fbbeda6c86 Fix NextScale asynchronous web client
The NextScale SMM path still used the removed http.client-style interface against the asynchronous WebConnection implementation. Login, configuration, diagnostic, and firmware operations consequently called unavailable methods or left request coroutines unresolved.

Make web-client creation asynchronous, migrate the affected requests to grab_response_with_status(), and await the cached client accessor. Correct the cache expiry comparison so fresh authenticated clients are reused and stale clients are renewed.
2026-07-27 04:54:08 +02:00
Jarrod Johnson a65583c325 Clean up some headers missed in the rebase to aio http 2026-07-24 12:38:33 -04:00
Jarrod Johnson d81ab1d239 Remove microseconds from the last updated timestamp 2026-07-24 09:30:42 -04:00
Jarrod Johnson 43c98552f4 Change to use standard iso format 2026-07-24 09:12:17 -04:00
Jarrod Johnson 2fed3ddb57 Add timestamp to booted information on diskless boot
It can be ambiguous if the node booted recently or not.
2026-07-24 08:46:47 -04:00
Jarrod Johnson 538a51310b Allow confluentdbutil to operate on an alternate directory 2026-07-24 08:12:02 -04:00
Jarrod Johnson 933354f454 Merge pull request #251 from Obihoernchen/yaml-fixes
Harden and clean up config dump/restore
2026-07-20 14:29:55 -04:00
Markus Hilger b5f54c9382 Fix tenant enumeration path in dump_db_to_directory
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
2026-07-20 19:18:08 +02:00
Markus Hilger 93a6c535b2 Clean up YAML dump/restore code
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
2026-07-20 18:03:42 +02:00
Markus Hilger 9101b07d54 Report stateless boot completion via new 'booted' status
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.

Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
2026-07-20 15:59:30 +02:00
Markus Hilger d415fcd97f Restrict YAML implicit typing on restore
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization).  Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
2026-07-18 01:26:14 +02:00
Markus Hilger a8b443dc93 Provide clearer error on restore with mismatched dump format
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
2026-07-17 23:56:05 +02:00
Jarrod Johnson 5db32996f9 Provide cleaner error on requesting non-existant tenant. 2026-07-16 17:55:22 -04:00
Jarrod Johnson 8b0fb03c66 Disable implicit tenant creation
If we support more tenants, we will modify that branch.
2026-07-16 12:39:22 -04:00
Jarrod Johnson ea9be4aae0 Merge pull request #239 from Obihoernchen/ci
Add CI pipeline
2026-07-15 13:54:15 -04:00
Jarrod Johnson 2644ad861d Merge pull request #244 from Obihoernchen/caperm
TLS CA permission fixes
2026-07-15 13:51:47 -04:00
Markus Hilger 63a0cd237f Add missing postinst steps to Ubuntu
The following post install steps were missing on Ubuntu builds:

- Permission fixes
- sysctl load
- Service restart
- confluent PAM symlink to /etc/pam.d/sshd. It works without it on
  Ubuntu because it falls back to other which allows login on Ubuntu,
  but the behaviour should be the same on every OS. Furtheremore, an
  admin might implement additional steps to sshd PAM and would like to
  have this in Confluent, too
2026-07-15 06:02:03 +02:00
Markus Hilger 86d90281e0 Speed up find
Spawn just one find process and stop on first hit
2026-07-15 05:55:48 +02:00
Markus Hilger 4b61351373 Create and maintain the TLS CA as the confluent service account
The CA database under /etc/confluent/tls/ca is typically created by a root context such as osdeploy initialize -t, but the confluent service runs as the owner of /etc/confluent, and openssl ca rewrites the database (index, serial) as the invoking user on every issuance. Certificate issuance through the running service (e.g. the /self/tlscert deployment API) then fails on the root-owned database until packaging happens to repair the ownership.

Run the CA creation (full CA and the currently unused simple CA variant) and the openssl ca invocation under normalize_uid, the convention already used when publishing the CA certificate. The issued certificate is staged through a temporary file since the destination may only be writable by the invoking user, e.g. the web server certificate paths.

Existing root-owned CA databases are repaired by packaging or manually via: chown -R --reference=/etc/confluent /etc/confluent/tls
2026-07-15 05:55:48 +02:00
Markus Hilger fb41537555 Increase net.core.rmem_max to 4 MB
This was introduced 8 years ago. Newer OSs have 4194304 as default.
Confluent shouldn't decrease the default.

RHEL 8           212992
RHEL 9          2097152
RHEL 10         2097152
Fedora 44       4194304

Ubuntu 24.04     212992
Ubuntu 26.06    4194304

SLES 15          212992
SLES 16          212992

It was changed to 4MB in upstream kernel, too:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a6d4f25888b83b8300aef28d9ee22765c1cc9b34
2026-07-15 03:26:02 +02:00
Markus Hilger fa605160e0 Merge branch 'master' into ci 2026-07-14 17:04:16 +02:00
Jarrod Johnson 07df9a8c94 Ensure prefix is a string 2026-07-14 11:02:15 -04:00
Jarrod Johnson 5cfc3f07af Fix nodemedia attach
The hardening blocked all URL patters.
2026-07-14 11:02:08 -04:00
Markus Hilger 06c7352394 Log exceptions do not swallow it 2026-07-14 17:01:33 +02:00
Jarrod Johnson 0263ad0914 Merge pull request #241 from Obihoernchen/ipv6fix
Preserve scoped IPv6 console addresses
2026-07-14 07:55:03 -04:00
Jarrod Johnson 065426b161 Remove derelict devnull open
This was leftover from pre-async days to support old subprocess
mechanism.
2026-07-14 07:51:47 -04:00
Jarrod Johnson 2b1facb2c5 Merge pull request #240 from Obihoernchen/enos
Reuse ENOS health data
2026-07-14 07:50:56 -04:00
Markus Hilger ab6eeb3ced Merge branch 'master' into ruff 2026-07-14 05:28:53 +02:00
Markus Hilger 2af402b13c ruff auto fixes
Apply ruff's safe autofixes.
The changes are mechanical and behaviour-preserving. Issues fixed:

- F401: remove unused imports.
- F841: drop unused local variables and assignments, including discarded
  await/return values, unused "except ... as e" bindings, and unused
  "with ... as name" targets.
- F541: remove the f prefix from f-strings that contain no placeholders.
- E711: compare against None with "is"/"is not" instead of "=="/"!=".
- E712: test truthiness directly instead of comparing to True.
- E713: use "x not in y" instead of "not x in y".
- E714: use "is not" instead of "not ... is".
- E731: convert lambdas bound to a name into def statements.
- W291/W293: trim trailing whitespace on touched lines.
2026-07-14 05:03:58 +02:00
Markus Hilger b8739b1feb Preserve scoped IPv6 console addresses
Before the async port the bmc var was used for a "host=bmc" parameter
which does not exist anymore.

Now add [] around IPv6 addresses with missing brackets but keep the scope zone like %eth0
as this is needed in current code.
2026-07-14 04:00:45 +02:00
Markus Hilger 686fef6730 Reuse ENOS health data 2026-07-14 03:49:55 +02:00
Markus Hilger dab2f0bb02 Move returns out of finally blocks
Fixes python compile warning:

SyntaxError: 'return' in a 'finally' block
2026-07-14 01:30:16 +02:00
Markus Hilger 69a6714108 Use raw string notation to fix python compile warnings
E.g.: SyntaxError: "\d" is an invalid escape sequence. Did you mean "\\d"? A raw string is also an option.
2026-07-14 01:26:19 +02:00
Jarrod Johnson 090a887ed9 Merge pull request #235 from Obihoernchen/aiohmi
Various asyncio fixes
2026-07-13 13:26:07 -04:00
Markus Hilger 44d2534b4b Revert changes flagged by review
Restore pre-PR behavior for three changes flagged by @jjohnson42.
Bigger changes are needed for these. Will be done in a separate PR.
2026-07-13 17:25:58 +02:00
Markus Hilger dc906b9cd0 Await IPMI session challenge callbacks
The IPMI 1.5 session-challenge callback returned coroutine objects from error reporting and session activation instead of expressing an asynchronous callback contract directly. That made completion depend on the dispatch path noticing and awaiting the returned object.

Make the callback asynchronous and explicitly await both onlogon() and _activate_session(), ensuring failure notification and activation finish before callback dispatch continues.
2026-07-13 17:25:22 +02:00
Jarrod Johnson e853db5c36 Support affluent peeraddresses, when available 2026-07-13 11:16:47 -04:00
Jarrod Johnson 1660fe9c7d Merge pull request #231 from Obihoernchen/ipmioemfix
Improve SEL record type handling in aiohmi
2026-07-13 08:56:28 -04:00
Markus Hilger 670a11666d Fix remaining hardware async responses 2026-07-13 02:50:12 +02:00
Markus Hilger ca6a54ab05 Fix asynchronous console control dispatch 2026-07-13 02:50:12 +02:00
Markus Hilger 430260becf Await collective address propagation 2026-07-13 02:50:12 +02:00
Markus Hilger 3c6e7d202f Await BMC discovery configuration operations 2026-07-13 02:50:12 +02:00
Markus Hilger b5c5f62789 Fix OEM asynchronous operation dispatch 2026-07-13 02:50:12 +02:00
Markus Hilger 0165fc9935 Fix IPMI coroutine result handling 2026-07-13 02:50:11 +02:00
Markus Hilger 38746b19d5 Import signal for SSH agent cleanup 2026-07-13 02:50:11 +02:00
Markus Hilger b5e0e9f9e4 Fix asynchronous console and shell contracts 2026-07-13 02:50:11 +02:00