2
0
mirror of https://github.com/xcat2/confluent.git synced 2026-08-03 16:07:00 +00:00
Commit Graph

6510 Commits

Author SHA1 Message Date
Markus Hilger 474e2bd975 Drop unreachable web client check in get_diagnostic_data
wc() either returns a client or propagates the exception raised while
logging in; it cannot return None the way connect() could.
2026-07-27 04:56:32 +02:00
Markus Hilger 6e6cbce0a2 Do not report an interrupted firmware update as complete
The retry counter is there to ride out a few unanswered polls, but
exhausting it broke out of the loop with complete still unset and fell
through to the 'complete' return, so an SMM that stopped answering
part way through an apply was reported to the operator as updated.
Raise instead; a genuine finish still leaves the loop on the progress
reaching 100.
2026-07-27 04:56:32 +02:00
Markus Hilger 5135a6cd3d Make the SMM web session cache safe to share
Now that the expiry comparison actually caches a client, the session it
holds is shared, so tearing it down and replacing it needs the same care
the IMM handler already takes:

Dispose of an expired session with a logout instead of dropping the
reference, otherwise every refresh leaves an authenticated session
behind on an SMM that only has a handful of slots.  That logout has to
tolerate a session the SMM has already reaped, hence the except.

Guard the login itself, so two coroutines arriving at an empty or
expired cache do not both log in and orphan one of the two sessions.

Stamp the vintage once the login round trips are done rather than
before, so a slow SMM cannot hand back a client that is already expired.
2026-07-27 04:56:32 +02:00
Markus Hilger ca81907d25 Restore SMM web request semantics lost in the async port
The old WebConnection.request() added a
'Content-Type: application/x-www-form-urlencoded' header to any POST
carrying a body, but grab_response_with_status() only sets a content
type for dict payloads, so the SMM login and every /data form POST now
go out as text/plain.  This is not a fix for an observed failure: an SMM
running FPC variant 38 was measured accepting a text/plain login exactly
as readily as a urlencoded one.  It restores the header the synchronous
code always sent and that the TSM and IMM handlers still set explicitly,
rather than relying on every SMM firmware level being equally lax about
what it will parse.

Also stop hard failing on responses the synchronous code discarded on
purpose.  'set=securityrollback:1' is only understood by newer SMM2
firmware.  And /data/logout answers 401 once the session is gone, as
measured on that same SMM, so raising on a non-200 there turns a
completed hostname, domain or NTP operation into a spurious error.
2026-07-27 04:55:57 +02:00
Markus Hilger 8817ee6deb Await NextScale SMM settings operations
The SMM hostname, domain, and NTP helpers looked synchronous even though their web transport is asynchronous. Removing awaits in the Lenovo OEM handler therefore returned unresolved coroutine work instead of completed settings results.

Convert the SMM settings and logout helpers to the asynchronous web interface, validate HTTP status responses, and await each operation from the OEM handler so callers only observe completed results.
2026-07-27 04:54:44 +02:00
Markus Hilger fbbeda6c86 Fix NextScale asynchronous web client
The NextScale SMM path still used the removed http.client-style interface against the asynchronous WebConnection implementation. Login, configuration, diagnostic, and firmware operations consequently called unavailable methods or left request coroutines unresolved.

Make web-client creation asynchronous, migrate the affected requests to grab_response_with_status(), and await the cached client accessor. Correct the cache expiry comparison so fresh authenticated clients are reused and stale clients are renewed.
2026-07-27 04:54:08 +02:00
Jarrod Johnson 3f2ad75b6d Change output of the nodedeploy timestamp
The 'updated' could be confused for OS updates or similar.
2026-07-24 16:23:27 -04:00
Jarrod Johnson bf195bfdeb Fix stray typing in nodedeploy 2026-07-24 15:10:06 -04:00
Jarrod Johnson a65583c325 Clean up some headers missed in the rebase to aio http 2026-07-24 12:38:33 -04:00
Jarrod Johnson 9fa89d712a Format timestamp consistent with nodeveentlog 2026-07-24 12:05:58 -04:00
Jarrod Johnson b3b16c6497 Do not fail on inability to do REUSEPORT 2026-07-24 11:52:41 -04:00
Jarrod Johnson d81ab1d239 Remove microseconds from the last updated timestamp 2026-07-24 09:30:42 -04:00
Jarrod Johnson 43c98552f4 Change to use standard iso format 2026-07-24 09:12:17 -04:00
Jarrod Johnson 2fed3ddb57 Add timestamp to booted information on diskless boot
It can be ambiguous if the node booted recently or not.
2026-07-24 08:46:47 -04:00
Jarrod Johnson 538a51310b Allow confluentdbutil to operate on an alternate directory 2026-07-24 08:12:02 -04:00
Jarrod Johnson 61e0524a56 Some fixup of SELinux contexts for EL10 diskless boot
Unfortunately, the problem of urlmount's selinux context is left open.

urlmount starts before policy load, preventing transition.

However the policy blocks access urlmount needs when loaded.
2026-07-23 15:58:07 -04:00
Jarrod Johnson 57418696f1 Merge remote-tracking branch 'xcat/master' 2026-07-20 17:11:32 -04:00
Jarrod Johnson 5136b95cde Adjust to EL10 grub stub cfg
The syntax changed, make the code more adaptive to a variety of situations.
2026-07-20 17:11:19 -04:00
Jarrod Johnson 933354f454 Merge pull request #251 from Obihoernchen/yaml-fixes
Harden and clean up config dump/restore
2026-07-20 14:29:55 -04:00
Markus Hilger b5f54c9382 Fix tenant enumeration path in dump_db_to_directory
os.path.join(ConfigManager._cfgdir, '/tenants/') discards the cfgdir
because the second component is absolute, so it resolves to /tenants/
rather than <cfgdir>/tenants.
Tenants are not used yet but let's not face this issue in the future.
2026-07-20 19:18:08 +02:00
Jarrod Johnson 9dfb3ea42b Merge pull request #250 from Obihoernchen/stateless-booted-status
Report stateless boot completion via new 'booted' status
2026-07-20 12:55:55 -04:00
Jarrod Johnson 9128f774ea Merge pull request #249 from Obihoernchen/merge
Comment out the MERGE statement in syncfiles
2026-07-20 12:53:41 -04:00
Markus Hilger 93a6c535b2 Clean up YAML dump/restore code
Rename the format parameter to fmt to stop shadowing the builtin,
pass the already parsed key data dict directly to _restore_keys
instead of reserializing it, use yaml.safe_dump for symmetry with
the safe loader, and consolidate the five repeated per-format dump
blocks into one helper.
2026-07-20 18:03:42 +02:00
Markus Hilger 9101b07d54 Report stateless boot completion via new 'booted' status
Diskless profiles had the updatestatus callback in onboot.sh commented
out because no suitable status existed: 'complete' clears
deployment.pendingprofile, which the PXE responder requires to answer
the next network boot of a diskless node.

Add a 'booted' status that records the pending profile as
deployment.profile while leaving pendingprofile armed and skipping
autolock, and enable the onboot.sh callback in all diskless profiles.
nodedeploy now shows 'pending: <profile> (booted)' for a running
stateless node.
2026-07-20 15:59:30 +02:00
Markus Hilger 73c5b9cebf Comment out the MERGE statement in syncfiles
It's confusing for users to have this enabled by default.
This should be opt-in as everything else.
2026-07-18 03:40:48 +02:00
Markus Hilger d415fcd97f Restrict YAML implicit typing on restore
PyYAML implements YAML 1.1 implicit typing, so hand edited values like
'yes', '52:54:00:12:34:56', or '2026-07-17' in a YAML dump would be
restored as bool, sexagesimal int, or date instead of strings (the
date additionally crashing the JSON re-serialization).  Load with a
SafeLoader subclass that only implicitly types scalars the PyYAML
dumper would have quoted when emitting strings, keeping dump/restore
round trips faithful.
2026-07-18 01:26:14 +02:00
Markus Hilger a8b443dc93 Provide clearer error on restore with mismatched dump format
Restoring a YAML dump without --yaml (or vice versa) previously
reported 'Cannot restore without keys, this may be a redacted dump'.
Point at the actual format of the dump instead when the keys file
exists in the other format.
2026-07-17 23:56:05 +02:00
Jarrod Johnson 12ef3fc529 Merge pull request #245 from Obihoernchen/selinux
SELinux label diskless runtime files on EL
2026-07-16 20:10:43 -04:00
Jarrod Johnson 2b902cb67f Merge pull request #246 from Obihoernchen/el10capture
Enable EL10 imgutil capture
2026-07-16 20:09:42 -04:00
Jarrod Johnson 8e5b37e09b Merge pull request #248 from Obihoernchen/add_local_repositories
Skip add_local_repositories if no imgutil build --source is set
2026-07-16 20:07:58 -04:00
Markus Hilger b6fb58b31f Skip add_local_repositories if no imgutil build --source is set
BUILDSRC is only set if imgutil build is run with --source, otherwise
the build host repos are used. If --source is not used, there is no
distribution symlink and add_local_repositores failed with 404.
Check if BUILDSRC is set and skip add_local_repositories if this is the
case.
2026-07-17 00:06:57 +02:00
Jarrod Johnson 5db32996f9 Provide cleaner error on requesting non-existant tenant. 2026-07-16 17:55:22 -04:00
Markus Hilger dd120dc5de Allow el10 capture in imgutil 2026-07-16 19:59:43 +02:00
Markus Hilger e1a066bf49 Use dhcpcd for EL10 capture prerequisites
EL10 diskless networking uses dhcpcd and no longer dhclient and ipcalc.
Keep the existing dhclient and ipcalc requirements for older EL capture targets.
2026-07-16 19:58:53 +02:00
Markus Hilger c3c4805f9d SELinux label diskless runtime files on EL
Centralize the SELinux chcon helper and use it for downloaded
systemd units, onboot hooks, and apiclient files across EL7 through EL10.
Include chcon in captured EL initramfs images.

Without this fix the onboot services failed to start on SELinux enabled
captured image.
2026-07-16 19:52:52 +02:00
Jarrod Johnson 8b0fb03c66 Disable implicit tenant creation
If we support more tenants, we will modify that branch.
2026-07-16 12:39:22 -04:00
Jarrod Johnson a0995b63ba Merge remote-tracking branch 'xcat/master' 2026-07-15 14:47:51 -04:00
Jarrod Johnson ea9be4aae0 Merge pull request #239 from Obihoernchen/ci
Add CI pipeline
2026-07-15 13:54:15 -04:00
Jarrod Johnson 2644ad861d Merge pull request #244 from Obihoernchen/caperm
TLS CA permission fixes
2026-07-15 13:51:47 -04:00
Jarrod Johnson b812d2aa21 Merge pull request #243 from Obihoernchen/sysctl
Increase net.core.rmem_max to 4 MB
2026-07-15 13:47:34 -04:00
Jarrod Johnson 164a168694 Add fallback to mac table
If LLDP is uncooperative, maybe the mac was learned.

If no mac apparently learned, then we ping_everywhere in hopes of soliciting traffic, and then rescan the switches.

Then get all mac addresses, try to determine zone from generated mac, and print on success.
2026-07-15 08:24:33 -04:00
Markus Hilger 63a0cd237f Add missing postinst steps to Ubuntu
The following post install steps were missing on Ubuntu builds:

- Permission fixes
- sysctl load
- Service restart
- confluent PAM symlink to /etc/pam.d/sshd. It works without it on
  Ubuntu because it falls back to other which allows login on Ubuntu,
  but the behaviour should be the same on every OS. Furtheremore, an
  admin might implement additional steps to sshd PAM and would like to
  have this in Confluent, too
2026-07-15 06:02:03 +02:00
Markus Hilger 86d90281e0 Speed up find
Spawn just one find process and stop on first hit
2026-07-15 05:55:48 +02:00
Markus Hilger 4b61351373 Create and maintain the TLS CA as the confluent service account
The CA database under /etc/confluent/tls/ca is typically created by a root context such as osdeploy initialize -t, but the confluent service runs as the owner of /etc/confluent, and openssl ca rewrites the database (index, serial) as the invoking user on every issuance. Certificate issuance through the running service (e.g. the /self/tlscert deployment API) then fails on the root-owned database until packaging happens to repair the ownership.

Run the CA creation (full CA and the currently unused simple CA variant) and the openssl ca invocation under normalize_uid, the convention already used when publishing the CA certificate. The issued certificate is staged through a temporary file since the destination may only be writable by the invoking user, e.g. the web server certificate paths.

Existing root-owned CA databases are repaired by packaging or manually via: chown -R --reference=/etc/confluent /etc/confluent/tls
2026-07-15 05:55:48 +02:00
Markus Hilger fb41537555 Increase net.core.rmem_max to 4 MB
This was introduced 8 years ago. Newer OSs have 4194304 as default.
Confluent shouldn't decrease the default.

RHEL 8           212992
RHEL 9          2097152
RHEL 10         2097152
Fedora 44       4194304

Ubuntu 24.04     212992
Ubuntu 26.06    4194304

SLES 15          212992
SLES 16          212992

It was changed to 4MB in upstream kernel, too:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a6d4f25888b83b8300aef28d9ee22765c1cc9b34
2026-07-15 03:26:02 +02:00
Markus Hilger fa605160e0 Merge branch 'master' into ci 2026-07-14 17:04:16 +02:00
Jarrod Johnson 07df9a8c94 Ensure prefix is a string 2026-07-14 11:02:15 -04:00
Jarrod Johnson 5cfc3f07af Fix nodemedia attach
The hardening blocked all URL patters.
2026-07-14 11:02:08 -04:00
Markus Hilger 06c7352394 Log exceptions do not swallow it 2026-07-14 17:01:33 +02:00
Jarrod Johnson 3d0b1c1fce Merge remote-tracking branch 'xcat/master' 2026-07-14 08:32:08 -04:00