Own collection, sensitivity, finalization, and reset as one request-scoped state object so xcatd only forwards callbacks and appends the finalized text.
The template renderer writes the HTTP port into the URLs of a kickstart
file, an autoyast profile and a preseed file. It writes the port always, so
a URL gets the text ":80" when the site keeps the default port.
A URL that gives no port already goes to port 80. Write the port only when
the site sets a port that is not 80. The netboot plugins xnba, dhcp and mknb
already do this.
The module gave the port in three different ways. One place wrote the port
always. One place wrote the port only when the port was not 80. One place
wrote the port always from the environment. Put the rule in one routine and
let the four places use that routine.
The routine also accepts a port that is set to nothing. Before, an empty
value made a URL that ends with a colon.
The commands.log response classifier used a "passw" text match on the
request arguments. A secret whose name has no such text passed the
check, so a read of an authentication key, a privacy key or the snmpc
site value logged its bare value in the response. A command that
expands an argument also passed the check: nodels with a table name
returns every column of the table, and lsdef returns attributes that
the request never names. The daemon also ran redact_password over the
whole connection log on each request, so the redactor split at the
first request of the connection and the change signal swept the text of
earlier requests and responses.
Add secret_in_request. The routine reports a request that names a
secret attribute, selects a secret site key, or dumps a table that owns
a secret column through tabdump or nodels, from the same secret set
that the argument redaction uses. The response classifier calls it, so
the response of such a request logs as redacted.
Add secret_in_response. The routine reports response text that holds
"passw" or a secret attribute name in assignment or column form. The
response finalizer calls it in place of the bare text match, so an
expanded listing that carries an authentication key or a product key
logs as redacted even when the request never names it. The lsvm
response is the directory entry, whose passwords are positional, so
the classifier marks the command itself.
Build each request segment alone, redact the segment, and then append
it to the connection log. The redactor now always sees the current
command, and the change signal covers only the current request.
Treating an empty noderes.installnic as "mac" skipped noderes.primarynic: a node
that configures only primarynic was installed with a netplan matching on mac.mac
that never renamed the interface, and a primarynic holding a MAC address was
ignored. The order xCAT uses everywhere else is installnic -> primarynic ->
mac.mac, with either attribute allowed to hold an interface name or a MAC
address, and xCAT::NetworkUtils::gen_net_boot_params already implements it for
the netboot kernel parameters.
Resolve it in Perl and hand the template the answer:
- xCAT::Template::subiquity_install_netcfg() delegates the order to
gen_net_boot_params and returns the netplan set-name (empty when the device is
matched by MAC alone) and the address to match on, with mac.mac parsed by
xCAT::Utils::parseMacTabEntry so |-separated, !hostname-suffixed entries
resolve to THIS node rather than to whichever entry came first.
- #SUBIQUITYINSTALLNIC# / #SUBIQUITYINSTALLMAC# render those two values.
installnic and primarynic are read blank-okay -- a node that sets neither is
normal, and is exactly what "match on mac.mac" means -- while mac.mac stays a
required lookup, as it was when the template read it directly.
- compute.subiquity.tmpl consumes the resolved values. No part of the fallback,
and no mac.mac parsing, is left in shell: it only chooses whether to emit
set-name, based on whether a name was resolved.
ubuntu_subiquity_installnic.t now passes: the six resolution cases, multi-entry
mac.mac, and the netplan the template's own late-command writes for a resolved
name and for none. ubuntu_subiquity_template.t asserts the template no longer
reads installnic/primarynic or parses MACs itself.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The daemon redacted secret attributes on the joined command string. The
match failed when a value held a space. The match also failed for a "+="
splice assignment. The validate() path did not quote the arguments, so a
multi-word secret value kept its later words in syslog and in the
auditlog table. A password that a command receives through an option or
a positional operand was not redacted at all. The debug dispatch trace
wrote the raw arguments to syslog when site.xcatdebugmode was set.
Redact the argument vector before the daemon joins it. Add
redact_password_args for this task. The routine masks the value of a
secret attribute in any argument, at the start or embedded after another
token. An embedded secret assignment masks to the end of the argument,
because a shell value may hold quotes and spaces. The routine allows
spaces around the operator. It accepts the "=", "+=", ",=", "^=", "!=",
"=~" and "!~" operators that chdef, nodech and node selection use. It
masks a password option value in each form that Getopt::Long accepts: a
separate argument, a compact short option, a bundle of short options
with the "?" help letter, a "+" option prefix, a single-letter option
with two dashes, a long option, a long option with an equals sign, and
an abbreviated long option. The long-name match runs first, so a long
option keeps its name and masks its value. A walk over each bundle then
finds the first secret letter, so the mask always starts at the option
and the result does not depend on hash order. The walk knows which
other letters of a command take a value, so a secret letter inside such
a value does not redact and the audit text stays correct. The walk also
knows which letters take an integer, because the z/VM cpu option
consumes only its signed digits and the parser then continues the
bundle into the password option. The value stops match letter case,
because a bundle keeps short options case sensitive and an unknown
capital letter does not absorb the rest. The mkvm secret match ignores
letter case, because the z/VM parser keeps the Getopt::Long default for
long names. The mkhwconn match keeps letter case, because -p
is the hardware control point and -P is the password. The routine knows
the password options of bmcdiscover, switchdiscover, mkhwconn, mkvm,
createvcluster, lsvcluster and rmvcluster, the rspconfig password
assignments, the mkvm clone pw= operand, and the positional password
operands of chvm. It masks the site.value argument of tabch and chtab
when a selector or a site.key assignment names snmpc. An exact short
option that takes a non-secret value stays visible, so the PPC mkvm -p
profile is not an abbreviation of --password. The dispatch trace builds
its text from the redacted vector.
Add snmpc, productkey, prodkey.key, tokenid and token.tokenid to the
secret list, with community and pdu.community. The secret list holds
only attributes that map to a secret column, so key and sshkeydir stay
visible.
redact_password keeps a second pass over the joined string. This pass
masks an embedded secret assignment to the end of the line, because the
argument boundaries are gone after the join.
The commands.log response classifier marks a response sensitive when the
request was redacted. The argument vector pass sets that signal, so a
secret whose name has no "passw" text still marks its response.
xcatd expands the request noderange before it authorizes the caller: once to
count the nodes, and once in validate() to match the policy rules. The ^
operator makes xcatd open a caller-named file at that point. A client can
connect without a certificate, because the listener does not require one, and
such a client has no peername.
Expand these two pre-authorization noderanges with nofile when the caller has
no peername (checked with defined, so the identity "0" still counts as
authenticated). If validate() finds a rejected ^file atom on such a request,
deny it. An authenticated caller expands ^file as before.
xCAT::XML wraps XML::Simple. It blocks XML external entities. An external
entity can read a local file or a network resource.
XML::Simple with new_xml_parser() installs an ExternEnt handler. The
handler returns the system id, not the content.
XML::Simple 2.20 to 2.24 has no new_xml_parser(). On these versions xCAT
uses build_tree_xml_parser() instead. This function did not install the
handler. A SYSTEM entity then expanded. It leaked a local file into the
parsed data.
This change adds the same handler to build_tree_xml_parser(). Both paths
pass parser options in an array reference. XML::Parser->new ignores that
reference. The handler is therefore the real defense. This fix keeps
#6917, which retains the compatibility path for those releases.
This fix was recovered from the lenovobuild branch. See PR #6505.
Only syslog received the redacted arguments. The auditlog table was given the
raw string, so a password removed from syslog and from commands.log was still
written to the database, which persists and is readable by anything with access
to it.
Use the redacted text for both. This also covers the mkvm --password and -w
masking a few lines above, which the table did not have either.
redact_password only knew about bmcdiscover, mkhwconn and rspconfig, so setting
a password the ordinary way left it in the clear:
[Request] chdef node01 'bmcpassword=SEKRET'
Those secrets are carried as an assignment on whichever command happens to set
them, so match them by name rather than by command. The names are every
attribute Schema.pm maps to a password, passwd, authkey or privkey column, and
the columns themselves, since a table qualified assignment such as
passwd.password= is accepted too. An assignment may be written with spaces
around the equals sign and the value may contain spaces, so a quoted argument is
redacted to its closing quote.
Attributes such as key, which names a monitoring attribute, and sshkeydir, which
is a directory, are not secrets and are left readable. The existing per-command
table is kept for a flag like bmcdiscover -p, which carries its secret in the
following argument where there is no name to match.
Goconserver.pm shells out to `service goconserver start|stop|restart` (and
`service conserver stop`). That command is provided by the initscripts package,
which is NOT installed on a minimal EL9/EL10 management node -- there, every one
of these calls fails with rc!=0.
The damaging case is restart_service(). makegocons calls build_conf(), which
rewrites /etc/goconserver/server.conf with the cert-enabled configuration
(global.ssl_key_file/ssl_cert_file/ssl_ca_cert_file), and then calls
restart_service() so the daemon picks it up. When the restart fails, xCAT only
logs "Could not restart goconserver service." and the daemon keeps running with
the configuration it read at boot -- the packaged default, which has no ssl_*
fields. goconserver's TLS is gated on those fields being set (sslEnable stays
false), so it silently serves plain HTTP on the api port while Goconserver.pm
always talks to it over https. Every subsequent request then dies with
SSL connect attempt failed error:0A0000C6:SSL routines::packet length too long
so makegocons/makegocons -d report "Failed to send delete request." and the
console can never be registered. On EL8 the bug is invisible because initscripts
happens to be installed there.
Use the xCAT::Utils service helpers (startservice/stopservice/restartservice),
which resolve the unit through servicemap() and issue `systemctl <action> <unit>`
on systemd hosts, falling back to `service`/`initctl` only where appropriate.
This is the same mechanism the rest of the tree (AAsn.pm and friends) already
uses for named, dhcpd, nfs and others.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
reseat_node opens an IPMI session to the FPC/CMM to reseat a node. The FPC
needs packets sent without maxpending flow-control, but that state is only held
transiently. Add a persistent per-session 'neverwait' flag, set it on the FPC
reseat session, and honor it in sendpayload.
The flag defaults off, so it is a no-op for every other IPMI session; only the
FPC reseat path opts in. (The lenovobuild original also reworked the shared
login/admin_level_set nowait handling; that part is left out to avoid changing
flow-control for all BMCs.)
Not lab-validated (no NeXtScale FPC/CMM available).
Recovered from the unmerged lenovobuild branch (eda85df3, neverwait hunks only).
envvar() resolves a template variable purely from %ENV, so a template that
references $XCATROOT expands to an empty string whenever XCATROOT is not
present in the environment of the process doing the substitution. $::XCATROOT
is set as a package global at module load and is reliably available, so use it
for that one variable before falling back to %ENV.
Recovered from the unmerged lenovobuild branch (original da72c6b8).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
The Ubuntu live-server ISO copycds imports is not a complete apt pool, so
both Ubuntu install paths fail without an external mirror: the netboot
genimage's debootstrap from the ISO-only pkgdir cannot resolve packages,
and the Subiquity diskful install (Template.pm) fell back to the airgapped
file:///cdrom config. Default site.ubuntu_apt_mirror to
http://archive.ubuntu.com/ubuntu when unset (overridable for local/geo/
ports mirrors). In the Ubuntu genimage, when pkgdir has no http mirror,
build the debootstrap source and chroot sources.list from that mirror with
the codename derived from osvers and the release/-updates/-security pockets
(main + universe) -- the compute pkglist needs universe (busybox-static,
dracut*) and the security-updated openssh. Ubuntu-only; EL netboot
(netboot/rh) and copycds/packimage are untouched.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu live-server install media is not a complete apt package mirror.
With the generated apt configuration pinned to the media only, an install that
needs a package absent from the media stalls instead of fetching it. Emit an
online-mirror apt configuration when one is configured (site.ubuntu_apt_mirror)
so missing packages are retrieved from the archive; offline behaviour is
unchanged when it is unset.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Extend the existing sha256-to-sha1 fallback (already present in
got_rmcp_response for Open Session errors) to also cover RAKP2
rejections with "Unauthorized name" (0x0d) or "Invalid role" (0x09).
Ref: #7511
Set bit 4 (0x10) of the requested privilege byte in RAKP Message 1
for name-only user lookup, matching ipmitool behavior. Use the same
value consistently in all HMAC calculations (RAKP2 verification,
RAKP3 auth code, SIK derivation).
Without this, some BMCs fail user lookup with "Unauthorized name"
even though the credentials are correct.
Ref: #7511
OpenBMC-based BMCs return message tag 0 in RAKP2/RAKP4 instead of
echoing the tag from the request. xCAT rejected these as stale
responses and retried indefinitely until timeout.
Accept tag 0 but verify the remote console session ID in the response
matches our current sidm. This prevents stale retries from corrupting
session state while allowing OpenBMC responses through.
Applied to got_rmcp_response, got_rakp2, and got_rakp4.
Ref: #7511
cbc_pad in decrypt mode reads the last byte as the pad count, then
calls splice(@block, 0 - $count). If decrypted data is corrupt, the
pad count can exceed the array size, crashing with "Modification of
non-creatable array value attempted, subscript -16".
Return empty string on invalid padding so the caller treats it as a
decryption failure rather than accepting corrupted data as a valid
IPMI response.
Ref: #7511
Daniel Hilst
Vinícius Ferrão
Vinícius Ferrão
Vinícius Ferrão
Markus Hilger