REQUIRED_PACKAGES named util-linux-extra for every release. focal and jammy
have no such package -- apt reports "Candidate: (none)" -- so apt-get install
exits non-zero and, under set -euo pipefail, the build stops before dracut
runs. hwclock is in util-linux there, which is essential and already present.
optional_packages() keeps a package only where apt has a candidate for it, and
util-linux-extra goes through it. The unconditional list keeps isc-dhcp-client
and ifenslave, which every release has and neither of which the build root
carries by itself.
The call to verify-genesis-payload goes with it. That script is added by the
genesis payload branch, not this one, so the line stopped the build at the
point it was meant to guard.
Also corrects the plan count and a dereference in the test committed before
this one.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddeb-genesis-base names util-linux-extra in REQUIRED_PACKAGES for every
release. Measured on the four Ubuntu management nodes: focal and jammy report
"Candidate: (none)" for that package and carry hwclock in util-linux, which is
essential and already in the build root; noble and resolute carry it in
util-linux-extra. apt-get install with a package it cannot locate exits
non-zero, and the script runs under set -euo pipefail, so the build stops on
two supported targets before dracut runs.
util-linux only Suggests util-linux-extra, and the install passes
--no-install-recommends, so a release that split the package has to name it.
The test asserts the unconditional list does not name it, and drives the
selector that decides, with apt-cache shadowed for a release that has the
package and one that does not.
It also drops the assertion that matched "verify-genesis-payload" against the
text of the build script. That proved the string was present, not that the
verifier ran, ran before packaging, or stopped the build -- and the script it
names does not exist on this branch.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs_release_placeholder.t called plan skip_all when builddebs.pl was absent, so a checkout that lost
the file reported 0 tests and exit 0. A test that cannot fail measures nothing.
Die instead, which is what makentp_ntp_deps.t already does for setupntp.
With builddebs.pl moved aside the file now exits 2 and prints "builddebs.pl not found";
before this change it exited 0 and printed "1..0 # SKIP builddebs.pl not found". With the file
present the test passes either way.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
utils_gettimezone.t called BAIL_OUT when it could not lift the two
routines out of Utils.pm. prove stops every remaining file on a bail-out,
so a rename in Utils.pm that breaks the regex in this file also hides
every test that would have run after it. die is just as loud and costs
only this file.
The header also carried the incident report: a node name, an autoinst
file and line, the anaconda message and the minutes the node spent in
status=installing. The reader needs the contract, which is that the value
must be one token.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The comment on the use-domains assertion in ubuntu_subiquity_installnic.t
followed the missing search domain from systemd-networkd through
updateflag.awk to a node stuck in postbooting. The assertion below it
already names what is checked. The comment now carries the one fact the
code does not show: networkd defaults UseDomains to no.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs_release_placeholder.t called BAIL_OUT when its extraction of
the release decision stopped matching. prove stops every remaining file
on a bail-out, so a rewrite of builddebs.pl that breaks the regex in this
file also hides every test that would have run after it. die is just as
loud and costs only this file.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
genesis_ubuntu_build_root.t called BAIL_OUT at four places where an
extraction stopped matching. prove stops every remaining file on a
bail-out, so one stale regex in this file hides the results of the tests
that would have run after it. die is just as loud and costs only this
file.
Four comments in the same file also explained more than the code hides:
the header retold the failure the test exists for, two narrated the two
helper subroutines, and one traced the lease failure across doxcat, the
image and the node.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Three comments in debian_install_boot_files.t explain more than the code
hides. One narrates the media table the assertions already list. One
repeats the nodeset error message and the releases that produce it. One
describes what mkinstall used to refuse before the change.
A comment states why, once, where the reader cannot re-derive it. Each of
the three now carries only the fact the assertions do not show.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The EL 10 riscv64 compute node installs no package. Its kickstart carries
"timezone Could not determine timezone checksum --utc" at line 21, anaconda
answers "One or zero arguments are expected for the timezone command", and the
node holds status=installing until retry_install.sh reinstalls over it
(build 114).
gettimezone in Utils.pm compares /etc/localtime against every file under
/usr/share/zoneinfo and, when that pipeline exits non-zero, returns the string
"Could not determine timezone checksum" to its caller as a name. The Rocky 10.2
riscv64 cloud image ships no /etc/localtime and runs on UTC, so the pipeline
prints nothing and exits 1.
gettimezone now reads the /etc/localtime symlink first, which is what every
current distribution uses and what the scan cannot answer for a host with no
/etc/localtime, then the scan, then /etc/timezone, and returns UTC when none of
them names a zone. It never returns a value with a space in it, which is all the
kickstart timezone command accepts. The paths take an optional root prefix so
the test drives them against a scratch tree.
xCAT-test/unit/utils_gettimezone.t stops without _zone_from_path, and each of the
three sources it asserts fails a separate assertion when it is removed.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The EL 10 riscv64 compute node xcat56-cn never installs a package. Its kickstart
carries, at line 21, "timezone Could not determine timezone checksum --utc";
anaconda answers "One or zero arguments are expected for the timezone command"
and stops. The node holds status=installing until retry_install.sh reinstalls
over it (build 114).
gettimezone in Utils.pm compares /etc/localtime against every file under
/usr/share/zoneinfo and returns the string "Could not determine timezone
checksum" when that pipeline exits non-zero. The Rocky 10.2 riscv64 cloud image
has no /etc/localtime and runs on UTC, so the pipeline prints nothing, exits 1,
and the sentence is written into the kickstart as a name.
This test drives the routine against a scratch root and asserts it names a zone
for a root with a symlink, with /etc/timezone, and with neither -- and that the
value is one token, which is all the kickstart timezone command accepts.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
On Ubuntu 26.04 the installed compute node keeps systemd-resolved, which takes
the DNS server from DHCP and no search domain, so /etc/resolv.conf reads
"search .". mypostscript then runs "updateflag.awk $MASTER 3002" with the short
management node name, the flag update fails eleven times, and the node stays at
postbooting until retry_install.sh gives up. The management node does offer
domain-search; the node discards it.
compute.subiquity.tmpl writes /target/etc/netplan/00-xcat-install.yaml with
dhcp4: true alone, so systemd-networkd applies its UseDomains default of no.
Add dhcp4-overrides: use-domains: true to both branches, the one that renames
the interface and the one that matches by MAC alone.
ubuntu_subiquity_installnic.t runs the template's own late-command and asserts
the netplan it writes carries the setting. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The netplan xCAT writes for a compute node sets dhcp4: true and nothing else,
so systemd-networkd keeps its UseDomains default of no. The node takes the DNS
server from DHCP and drops the offered domain-search, and no single-label name
resolves on the installed node.
Assert the rendered netplan asks networkd to use the DHCP domains. The
assertion runs the template's own late-command, so it reads the file the
installer writes, not the template text.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Every Genesis deb the Ubuntu dep pipeline publishes is versioned
2.19.0-snap000000000000. apt never upgrades an installed
xcat-genesis-base, so the channel cannot ship a Genesis fix.
Release is a tracked placeholder that buildrpms.pl overwrites with the
commit time. The dep job clones xcat-core only to build the Genesis
debs, so buildrpms.pl never runs there and builddebs.pl reads the
placeholder as an authoritative release.
builddebs.pl now treats an all-zero Release as an unstamped tree and
falls back to the commit time. snap_release() renders a real time, so
no build ever writes that value.
builddebs_release_placeholder.t covers it. The test fails on the
unfixed tree with snap000000000000.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Every Genesis deb the Ubuntu dep pipeline publishes is versioned
2.19.0-snap000000000000, so apt never upgrades an installed
xcat-genesis-base and the channel cannot ship a Genesis fix.
Release is a tracked placeholder. buildrpms.pl overwrites it with the
commit time, but the dep job clones xcat-core only to build the Genesis
debs, so buildrpms.pl never runs and builddebs.pl reads the placeholder
as an authoritative release.
The test extracts the release decision from builddebs.pl and runs it
against a tree that carries the placeholder. It fails today with
snap000000000000 and passes for a real Release file, no Release file,
and an explicit --release.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddeb-genesis-base installs its build root from REQUIRED_PACKAGES. That list
has no isc-dhcp-client, no ifenslave and no util-linux-extra, so dracut cannot
find dhclient, ifenslave or hwclock. dracut_install prints one line for each and
returns, the script never reads dracut's result, and the .deb is packaged with
three holes. A Genesis image with no dhclient never obtains a provisioning lease.
Add the three packages, and run verify-genesis-payload on the extracted payload
before dpkg-buildpackage. --commands-from reads the command names back from the
module that is about to be built, so a later addition to the module is checked
too. This is what xCAT-genesis-base.spec already does for EL.
genesis_ubuntu_build_root.t fails five of its eight assertions without this
change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
nodeset stops the diskful install on every ppc64le cell of
xcat-core-devel-ubuntu-cd (build #119, ubuntu-24-ppc64le-devel):
xcat25-cn: The network boot initrd.gz is not found in
/install/ubuntu24.04.4/ppc64el/install/netboot.
The Ubuntu ppc64el live-server ISO carries no netboot tree. The installer
kernel and initrd sit under casper. Two places in
xCAT-server/lib/xcat/plugins/debian.pm reject that media: %INSTALL_BOOT_FILES
describes no casper layout for POWER, and mkinstall applies a second,
POWER-only precondition on install/netboot/initrd.gz before it calls the
resolver.
The POWER table now carries casper/hwe-vmlinux + casper/hwe-initrd and
casper/vmlinux + casper/initrd, after the netboot entries so a netboot tree
still wins. install_media_is_bootable replaces the POWER-only precondition, so
one routine decides what media can boot.
debian_install_boot_files.t resolves the POWER casper layouts and drives
install_media_is_bootable. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddeb-genesis-base builds the Genesis payload in the host root. Its
REQUIRED_PACKAGES list omits isc-dhcp-client, ifenslave and util-linux-extra, so
dracut cannot find dhclient, ifenslave or hwclock. dracut_install reports each
missing command and returns, the script does not read dracut's result, and the
.deb is packaged with the holes. A Genesis image with no dhclient never obtains a
provisioning lease, so the node netboots and never reports in.
The EL path gates the payload: xCAT-genesis-base.spec runs
verify-genesis-payload. The Ubuntu path runs no gate at all.
genesis_ubuntu_build_root.t sources the Ubuntu dracut module with dracut_install
shadowed, calls install(), and reads back the commands it installs
unconditionally. It then evaluates the REQUIRED_PACKAGES assignment and asserts
the build root supplies each one. Five of its eight assertions fail on this
commit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
reg_linux_diskfull_installation_flat fails on every ppc64le cell of
xcat-core-devel-ubuntu-cd. nodeset stops before it writes a grub2 entry
(build #119, ubuntu-24-ppc64le-devel):
xcat25-cn: The network boot initrd.gz is not found in
/install/ubuntu24.04.4/ppc64el/install/netboot.
The Ubuntu ppc64el live-server ISO carries no netboot tree. It keeps the
installer kernel and initrd under casper: 22.04 and 24.04 ship hwe-vmlinux and
hwe-initrd beside vmlinux and initrd, 26.04 ships the release pair only.
%INSTALL_BOOT_FILES describes that layout for x86 and not for POWER, and
mkinstall carries a second, POWER-only precondition on
install/netboot/initrd.gz that no live image satisfies.
The new cases resolve the POWER casper layouts, and ask a single routine
whether media is bootable. They fail now.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The compute pkglist of each Subiquity release and architecture is
resolved as mkinstall resolves it, its packages read as ospkgs reads
them, and joined with the template's fixed set: chrony must be there, ntp
must not, and the union must carry exactly one time daemon. 16.04 must
still resolve the shared list with ntp. Against the previous tree the
20.04, 22.04 and 24.04 cases fail.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Ubuntu 20.04, 22.04 and 24.04 resolved the shared compute.pkglist on
every architecture but x86_64 20.04, and that list names ntp for the
releases before Subiquity. The Subiquity template installs chrony, and on
these releases ntp pulls ntpsec, which conflicts with it, so one apt
transaction with both cannot be satisfied and ospkgs replaced chrony
after the first boot.
Each of the three releases now has its own default list with chrony, the
list 26.04 already had. The shared list keeps ntp for the releases where
chrony was not the default.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The install compute profile gets a 24.04 pkglist of its own next, so the
assertion that it resolves the shared list with ntp is removed ahead of
it. The service, kvm and netboot cases keep that pin.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The packages list must carry the token and keep openssh-server and wget,
which xCAT and the template's own commands need. Against the previous
template the token assertion fails.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
compute.subiquity.tmpl named a fixed package set, so the osimage pkglist
took effect only when ospkgs ran after the first boot. The packages list
now ends with the autoinstall token, so the pkglist packages install from
the configured apt mirror during the autoinstall. The fixed set stays, so
a node installs the same packages as before plus its pkglist.
The apt sources the installer gets for the pkgdir mirrors and the
otherpkgs repository are removed from the target at the end of the
install. ospkgs and otherpkgs write their own after the first boot, and a
second source for one repository with other options makes apt refuse the
whole list.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The entry filter is called directly, and a template with the token is
rendered through subvars against a pkglist with a comment, a removal, a
group and an include. The rendered list must carry one item per package
at the token's indentation, and no token line without a pkglist. Against
the previous module the helper does not exist.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
A Subiquity autoinstall installs what its user-data packages list names,
and the templates could only name a fixed set, so the osimage pkglist
reached an Ubuntu node through ospkgs after the first boot. The preseed
token has no autoinstall form: the package list is YAML, one item per
line.
A list line that carries #INCLUDE_DEFAULT_PKGLIST_AUTOINSTALL# is now
replaced by one item per pkglist package at the same indentation, with
includes followed and without repeating the items the template lists
above it. The line is replaced in the include pass of subvars, so a site
template that includes the stock one is served too. A plain name and a task are
installed this way, and a comment after them ends the record. A version
pin or a target release stays with ospkgs, because the installer runs
apt-get without --allow-downgrades and a pin can require one, and so does
a name with an architecture qualifier, because a foreign architecture is
enabled by a postscript that runs later. A record that begins with
a removal or a group is left out whole, as ospkgs removes or installs it
whole, and so are a removal written with a trailing hyphen, markers and
preseed directives. A list that carries a #ENV: setting or an unreadable
include is left to ospkgs whole. So is the list of an osimage with
environvar, which mkinstall now hands over, and such an image's pkgdir
mirrors stay out of the installer's sources as well: those variables
reach apt-get only through ospkgs, and a mirror may need them. An osimage
without a pkglist loses only the token line.
The installer's apt configuration turns recommended packages off, as
ospkgs installs the list without them; curtin writes that setting into
the target, where the template removes it with the installer's sources.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
get_pkglist_tex joins the records of a pkglist with commas for the preseed
and OSPKGS consumers, so a caller that needs the records themselves cannot
recover a record that contains a comma, such as a tasksel directive. The
new reader returns the records whole, comments dropped and includes
followed in place. It reads each line as get_pkglist_tex does and resolves
every include, nested ones too, against the directory of the listed
pkglist, as get_pkglist_tex does. The comma text is unchanged.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The source specs are derived from a pkgdir value alone, and the apt
configuration is rendered with only the database readers stubbed, online
and offline, Deb822 and legacy, with and without otherpkgs repositories.
Against the previous module the spec helper does not exist.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
An osimage pkgdir can name mirrors after the install media, and ospkgs
installs from all of them, but the autoinstall apt configuration offered
the installer only the archive mirror and the otherpkgs repositories. A
package that only a pkgdir mirror carries could not be installed before
the first boot.
The mirrors now join the installer's apt sources, next to the otherpkgs
ones, in every form the configuration takes. An entry written as URL,
suite and components is an apt source line, as ospkgs writes it, and a
suite that is an exact path needs no component. A local directory that
is a flat repository is served by the management node and trusted, as an
otherpkgdir is. An entry that names an Ubuntu archive mirror the
installer already has a source for, the configured one or a default one,
carries that source's signing key, the archive keyring on the Deb822
releases and none before them, because apt rejects a second source for
the same suite whose signing key differs. For the same reason a
repository is offered once, whether pkgdir names it twice, as a directory
and as its URL, or the otherpkgdir names it too. Anything
else is no apt source for ospkgs either and is left out.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The online configuration is rendered with an otherpkgs repository on a
classic release and on a Deb822 release: the first must carry the
one-line trusted source, the second a Deb822 stanza with Trusted: yes and
no one-line form. Against the previous module the Deb822 case renders the
one-line form.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The online autoinstall apt configuration offered every otherpkgdir entry
as a flat one-line source with trusted=yes. From 24.04 on curtin
converts a one-line source to Deb822 before it writes the file and keeps
only the type, URI, suite and components, so the unsigned repository
reached apt without the option and apt rejected it during the install.
An entry written as URL, suite and components, the form the otherpkgs
documentation gives for a mirror, was written whole as the URL, which
apt cannot parse either.
The sources are now Deb822 stanzas on those releases, which curtin
writes as they are, with Trusted: yes, and a mirror entry keeps its
suite and components as fields. The releases before 24.04 keep the
one-line form, with the same fields.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Inside the private network namespace the fake tcpdump records its parent,
its output file and the TERM it receives, ps records any use, and a second
fake tcpdump fails at once. Both copies must start tcpdump directly, write
under TMPDIR, stop it by pid, leave no file behind, and exit 1 when the
capture fails. The block skips where no namespace is available.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Both copies wrote the capture to /tmp/dhcpdumpfile.log, so a file left by
another user or an earlier root run blocked the probe and two runs
overwrote each other. tcpdump ran behind a shell, so the script killed it
by searching the process table for any tcpdump on the interface, and a
tcpdump that failed to start left an empty file that read as zero DHCP
servers. An interrupt left the capture running.
The capture file is now a private temporary file removed on every exit.
The child execs tcpdump itself, so the script stops and reaps exactly that
pid, and a tcpdump that ended before the window did fails the run. A
failed send and an INT or TERM stop the capture and exit 1 as well.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The ospkgs apt block runs with a shadowed apt-get that fails only for the
cuda install, and RETURNVAL must carry that status. Against the previous
script the test fails with RETURNVAL=0.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Both cuda install blocks read the package manager status after they had
restored and exported ARCH, so R was always 0 and a failed cuda install
left the node reporting success. The status is now read directly after
the install on the apt path and on the yum and dnf path.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat_apt_get is sourced from xcatpkgutils.sh and called with a shadowed
apt-get that records its environment and arguments. The apt block of
ospkgs and the two apt commands of otherpkgs are extracted from the
scripts and executed the same way, since neither script can be sourced.
Against the previous scripts 6 of the 7 tests fail: the helper does not
exist, and the recorded calls carry --force-yes.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
ospkgs and otherpkgs passed --force-yes to every apt-get upgrade and
install. apt has printed a deprecation warning for it since 1.1, and the
flag also allowed downgrades, changes to held packages and the removal of
essential packages, none of which an unattended package update should do.
Three of the ospkgs installs relied on it alone for the unsigned xCAT
repositories and ran without DEBIAN_FRONTEND=noninteractive, unlike the
upgrade.
The 8 call sites now go through xcat_apt_get in xcatpkgutils.sh, which
runs apt-get with -y and --allow-unauthenticated under
DEBIAN_FRONTEND=noninteractive. Each call site keeps its own quiet, Dpkg
and recommends options. A held package or an explicit older version now
fails the install instead of being forced, as on the rpm side.
--allow-unauthenticated is accepted by every apt release xCAT provisions.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The control files are parsed the way dpkg reads them, one stanza per binary
package, and the declarations of perl-xcat, xcat-client, xcat-server, xcat
and xcatsn are held to the modules and tools their files use, with the
ipmitool-xcat floor. 25 of 26 assertions fail against the previous
control files.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The deb metapackages and xcat-client only recommended nmap and
ipmitool-xcat, while xCAT.spec, xCATsn.spec and xCAT-client.spec require
them. An install with --no-install-recommends therefore left bmcdiscover
without a scan method and out-of-band management without the tool ipmi.pm
and bmcdiscover.pm hard-code. Both move to Depends.
The ipmitool-xcat floor rises from 1.8.17-1 to 1.8.18-4, the floor of
xCAT.spec. Builds before 1.8.18-3 lack the CVE-2020-5208 patch.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xcat-server ships xcatws.cgi and restapi.pl with the endpoint enabled in
apache, and both load CGI at top level. Nothing declared the module, so on
a management node where nothing else pulled it every /xcatws request
answered 500. The rpm package requires perl(CGI).
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão
Vinícius Ferrão