The stateless xcatroot of dracut_047, run in a chroot with NetworkManager
leases of the dhclient and internal clients in the initrd, must copy
both into the root under their own names. CI now runs the xcatroot test
as root.
xcatroot passes the DHCP lease of the initrd to the root image only from
/tmp/dhclient.*.lease, which the old dracut network scripts write. An
initrd that runs NetworkManager keeps its lease in /var/lib/NetworkManager.
The NetworkManager of the root takes over the same connection, finds no
lease and starts again with DHCPDISCOVER while the address stays on the
interface. A DHCP server that does not answer in time makes
NetworkManager-wait-online fail, and when the DHCP timeout expires
NetworkManager removes the address from the boot interface.
xcatroot now also copies the NetworkManager leases, so the root renews
the lease of the initrd with a DHCPREQUEST. Only the stateless xcatroot
of dracut_047 changes; statelite and the other dracut modules keep their
handoff.
imgutils.pm imports File::Copy with an empty list, so the module exports no
function and a bare move() is not defined. use_host_resolver calls move()
unqualified, so it dies with "Undefined subroutine &imgutils::move called" the
first time genimage reaches it, before it has moved either file aside.
Every other call site in the file already qualifies the name:
File::Copy::cp at line 139, File::Copy::copy at 208 and at 302, the last of
them two lines below the defect. The call is now qualified the same way.
xCAT-test/unit/imgutils_host_resolver.t and imgutils_reused_root.t both die at
load on the unfixed tree and pass with the fix, 34 tests.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Red Hat-family genimage took the image operation lock with
--onlyinitrd only for openEuler. Two geninitrd runs on another release,
or a geninitrd run during packimage, could publish and roll back boot
files over each other.
genimage now takes the lock with --onlyinitrd for every release, as it
already does for a full build.
publish_staged_boot_files looked for the backup of a file during the
rollback to decide whether to restore it or remove the published file.
When that lookup failed, it removed the published file, and the staging
cleanup then removed the backup.
publish_staged_boot_files now records each backup that it makes and
rolls back from that record. A file without an earlier copy is one for
which link reports ENOENT. Any other link error stops the publication.
On NFS, a rename can take effect and still report a failure after the
client retries the request. publish_staged_boot_files then put back
only the files that it published before, so the new file stayed next
to the older set, and the staging cleanup removed its backup.
When the staged copy of the file is gone after a failed rename,
publish_staged_boot_files now puts back that file too.
The Red Hat-family genimage staged and published the boot files only
for openEuler. For the other releases, it copied the kernel into the
image directory before it built the initial ramdisks, and it ignored a
failed kernel copy and a failed move of a ramdisk. When dracut failed,
the new kernel stayed next to the ramdisks of the earlier build. On
images without dracut, a failed cpio replaced the ramdisks with empty
archives.
genimage now builds the kernel and both ramdisks in a temporary
directory inside the image directory for every release, and publishes
them with publish_staged_boot_files after both ramdisks are built. It
stops when the kernel copy, dracut, the move of a ramdisk or any command
of the cpio pipeline fails. openEuler keeps its single stateless
ramdisk.
imgutils_staged_boot_files.t covers the publication of boot files. The
remaining checks of the unit test openeuler_genimage_bootfiles.t cover
disable_vendor_repositories, so the file now has that name.
The native EL cases checked that genimage keeps copying the kernel into
place before dracut runs, which the next change removes.
The Ubuntu and the Red Hat-family genimage each had their own code to
find and copy the kernel of the image and to pack a cpio initial
ramdisk. imgutils now has copy_kernel and pack_initrd for this, and the
Ubuntu genimage uses them.
copy_kernel also takes lib/modules/<release>/vmlinuz, which the Red
Hat-family genimage already takes.
geninitrd ran genimage --onlyinitrd for netboot and statelite images and
ignored its exit status. When genimage failed, geninitrd still exited
with 0. The errors that the genimage client writes to stderr did not
reach the user.
geninitrd now passes the genimage output to the client with stderr
included. When genimage ends with a status other than 0, geninitrd
reports an error and exits with the genimage exit code, or with 1 when
genimage could not run.
genimage copied the kernel to the image directory before it built the
initial ramdisks. When dracut failed for either ramdisk, a new kernel
stayed next to the old ramdisks, and nodeset copied that set. The
legacy ramdisk ignored a failed cpio pipeline and published what the
compressor wrote.
genimage now builds the kernel and both ramdisks in a staging
directory in the image directory. It publishes them together with
publish_staged_boot_files after both ramdisks are built, and writes the
root image check and UFTP markers after that. A failure leaves the
previous kernel and ramdisks in place.
publish_boot_files publishes one kernel with one initial ramdisk, so a
genimage that builds the statelite and the stateless ramdisk cannot
use it. publish_staged_boot_files publishes the kernel and the ramdisks
of several modes from one staging directory. When a rename fails, it
puts back every file that it already replaced.
publish_boot_files keeps its interface for the openEuler build.
genimage ran debootstrap on every run. For Ubuntu 19.04 and later,
debootstrap extracts with tar -k, so a second run on an existing root
image failed at base-files and blamed the mirror.
genimage now skips debootstrap when an earlier run finished it and
updates the image with apt. The image holds the dummy resolv.conf and
the otherpkgs apt sources of that run. Before the first apt command,
genimage removes the apt sources that it writes again later and gives
the image the hosts and resolv.conf of the build host. A .bak file
that an interrupted run left is kept, so the image still gets its own
file back.
genimage stops with an error and leaves the root image unchanged in
three cases. The root image holds another Ubuntu release or
architecture, the debootstrap directory of an unfinished run, or the
bin/uname-binary backup of an interrupted run.
genimage took the last kernel of a sorted file list. When an image
held 6.8.0-99-generic and 6.8.0-100-generic, it published -99 and
built the initrds for it. The uname of the image build also reported
-99 to the package scripts, and it ignored an explicit kernelver.
genimage and that uname now compare kernel releases number by number,
so both choose -100. An explicit kernelver decides for both.
packimage, rmimage and the EL genimage take the image operation lock
of a root image, but the Ubuntu genimage did not. packimage or rmimage
could work on a root image while genimage installed packages in it,
and two genimage or geninitrd runs could change one root image at the
same time.
genimage now takes the lock before it changes the root image, for a
full build and for the --onlyinitrd run of geninitrd, and holds it
until it exits.
The test runs both detect_dhcpd scripts with a fake tcpdump, and its
capture cases need private mount and network namespaces. It now lives in
xCAT-test/native with the other namespace tests, and the pull-request
workflow runs it in its own step. The capture cases keep their own unshare
call, because they add a default route on the loopback interface.
The unit test ran the Debian init helpers in the host shell, with fake
ucf, update-rc.d, systemctl and mv first in the host PATH. It now runs them
with sandbox_run in xCAT-test/native, with the compatibility root apart from
the /etc of the sandbox, and the Debian package lifecycle step runs it.
The unit test sourced scriptlib in the host Bash, with a fake nmcli first
in the host PATH. It now runs the helper with sandbox_run in
xCAT-test/native, and the pull-request workflow runs it in its own step.
State that on Subiquity installs the partition definition file or script
provides a curtin storage section that replaces the default layout, that
a layout without a match is restricted to the disk a disk entry selects,
that a failed layout or disk entry stops the install, that the first disk
of a disk file is used, and that a disk script runs in the live installer
under dash. Name the file that the disk file and the disk script write as
/tmp/xcat.install_disk, which both Ubuntu installers read, instead of
/tmp/install_disk.
A layout without a match that follows a disk file gets a match on the
disk the file selects, resolved when it is a link, and a use_gap layout
that follows a disk file stops the install. A layout with its own match
and a layout without a disk entry are kept as written.
A Subiquity layout written with "layout:" and no "match" goes to the
largest disk. With a d: or s:d: entry in linuximage.partitionfile, that
disk need not be the one the entry selected, so the install could wipe a
disk the entry was meant to keep it off.
When a disk entry is configured, give such a layout a match on the
selected disk. Refuse a use_gap layout in that case, because some
Subiquity releases ignore match in use_gap mode and would install into
free space on another disk. A layout with its own match, a "config:"
layout and a layout without a disk entry are kept as written.
A disk script or layout script that fails, or that writes no disk or
layout, and a disk file or layout file that holds only blank lines,
comments or an empty storage section each stop the pre-install script and
leave no storage config for Subiquity. A layout file that starts a YAML
document still leaves autoinstall.yaml one document.
A disk script or layout script that failed, or a disk or layout entry that
held nothing usable, left the disk getinstdisk chose or the default layout
in place, and the install went on to wipe that disk. A layout whose
storage section is missing or empty is worse: Subiquity then puts LVM on
the largest disk. The entry exists to keep the install off some disks, so
these fallbacks can destroy the disk it protects. A valid layout that
started a YAML document also broke the install, because it was appended
to autoinstall.yaml as a second document.
When a disk or layout entry is configured, require it to succeed and to
write a disk name, or a storage section with a config or a layout, and
stop the pre-install script with an error otherwise. Subiquity then stops
before it partitions. Rewrite an accepted layout as a YAML fragment that
holds only the storage section. Without an entry, the detected disk and
the default layout are used as before.
A layout file followed by a disk script and a layout script followed by a
disk file each replace the default storage config with the administrator
layout. A layout file after a layout script and a disk file after a disk
script are read as files, on Subiquity and in the debian-installer recipe
step.
linuximage.partitionfile takes a comma-separated list of layout, d: disk
and c: configuration entries, any of them prefixed with s: for a script.
The Ubuntu layout step chose between decoding a layout file and running a
layout script by the s: prefix of the last entry in the whole list, so a
layout file followed by an s:d: disk script was run as a script, and a
layout script followed by a d: disk file was copied into the layout as
text. The parser also kept a script flag set by an earlier entry of the
same kind after a later entry replaced it.
Give each kind the prefix of the entry that is used, the last one of that
kind, and decode the Ubuntu layout by it.
Render the node's pre-install script through the Template entry point and
run its partitioning under dash. Without a partitionfile the storage config
keeps the disk getinstdisk chose. A disk file puts its first disk there,
resolved when it is a link, a disk script puts the disk it writes there,
and a layout file or layout script replaces the default storage config.
linuximage.partitionfile had no effect on Ubuntu 20.04 and later installs.
The Subiquity pre-install script decodes each entry with base64decode,
which uses read -n, and Subiquity runs the script under dash, which has no
read -n. A layout file decoded to an empty storage config, which stops the
install, and a layout script or disk script decoded to nothing. The d: and
s:d: disk step also ran after the script had written the storage config,
so a disk file could not change the disk that getinstdisk chose, which
prefers a disk that already holds an operating system.
Decode with coreutils base64, which the live system ships, and run the
disk step before the storage config is written. The storage config uses
the first disk the disk file names, because the layout uses one disk, and
names it by its device node: Subiquity matches the path literally and
finds no disk for a /dev/disk/by-id link.