The test runs each changed xcatroot in a bubblewrap sandbox, with stub
curl, wget, logger, mount and sleep. An image that matches its checksum
must be extracted without a wait, also when a later line of the checksum
file holds it. An image that matches no line must not be extracted. A
download that began before packimage replaced the image must match the
checksum file fetched before it. A damaged download, a failed checksum
download and a checksum file without a checksum must each lead to one
more download. A file that an interrupted download left must be checked
before it is used. A missing checksum file and a missing sha256sum must
keep the node retrying. Without rootimgchecksum=yes, or with
rootimgchecksum=no, the image must be extracted unchecked.
xcatroot extracted whatever curl or wget downloaded, so a damaged
download became the root file system of the node.
After the download, xcatroot fetches the .sha256 file next to the image
and compares it with sha256sum. It also fetches that file before each
download. A download passes when it matches a line of either file, so a
download that began before packimage replaced the image still passes.
The check can fail for any reason, such as a missing checksum file or a
missing sha256sum. xcatroot then removes the download, logs the reason,
waits, and downloads the image again. A file that an earlier attempt
left is checked like a new download. xcatroot checks only when the
kernel argument rootimgchecksum=yes asks for it, and logs when it does
not check. An older xCAT server never passes the argument, so a new
initial ramdisk boots from it unchecked, as before.
The change covers the stateless modules of dracut 033, 047 and 105 for
the Red Hat family, dracut 033 for SUSE, and the Ubuntu module. These
modules also download statelite ramdisk images. Their initial ramdisks
now include sha256sum.
liteimg must take the image lock and keep the previous archive while it
packs and after a failed pack. It must publish the new archive with its
checksum, also when the check is turned off. It must change nothing
while the lock is held. With --tracker, it must name the given tracker
URL and describe the new archive under its final name.
liteimg deleted rootimg-statelite.gz before it packed a statelite
ramdisk image, and wrote the new archive in place. It ignored a failed
pack, so nodes could get no image or a partial one. It took no image
lock. --tracker took no value, so the metainfo named the tracker 1, and
it described rootimg.gz instead of the new archive.
liteimg now takes the image lock and packs into the temporary directory
that packimage uses. It fails when a command of the pack fails. It
writes the SHA-256 checksum, and publishes it with the archive through
publish_rootimg. --tracker takes the tracker URL, as it does for
packimage, and the metainfo describes the new archive. Nodes keep
downloading the previous archive until the new one replaces it.
An import of a cpio, tar or squashfs root image over a published image
must take the image lock that packimage takes. It must replace the
image, its checksum, the kernel and the ramdisk. It must keep the
previous image as .ORIG, remove the previous metainfo, keep other
formats, and leave no temporary directory. With the check turned off,
the import must still write the checksum. When the image lock is held,
the import must fail and keep the published image, checksum, kernel,
ramdisk and image definition. When the publication lock of another host
is held, or the imported image or its checksum cannot be published, the
import must fail. The published image, checksum and metainfo must then
stay, with their kernel and ramdisk.
imgimport moved the published root image away, then moved the copy from
the bundle in. When the bundle was on another file system, that move was
a copy. Nodes could get no image or a partial one. The checksum that
packimage had published for the previous image also stayed.
imgimport now takes the image lock that genimage, packimage and rmimage
use. It stages the imported image and its checksum in the image
directory. It publishes both with the packimage helper, under the same
publication lock. It writes the checksum whatever the rootimgchecksum
attribute of the imported image, as packimage does. The previous image
stays as .ORIG through a hard link, and its metainfo is removed.
imgimport also places the root image before the kernel and ramdisk now.
When it cannot publish the root image, it stops, so the kernel and
ramdisk keep matching the published image. It takes the image lock
before it imports the image definition, so an import that another image
command blocks changes nothing.
Each packed format must publish the SHA-256 checksum of the new archive
next to it, readable by nodes. The checksums of other formats must stay,
for nodes that already download them. A pack that fails while it
compresses, lists or reads the files must keep the previous checksum. A
pack must publish the checksum also when the image and the site turn the
check off. An image that turns the check on overrides the site, and 'no'
and '0' both turn it off.
While the archive is replaced, the checksum file must list the new and
the previous archive, also one packed before the check. The previous
metainfo must be gone by then. Then the checksum file must list only the
new archive. A failed rename of the checksum list, the archive or the
metainfo must leave the previous archive, checksum and metainfo, also
without hard links. When the previous archive can be neither linked nor
copied, nothing must change. A copy under umask 0077 must keep the mode
of the previous archive. A checksum file that keeps the previous line
must not fail the publication. A pack whose previous metainfo cannot be
moved aside must keep the previous archive and checksum. A previous
archive that cannot be put back must be kept outside the staging
directory, with its metainfo and checksum.
A node could not tell a damaged root image download from a good one,
because packimage published nothing to check it against.
packimage now writes rootimg.<suffix>.sha256 in sha256sum format next to
the archive. It builds the checksum in the temporary directory with the
archive, and publish_rootimg publishes both under the publication lock.
A pack that fails before publication keeps the previous archive and
checksum. When packimage removes the other formats, it keeps their
checksums, so a node that already downloads one can still check it.
Until the archive is replaced, the checksum file lists the new and the
previous archive. publish_rootimg computes the line of a previous
archive that has none. The published archive then matches a line of it
at each step, also when the publisher is killed. The previous metainfo
leaves first, so it never describes the new archive. publish_rootimg
keeps a link to the previous archive, or a copy with its mode without
hard links. When a step fails, the previous archive, checksum and
metainfo stay or come back. When it can keep the previous archive in
neither way, it publishes nothing. When it cannot put the previous
archive back, it keeps it in a .rootimg-previous-* directory and names
that directory in the error.
The new site.rootimgchecksum attribute is yes by default, and the
rootimgchecksum attribute of an osimage overrides it. They tell nodeset
whether nodes check the archive. packimage publishes the checksum
whatever their value, because nodes with older boot entries can still
check.
imgimport must replace a root image the way packimage does. Move the
host-scoped temporary directory and the locked publication from
packimage to xCAT::SvrUtils, so both commands can share them. The
publication still covers the archive and its metainfo. The behavior of
packimage does not change.
The publication lock moves to a helper that imgimport also uses, and its
message will name both commands. Match the holder and the recovery step
of the message instead of the command name.
The test also made publication fail with a directory in place of the
previous metainfo. Publication will move that metainfo aside, which
works for a directory. Make rename and unlink of that path fail instead.
Remove source-pattern and helper-call counts. Check key-file creation and
replacement through the real DDNS code, with file access redirected to a
temporary fixture. Keep the signing and named configuration checks.
Run the unchanged initrd builder with fixture binaries and inspect both generated archives. Replace source extraction with checks for selected libraries and their contents.
Read site settings from an isolated database and inspect the sources.list written into the image. Stop at an injected package-metadata failure without downloading packages.
Run genimage and inspect the arguments passed to debootstrap instead of evaluating an extracted assignment. Keep the existing ppc64le pass-through explicit.
Exclude node postscripts through the native RPM and dpkg options. Verify
that upgrade configuration, captured output and cleanup still complete.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
Generate self-contained RPM and Debian upgrade scripts from the shared
source. Package transactions no longer load functions from the node-facing
postscript payload. Keep the existing command, logging and failure policies.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The man page said that packimage overwrites the root image and that nodes
cannot use it while packimage runs. Describe the replacement instead, and
the disk space that it needs.
The test packs cpio and tar archives with gzip, xz and pigz, and squashfs
images, on the Red Hat path. A wrapper around the compressor, mksquashfs,
cpio or find records the file under the final name while the pack runs,
and then runs the real tool or fails.
It checks that nodes get the previous archive during the pack, that a
pack whose compressor, cpio or file listing fails keeps the previous
archive, metainfo and other formats, that a successful pack replaces
them, and that no temporary file is left. The next pack removes a
temporary file or directory that an interrupted pack on the same host
left, and keeps one that another host is writing.
For a StateLite tree, it checks that a pack that fails while listing the
files, compressing them or creating the temporary directory restores the
files that packimage switched, and that a retry keeps them. When a file
cannot be moved back, the original and the previous archive must stay,
and the next pack must refuse to run.
A pack must wait while another host holds .packimage-publish. When a
pack that did not finish left the lock, the next pack must fail, name the
holder, and keep the previous archive and the lock. With --tracker,
ctorrent must build the metainfo from the new archive under its final
name before the archive is replaced. A ctorrent failure must keep the
previous archive and metainfo, and so must a previous metainfo that
cannot be removed.
packimage deleted every rootimg.* file before it packed, and wrote the new
cpio or tar archive under its final name with mode 0600. Until the pack
finished, a node that booted got HTTP 403 for its image. When the pack
failed, the image was gone or left truncated, and no node could boot until
a later pack succeeded.
Build every archive in a temporary directory inside the image directory,
and rename the archive over the final name only after the pack succeeds,
as the openEuler path already did with a temporary file. With --tracker,
ctorrent builds the metainfo in the same directory, from the new archive
under its final name. The previous metainfo is removed before the archive
is renamed, and the new metainfo is renamed after it, so no failure
leaves a metainfo that describes another archive. The other rootimg.*
files are removed after the rename. The next pack on the same host
removes a temporary directory that an interrupted pack left. Image locks
are per host, so temporary directories of other hosts that share the
directory stay.
A pack now also fails when find cannot list the image or cpio cannot read
a file, as on the openEuler path. Before, only the compressor status
counted, and such a pack published an incomplete archive.
Every failure after packimage switches a StateLite tree to stateless files
now switches it back. That includes a failure to create the temporary
directory or the squashfs staging directory, which File::Temp reports
with die. packimage records each file that it moves and checks each move. When
a file cannot be moved back, the original stays in .statebackup,
packimage reports it, and the next pack refuses to run until the files
are restored. The new archive replaces the previous one only after the
StateLite files are back. Before, a failed pack left the tree switched,
and a retry overwrote the saved StateLite files.
The image lock is per host, and service nodes mount a shared image
directory without NFS locks. So a pack also holds the directory
.packimage-publish while it replaces the archive and removes the other
formats. mkdir is atomic for every host, also on NFS without locks. When
the lock is older than 60 seconds, or stays held for 60 seconds, the pack
fails, names the holder, and keeps the previous archive. packimage does
not take the lock over, because its holder can still be running.
A ctorrent failure now fails the pack, and the previous archive and
metainfo stay. Before, a failed ctorrent left a partial metainfo, and
nodeset pointed nodes at it.
The packimage tests skip their squashfs and pigz cases unless mksquashfs,
unsquashfs and pigz are installed. Install them in the unit test job so
that CI runs those cases.
Vinícius Ferrão
Daniel Hilst