Clean against the master that now carries xcat-core/#7968, #7970, #7971, #7975,
#7978 and #7980. #7968 touches buildrpms.pl and release_repository.t as this
branch does, in disjoint regions -- the locking sites here, createrepo_dir,
index_repo and sign_repo_dir there -- which a review of #7968 measured before it
landed.
The one conflict is the BuildUtils import list: master gained package_skip_reason
with xcat-core/#7978 and this branch adds acquire_lock_dir. Resolved as the union
of the two, keeping the newer POSIX comment -- this branch's abort_builds() is
what needs WNOHANG.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Drive the Kea reservation builder and the ISC makedhcp path for grub2
nodes of each architecture. Place the loader links in the site TFTP
directory or in noderes.tftpdir. An aarch64 node with only grub2-<node>
keeps that name. The table stub returns only the requested columns, so
a missing nodetype.arch or noderes.tftpdir read fails the aarch64 cases.
Drive grub2 setstate for the node loader links: both names for an
aarch64 node and one for the others. Cover a repeated nodeset, a node
with only the old link, and removal in the boot, offline and SLES 15
standby states.
Some aarch64 UEFI firmware ignores a DHCP boot file name that does not
end in .efi and requests grubaa64.efi from the TFTP root instead. xCAT
names grub2-<node>, so these nodes stop with a TFTP error before grub2
runs.
For an aarch64 node, nodeset now links grub2-<node>.efi to
grub2.aarch64 beside grub2-<node>. Both DHCP backends name the .efi
link once it exists in the node's TFTP directory (noderes.tftpdir, or
the site one). A node set by an older xCAT keeps grub2-<node> until its
next nodeset, so makedhcp after an upgrade does not name a missing
file. With site.sharedtftp off, every server that runs nodeset for the
node creates both links. This relies on service nodes running the same
xCAT version as the management node, which xCAT requires.
The old link stays for DHCP configurations that makedhcp did not
write. The boot and offline states remove both links. Other
architectures keep grub2-<node>.
Resyncs build-utils/lib/XCAT/NFSLock.pm with xcat-dep, where the defect was
found and fixed.
release deleted the owner metadata and marked the guard released before it
knew the removal could finish. A build lock with a live worker group
therefore lost its metadata when the rmdir failed, and every later build
refused it for ever rather than recovering it when the previous build died.
The sweep had the same ordering and left the same wreck.
Both clear the records of completed workers first and delete the metadata
only once the directory holds nothing else. release retries while a worker
blocks the removal, so cancel_build's retention finishes the release when
the group finally goes.
Depends on xcat-dep/#93, which carries the same change to the original.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xcat-dep-build-opensuse15-x86_64 fails its xCAT-genesis-base step. BuildRequires
resolve, and verify-genesis-payload then reports 566 missing zoneinfo/posix paths,
etc/redhat-release, and 45 EL paths and tool names -- net-tools-deprecated,
kmod-compat, iputils, systemd-sysvinit, timezone, nc, mkdosfs and the lvm symlinks.
The manifest is written for EL, so the package cannot be built on openSUSE.
openSUSE ships no legacy Genesis, so it does not need to be. package_skip_reason names
the pairs a target does not build and why, buildrpms.pl prints each skipped pair, and a
plan that skips everything returns 0 rather than failing: the dep pipeline asks for
xCAT-genesis-base alone, so on openSUSE its whole plan is empty and an empty plan is
the plan.
Dropping the package alone would make xCAT unresolvable there.
xCAT-genesis-scripts.spec requires xCAT-genesis-base-%{tarch} unconditionally and xCAT
requires xCAT-genesis-scripts, so zypper install xCAT would fail on a missing package
-- later than the build and looking like something else. That Requires now stands
behind %if ! 0%{?suse_version}; the Leap chroot's /usr/lib/rpm/suse/macros sets
%suse_version 1500, and no EL or openEuler target defines it.
xCAT-test/unit/opensuse_no_legacy_genesis.t covers the skip, the empty plan and the
conditional Requires.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Nothing holds the openSUSE Genesis scope down. xCAT-genesis-base is built for
every target the dep pipeline names, and xCAT-genesis-scripts requires the
payload on every family.
The test reads the build-plan decision and the payload requirement in the spec.
It fails on this commit: the build plan keeps xCAT-genesis-base for openSUSE,
and the requirement carries no condition.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Execute the complete postscript and check the daemon UID before real UDP
log delivery. Cover fresh and repeated MN/SN setup, restricted directories,
legacy ownership and modes, and a blocked directory path. Keep the upgrade
wrapper and forwarding cases as compatibility controls.
Select older configuration branches through their version-report boundary;
use the installed daemon, without claiming old-daemon qualification.
Run the isolated native cases in CI.
Create missing Debian-family log directories before assigning their owner.
Use mode 0750 for syslog-owned directories. In the legacy reference-owner
path, copy the parent directory mode so group-writing daemons retain access.
Preserve existing directory modes, log files and forwarding policy.
Stop with an error before restarting syslog if the directory cannot be
created. Keep existing ownership-command failures nonfatal.
Run routeop against real kernel routes. Cover repeated add, replacement,
deletion, another device, expanded addresses and failure preservation.
Check SLES, legacy Red Hat and Debian persistent records, including
unscoped records and neighboring interface names. Keep global-gateway
and connected-route controls.
Scope generic Linux route lookup and updates to the supplied interface for
link-local next hops. Retain the device in SLES routes, legacy Red Hat
route6 records and Debian interface commands. Remove matching saved
records without matching neighboring interface names, and migrate older
unscoped records when adding routes.
Keep global gateways, connected routes and NetworkManager policy unchanged.
BuildUtils::sh() runs each build command in its own process group, so
SIGKILL on the wrapper leaves that group running. The lock records the
wrapper, so the next build proved the wrapper dead and took the lock, and
two builds then wrote one checkout and one mock target. cancel_build keeps
the lock on purpose when a group survives its cleanup, and that recovery
undid the retention.
sh() now records the group on every lock this process holds, before it
restores the signal mask, so a cancellation cannot land between the fork
and the record. It drops the record after the group is reaped.
cancel_build drops it only when kill(0, -pgid) answers ESRCH, which is the
test it already used to decide it may release; on the retention path the
record stays, and a later acquire reads it and refuses the lock.
The vendored XCAT::NFSLock carries the rule: an owner is recoverable only
when it is provably dead and every attached group is gone. detach refuses
while any process is in the group, so the retention does not depend on
which call site remembered to run.
Depends on xcat-dep/#93, which carries the same change to the original.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
BuildUtils::sh() runs each build command in its own process group, so
SIGKILL on the wrapper leaves that group running. A lock recovered from the
dead wrapper then lets a second build into the same checkout, and
cancel_build's retention is undone by the same recovery: it keeps the lock
when workers survive, and the next build reads a dead owner and takes it.
build_lock_stale_owner.t covers a wrapper and nothing below it. The name
reads wider than the check: the holder it kills has no build worker, so the
worker case was never measured. Its comment says so now.
A new check is the reviewer's case. A worker runs in its own process group,
a wrapper takes the build lock and records it, the wrapper alone is killed,
and a second build must stay blocked. Then the worker exits and the same
build must take the lock: without that half, a lock that recovers nothing
would pass.
The handshake is two one-byte reads on separate pipes, each bounded. One
read of two bytes made it a timing test, which passed on this host and
killed the wrapper before it held the lock.
The vendored XCAT::NFSLock gains attach, detach and live_workers, which
these checks drive.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Resyncs build-utils/lib/XCAT/NFSLock.pm with xcat-dep, where the defect was
found and fixed. The module reclaimed a stranded lock.borrow by path after
validating its metadata, and rmdir names a path and not the inode that was
validated, so a contender stopped between the two steps removed whatever
was at the path when it resumed. Two builds of one target could then both
hold the build lock, which is the collision the lock exists to prevent.
The borrow is numbered now, and a transition whose taker is provably dead
is ended by creating the next generation rather than by removing the one
that was validated. A process removes only a generation it created and
still holds.
acquire_lock_dir's comment said the retry budget covers reclaiming a borrow
with no metadata over two attempts. That rule is withdrawn, because a live
taker can pause for any length of time before its first write. A take-over
needs one attempt; the budget leaves two for a contended lock.
Depends on xcat-dep/#93, which carries the same change to the original.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs.pl describes the regression it fixed by quoting "another build
already holds". acquire_lock_dir prints "another build still holds" now,
and only when the retry budget ran out, so the quoted text names nothing a
reader can find.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
acquire_lock_dir dies with "another build of <what> already holds
<lockdir>" whenever XCAT::NFSLock->acquire fails, whatever the failure was.
A lockdir under a directory that does not exist, or one the build cannot
write, both tell the operator to look for a build that is not running, and
both print the advice about --mock-uniqueext. The nested text said "Trying
to unlock", which is the reverse of what acquire was doing.
acquire_lock_dir now reports contention only when the retry budget ran out,
and prints the error that happened otherwise. The vendored
XCAT::NFSLock is resynced with xcat-dep: its refusal names which failure it
was, and it reclaims a lock.borrow that has no metadata and that two
attempts find unchanged, which was a permanent strand. A lock.d with no
metadata stays refused, so a directory left by a build from before this
change is not recovered by acquire; sweep_stranded removes that one, and
only under an exclusive claim on the root.
buildrpms.pl kept "use POSIX (); # EEXIST, for the directory build lock",
and the lock has not used EEXIST since it became an XCAT::NFSLock. The
import is still needed for WNOHANG in abort_builds.
Depends on xcat-dep/#93, which carries the same change to the original
copy of XCAT/NFSLock.pm.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Three tests match qr/already holds/ against the refusal that
acquire_lock_dir prints. That sentence is printed for every acquire
failure, so the checks pass whether the lock is held by a live build or the
mkdir failed with ENOENT or EACCES. They cannot tell contention from
anything else, which is the one thing the operator reads the message for.
Each of the three now asserts the contention report and the pid that holds
the lock. build_lock_stale_owner.t adds a lock under a directory that does
not exist, and asserts that the refusal carries the real error and claims
no concurrent build.
release_build_locks has no check at all. It is what a signal handler calls,
because DESTROY does not run when a signal ends the process, and it reaches
a lock only while that lock is in the registry. One check now takes a lock
and frees it through release_build_locks.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The build lock is an XCAT::NFSLock, so buildrpms.pl loads
build-utils/lib/XCAT/NFSLock.pm. Four tests copy buildrpms.pl and
BuildUtils.pm into a work tree and run the build there, and none of them
copies NFSLock.pm. release_repository.t therefore aborted at check 46 with
"Can't locate XCAT/NFSLock.pm in @INC", and xcat_pr_test failed on it.
buildrpms-namespace.sh bind-mounts a small /etc over the real one, and the
lock needs /etc/machine-id for its ownership identity. Without it the build
died with "Cannot read /etc/machine-id: an NFS lock needs a machine id".
Stage NFSLock.pm in each of the four work trees, and copy machine-id into
the namespace /etc beside passwd and group.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
SIGKILL and the OOM-killer run no handler, so a build killed that way leaves its
lock directory behind. Every later build of the same target then fails at mkdir
until an operator removes the directory by hand.
/var/lock/buildrpms.openeuler-20.03sp4-x86_64.lock.d on one build host was
created on 2026-10-14, held no owner file and no live process, and still failed
that target's dep build on 2026-10-21, one second into the genesis step. Every
other openEuler job in that run reported NOT_BUILT.
buildrpms.pl held its own lock, a bare mkdir that recorded no owner, and `use
autodie` wrapped that mkdir, so the EEXIST branch below it was dead code and the
run died on autodie's raw "Can't mkdir(...): File exists". That is why it never
refused anything and never named the build holding the lock.
XCAT::BuildUtils::take_build_lock kept a second hand-rolled lock with the same
gap.
Both now take the lock through XCAT::NFSLock, whose protocol recovers a lock
whose owner is provably dead. The lock stays in /var/lock, which is local,
because what it guards is host-local -- /etc/mock/<cfg>.cfg, /var/lib/mock/<cfg>/
and its buildroot.lock -- so a lock on the shared checkout would serialise the
per-arch builds of one checkout that run on two hosts and share nothing.
Depends on the matching xcat-dep change: that copy of NFSLock carries the
recovery of a borrow whose taker died, and a DESTROY, without which a guard
never releases at scope exit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The build lock is a directory created with mkdir, which cannot release itself. A
build killed with SIGKILL or by the OOM-killer leaves it behind, and every later
build of the same target fails until an operator removes it by hand. No test
covered that case.
These checks take the lock in a child, confirm a second build is refused by name
while the child lives, kill the child with SIGKILL, and assert the lock is
recovered. One more holds the boundary that recovery must not widen what a lock
excludes: a build of a different checkout still runs beside it.
XCAT::NFSLock arrives with them, vendored at build-utils/lib/XCAT/NFSLock.pm
beside BuildUtils.pm, because xcat-core builds with no xcat-dep checkout. Its
header names the other two copies and records that the three must stay
identical. Nothing calls it yet, so these checks fail here.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Document the restricted certificate policy, required tools and cleanup limits. Distinguish the fixture from the default certificate-free postscript path.
Exercise certificate policy and real SSH copies in a disposable MN/SN/node fixture. This test is opt-in and skips in ordinary CI without XCAT_SYNCFILES_LIVE_CONFIG.
Keep the original client name for the rejection diagnostic. Use the resolved
node only for dispatch, without changing identity or copy-command handling.
Execute each profile through /bin/sh. Check both firewalld entries as
regular files, valid links, dangling links, directories and missing paths.
Preserve unit targets, unrelated links and the older profiles without this
cleanup.
Execute all affected owners and their aliases in isolated image roots. Check
policy contents and metadata, missing files, repeated runs and host
sentinels. Run the cases in Ubuntu and EL CI.
Match the complete enforcing or permissive setting. Preserve comments and
whitespace, and do not change unrelated lines or create a missing policy
file.
RPM signing can change bytes without changing the file size or whole-second
modification time used by the metadata cache. Rebuild binary and source
indexes after signing so their checksums describe the final RPMs.
Keep incremental indexing for unsigned builds.
Run the complete discovery postscript in the shared shell sandbox. Keep the
key-generation, exit-status and listener side-effect checks. Preserve the
dedicated CI step and its cancellation condition.
openEuler stateless images use the rh genimage and dracut modules,
which add the UFTP receiver, and xcat-dep now builds uftp for the
openEuler targets. List openEuler with the supported images and the
uftp package.
Describe how genimage, nodeset, the listener and the booting node work
together, the supported images, the cases that nodeset refuses, and
what a transfer trusts. Describe the listener setup, the firewall and
multicast requirements, service nodes, and the transfer rate and jumbo
frame settings. Add the rootimgtransport attributes of the site and
the osimage to the reference pages.
The export of an image must name the UFTP marker of its initial
ramdisk, and the import must place the marker next to the ramdisk. An
import from a bundle that names the marker but lacks it must fail and
leave the published files.