2
0
mirror of https://github.com/xcat2/xcat-dep.git synced 2026-09-30 14:55:17 +00:00
Commit Graph

1126 Commits

Author SHA1 Message Date
Daniel Hilst 67a80db63a Merge pull request #77 from xcat2/feature/openeuler-lts
feat(os): build native openEuler LTS dependency repositories
2026-09-29 11:59:05 -03:00
Daniel Hilst b1880d042d Merge pull request #74 from VersatusHPC/fix/riscv64-net-dns-key-record
fix(xcat-dep): riscv64 ships a Net::DNS without the KEY record
2026-09-29 11:16:26 -03:00
Daniel Hilst fc0f0a198d test(xcat-dep): the Net::DNS tests pass on 1.56 and on a tarball out of sync
The review asked for Net::DNS 1.57 with the tarball, the spec and the
manifest pin in sync. No test held either. A 1.56 tarball passed every
check, although 1.56 recurses without bound when it re-encodes a reply
with a misplaced TSIG record (rt.cpan.org #181125, fixed in 1.57). A
second Net-DNS tarball beside the spec, or a Buildnote that names
another release, also passed.

Decode such a reply with the shipped Net::DNS and re-encode it, with
Packet::encode wrapped to stop and record recursion past 20 levels.
Also check that perl-Net-DNS/ holds only the tarball the spec builds,
and that the Buildnote names that tarball.

The recursion check fails on 1.56. The two sync checks fail on a tree
that keeps Net-DNS-0.80.tar.gz and the 0.80 Buildnote.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-29 10:59:17 -03:00
Daniel Hilst e125f6f79f fix(xcat-dep): the riscv64 Net::DNS carries CVE-2026-64193 and CVE-2026-64194
perl-Net-DNS built Net::DNS 1.47 without patches. That release decodes
the EDNS EXTENDED-ERROR text with a string eval (CVE-2026-64193) and
follows compression pointer chains of any depth (CVE-2026-64194).
Net::DNS 1.56 fixes both, and 1.57 also stops an unbounded recursion on
a misplaced TSIG.

Build Net::DNS 1.57. The tarball, the spec, the Buildnote and the
riscv64 manifest pin move together. The build requirements of 1.57 are
the same as those of 1.47.

t/net_dns_rr_types.t decodes a reply with a 200-pointer chain and fails
on 1.47.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-29 08:46:22 -03:00
Daniel Hilst 398f3703f3 test(xcat-dep): capture CVE-2026-64194 in the shipped Net::DNS
perl-Net-DNS builds Net::DNS 1.47. That release decodes a reply whose
owner name is a chain of 200 compression pointers, one recursion per
pointer (CVE-2026-64194). Net::DNS 1.56 stops the chain with "deep
compression recursion".

The test decodes such a reply with the Net::DNS from the shipped tarball
and expects that error. It fails on 1.47.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-29 08:45:03 -03:00
Vinícius Ferrão a5346a21a7 Clarify native repository finalization and publication comments 2026-09-29 00:11:13 -03:00
Vinícius Ferrão 1eb7088e8c Remove unverified SCP validation requirements 2026-09-29 00:03:57 -03:00
Vinícius Ferrão 3f3289bc95 Test native repository locks and publisher signature preservation
Exercise mixed repository finalization and changed publisher inputs on x86_64. Check native manifest selection and use File::Slurper in the goconserver fixture.
2026-09-29 00:03:57 -03:00
Vinícius Ferrão 882ca33693 Fix openEuler repository finalization and manifest selection
Exclude native cells from cross-architecture Genesis finalization. Add the missing 24.03 SP1 manifest and expose repository signing for direct validation.
2026-09-29 00:03:57 -03:00
Vinícius Ferrão 5a625e94dd Merge master into openEuler support 2026-09-28 23:56:29 -03:00
Daniel Hilst 14b97879c6 Merge master into fix/riscv64-net-dns-key-record
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 13:15:45 -03:00
Daniel Hilst 4bb240a502 Merge pull request #75 from VersatusHPC/fix/repo-lock-force-unlock-race
fix(xcat-dep): per-arch dep builds share one repository lock
2026-09-28 13:05:17 -03:00
Daniel Hilst 38301490fa fix(xcat-dep): print the lock log line without changing autoflush
_log set $| on the caller's selected handle for each line. The caller
owns its buffering, so the lock log only prints the line.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 11:39:21 -03:00
Daniel Hilst f9b9f1ac46 feat(xcat-dep): log each NFS lock event to the build log
A build log did not show when a lock was taken, waited on, taken over
or released, so the lock order of the parallel dep builds could not be
read from their logs.

XCAT::NFSLock now prints one line per event to the selected output
handle, with the UTC time in milliseconds, the host, the pid, the event,
the label and the lock path. The events are acquired, took-over, wait,
released and release-skipped. quiet => 1 turns the log off for a lock.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 11:38:30 -03:00
Daniel Hilst 8823065ed9 fix(xcat-dep): publish and finalize take locks of cells they do not own
The apt publish took only the run lock of the host arch, and then read
the staging of every expected arch. A ppc64el run could refill its
staging while the publish assembled from it.

Finalize ran on one host and took the cell locks of both arches. If it
died, only that host could reclaim the locks of the other arch's cells,
and the next builds of those cells waited on them and failed.

A publish now takes the run lock of every expected arch, in name order,
before the publish lock, and waits for them up to --publish-lock-wait.
--finalize-arch limits finalize to the cells of one arch: it writes,
locks, re-indexes and verifies only those, and reads the other arches.
Without the option, finalize writes both arches as before.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 11:33:20 -03:00
Daniel Hilst 56631f8f36 test(xcat-dep): publish and finalize take locks of cells they do not own
The apt publish reads staging/<codename>/<arch> of every expected arch,
but takes only the run lock of the host arch. A ppc64el run can refill
its staging while the publish assembles from it.

Finalize runs on one host and takes the cell locks of both arches. If it
dies, only that host can reclaim the locks of the other arch's cells,
and the builds that own those cells wait on them and fail.

These tests fail until the fix: the publish must refuse while any
expected arch holds its run lock, and finalize --finalize-arch must
write and lock only the cells of that arch.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 11:28:47 -03:00
Daniel Hilst f23a4e8786 fix(xcat-dep): lock takeover relies on rename and on lock.break
XCAT::NFSLock recovered a dead owner's lock under a second lock,
lock.break. A process that died after it created lock.break left it in
place, and every later acquire timed out even when both processes were
dead. The lock and its record were also put in place with rename, which
replaces an empty directory, so acquire took over a lock.d that had no
owner record.

The lock now follows the NFS lock protocol at the top of NFSLock.pm. It
uses mkdir, rmdir, unlink and plain writes, and no rename. lock.d/metadata
holds the owner identity and a hash, and invalid metadata only causes a
retry. Takeover and release both hold lock.borrow, beside lock.d. Only a
process on the owner's machine takes over a lock, after it reads the
same identity twice and proves that owner dead. acquire retries R times
with a wait of T +/- jitter, and --try-unlock-timeout maps to R.

Records written by the previous format are invalid metadata and are not
taken over. A process that dies while it holds lock.borrow still blocks
the lock. The protocol assumes that this does not happen.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 10:06:35 -03:00
Daniel Hilst bd7312690d Merge master into fix/repo-lock-force-unlock-race
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-28 08:41:19 -03:00
Vinícius Ferrão 1d071b75aa Merge pull request #78 from VersatusHPC/feat/ipxeboot
feat(ipxe-xcat): package the upstream iPXE release binaries
2026-09-27 18:27:17 -03:00
Vinícius Ferrão a35bb1037b test(ipxe-xcat): check the payload with the shims in place
payload.sha256 differs from the bare release tree in the two shims, and
matches the tree once the committed ipxe/shim shims replace them, as the
builders install them. SHA256SUMS covers the shims with the archives.
2026-09-27 12:01:46 -03:00
Vinícius Ferrão 9d944ee3c7 build(ipxe-xcat): install the shims signed by both UEFI CAs
The rpm and the deb install ipxe-shimx64.efi and ipxe-shimaa64.efi over
x86_64-sb/shimx64.efi and arm64-sb/shimaa64.efi of the release tree, so
firmware that trusts only the Microsoft UEFI CA 2023 loads the shim
with Secure Boot on. The ipxe-shim.efi and snponly-shim.efi links keep
their targets, and every other file of the tree is unchanged.
payload.sha256 lists the digests of the new shims, which both builders
check. The README says how to update them.
2026-09-27 12:01:19 -03:00
Vinícius Ferrão 0ff70ab9a6 test(ipxe-xcat): stop comparing the payload with the bare release tree
The next commit installs the ipxe/shim 16.1 shims over the shims of the
release tree, so payload.sha256 no longer describes the bare tree. The
commit after it checks the tree with the shims in place.
2026-09-27 12:01:19 -03:00
Vinícius Ferrão 4b5e9d1e51 build(ipxe-xcat): add the ipxe/shim 16.1 shims signed by both UEFI CAs
The shims in the iPXE v2.0.0 release tree carry only the Microsoft UEFI
CA 2011 signature, so firmware that trusts only the UEFI CA 2023 refuses
them with Secure Boot on. ipxe-shimx64.efi and ipxe-shimaa64.efi are the
ipxe/shim ipxe-16.1 release assets that ipxe replaced on 2026-05-27 with
a build signed by both CAs. SHA256SUMS holds the digests that GitHub
publishes for them.
2026-09-27 11:59:36 -03:00
Daniel Hilst 429c573626 fix(xcat-dep): per-arch dep builds share one repository lock
The per-arch runs of one dep build share --repo-dep and each took
<repo-dep>/.lock. The first run won and the others died, so the
pipeline passed --force-unlock. That option removed any lock it found:
a rerun of the same refs could take a cell from a run still writing it,
and common recovery could remove the staging tree of a live publisher.

mockbuild-all.pl now takes XCAT::NFSLock locks: <output>/.lock, one
<repo-dep>/rh<N>/.<arch>.lock per target, and .common-publish.lock while
it recovers or publishes common. --finalize-xcat-dep locks the cells it
rewrites. --try-unlock-timeout N replaces --force-unlock. A run that is
not a dry run recovers an interrupted common publication when no other
run holds the common lock.

sbuild-all.pl takes its per-arch run lock and its apt publish lock as
XCAT::NFSLock locks instead of flock, which fails on the shared tree.

createrepo_c runs without --database. SQLite needs locks, which a client
of an NFS re-export cannot take.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-26 08:39:49 -03:00
Daniel Hilst e2bc5493de feat(xcat-dep): add XCAT::NFSLock, a lock for the shared NFS tree
The dep builds lock trees on a shared NFS tree that the hypervisors
re-export. The kernel refuses flock and fcntl locks to clients of a
re-export, so every such lock fails with errno 524.

XCAT::NFSLock builds a lock as a directory under a private name, with an
owner record and the caller's metadata, and renames it into place. The
record holds the machine id, boot id, pid, process start time and a
random token. The owner removes the lock, or a process on the owner's
host that proves the owner dead: another boot, no such pid, or a pid
with another start time. That process takes a breaker lock and proves
the owner dead again before it removes the lock. Removal renames the
lock away first. Any other acquire waits up to a timeout, retrying every
3 s by default, then fails with the mv command that moves the lock away.
A lock left by mockbuild-all.pl before this module is taken when its
pid is gone on this host.

t/nfslock.t covers each rule without NFS.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-26 08:39:49 -03:00
Vinícius Ferrão b92423c9c0 docs(BUILD): list ipxe-xcat with the x86 boot loaders
Name the ipxe-xcat builder and add ipxe-xcat where the EL and Debian
sections list the noarch x86 boot loaders.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão 35ffbd5c21 test(xcat-dep): expect ipxe-xcat with the x86 boot loaders
Require ipxe-xcat in every EL target that lists xnba-undi, at the same pin
on riscv64 as on ppc64le, and on every Debian target. Hold its Debian pin
to its changelog, and check that its control file keeps it an
Architecture: all package built on amd64 only.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão 99c4120161 build(xcat-dep): build and publish ipxe-xcat with the x86 boot loaders
mockbuild-all.pl builds ipxe-xcat in every EL profile, riscv64 included,
in the noarch chroot like the other x86 loaders. sbuild-all.pl maps it to
its builder and builds it once on amd64, as an Architecture: all package.

Both manifests list ipxe-xcat wherever they list xnba-undi, so the
repository gate requires it in every EL and Debian repository that
carries the x86 boot loaders. A riscv64 or ppc64le management node serves
those loaders to the x86 nodes of a mixed cluster.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão 9de804526c build(ci): check and test the ipxe-xcat package
Compile the ipxe-xcat builders and payload check, parse the spec, run
perlcritic on the new scripts and test, and run the payload test.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão 6d37819cec test(ipxe-xcat): cover the payload check
Damage a copy of a fixture tree one way at a time: a changed byte, a
missing file, an extra file or directory, a symlink with another target,
and a symlink replaced by a copy of its target. Each must fail the check
and name the path. Also hold the committed payload.sha256 and SHA256SUMS
to the committed archives.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão c56e943dc3 build(ipxe-xcat): package the upstream binaries for EL and Debian
ipxe-xcat installs the release tree under /tftpboot/xcat/ipxe as it
is, with its relative symlinks, and installs the source archive and the
licence texts with the documentation. It is a noarch RPM and an
Architecture: all deb.

Both builders check the archives against SHA256SUMS before the build.
After the build they unpack the RPM or deb and compare its tree with
payload.sha256, entry for entry, before the package reaches the result
directory. The spec and the Debian rules do not strip or compress the
tree, so the signed EFI files keep their signatures.

The package does not obsolete, provide or conflict with xnba-undi.
2026-09-25 13:15:50 -03:00
Vinícius Ferrão 0e8e044fa3 build(ipxe-xcat): add the iPXE v2.0.0 release and source archives
ipxeboot-2.0.0.tar.gz is the ipxeboot.tar.gz asset of the iPXE v2.0.0
release, renamed, with the SHA-256 that the release publishes.
ipxe-2.0.0-source.tar.gz is the source archive of tag v2.0.0 (commit
12798ec). The loaders xCAT uses are GPLv2+ as a whole, so their source
ships with them.

licenses/ holds the iPXE licences from v2.0.0 and the notices of the
signed shim: its COPYRIGHT from ipxe/shim ipxe-16.1, the OpenSSL
1.0.2k licence, and gnu-efi README.efilib at the commit that tag pins.
2026-09-25 13:15:49 -03:00
Daniel Hilst 234af100c7 fix(xcat-dep): no CI job runs the openEuler native packaging tests
t/openeuler.t and five of the six native/ tests are never run by the
workflow, so the openEuler build targets, repository layout and
package signing they cover are untested on every pull request.

Run t/openeuler.t with the other package tests. Add a step that
relaxes the AppArmor restriction on unprivileged user namespaces,
installs rpm2cpio and cpio, and runs the five native tests that need
only the RPM tools. A native file that skips fails the step, because a
missing tool would otherwise pass silently. native/mock-configs.t
needs Fedora's Mock library and is left out.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-25 12:16:45 -03:00
Daniel Hilst 7aa0ae858f fix(xcat-dep): a bail-out in one test file stops the whole suite
BAIL_OUT stops prove for every test file, not only the one that calls
it. Twelve calls in five test files now use die, which fails only its
own file.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-25 11:40:39 -03:00
Daniel Hilst 2a47e63cdd fix(xcat-dep): riscv64 ships a Net::DNS without the KEY record
Net::DNS 0.80 leaves the KEY record to Net::DNS::SEC, which xcat-dep
does not build. On riscv64, makedns fails when ddns.pm builds its
update key.

perl-Net-DNS now builds 1.47, the release EPEL 10 ships, and the
riscv64 manifest pins it. The spec drops --noxs, which 1.47 rejects,
passes --noonline-tests, and excludes the false perl(OS_CONF) Requires.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-25 11:40:39 -03:00
Daniel Hilst d7acc0b13c test(xcat-dep): capture the missing KEY record in the riscv64 Net::DNS
The ddns plugin builds its update key with Net::DNS::RR->new(... IN KEY
...). The Net::DNS 0.80 that xcat-dep builds for riscv64 has no KEY
record, so the call dies and makedns returns non-zero.

t/net_dns_rr_types.t loads Net::DNS from the shipped tarball and builds
the KEY records that xCAT builds. It also requires the manifest pin to
be an exact version at 1.01 or newer. The package-tests workflow runs
the test.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
2026-09-25 11:40:39 -03:00
Vinícius Ferrão 0ba9df8dc8 test(xcat-dep): cover the shared mock config resolver
An AlmaLinux host resolves to the alma config file, an id that names its
file resolves to it, the os-release id wins when both files exist, and a
release with no config file is an error that names the files it tried.
2026-09-25 02:25:44 -03:00
Vinícius Ferrão f8f77db1d6 refactor(xcat-dep): share the mock config resolver
mockbuild-all.pl finds the mock config by its file in /etc/mock, and
knows that /etc/os-release says almalinux where mock names the file
alma. Move that resolver into MockBuildUtils, with the config directory
as an optional argument, so a per-package builder can use it. The
behavior of mockbuild-all.pl does not change.
2026-09-25 02:24:42 -03:00
Vinícius Ferrão 607768548c test(build): call repository target mapping directly
Replace the openEuler verifier subprocess fixture with direct path-to-target assertions. Cover EL paths, trailing slashes and unrecognized inputs.
2026-09-25 00:02:04 -03:00
Vinícius Ferrão b89dc6fa46 refactor(build): share repository target inference
Move repository path mapping into MockBuildUtils so callers can use it directly. Keep the openEuler and EL target rules unchanged.
2026-09-24 23:59:07 -03:00
Daniel Hilst 329856a6cd Merge pull request #76 from VersatusHPC/fix/riscv64-goconserver-cross-build
fix(xcat-dep): the riscv64 goconserver build deadlocks running the Go compiler under emulation
2026-09-24 15:26:00 -03:00
Daniel Hilst 8816b1cfba Merge pull request #73 from VersatusHPC/fix/ubuntu-genesis-native-deb
fix(xcat-dep): the Ubuntu Genesis image is converted from the EL rpm
2026-09-24 15:19:31 -03:00
Vinícius Ferrão 41e7dc90f1 docs: document native packaging tests 2026-09-22 23:09:27 -03:00
Vinícius Ferrão a41b0bd684 test: simplify openEuler native fixtures 2026-09-22 23:09:27 -03:00
Vinícius Ferrão 724602f485 fix(openeuler): integrate current dependency build and signing paths
Merge current master into the openEuler dependency branch. Preserve native
inputs and publisher signatures with the upstream timeout and keyring
helpers. Use the native repository path when carrying skipped builds.

Cover signed carry-over and mixed publisher/build-key verification.
2026-09-22 19:33:13 -03:00
Vinícius Ferrão 69b6bf1603 test(build): cover relative Genesis signing keyrings from the parent directory 2026-09-22 03:06:08 -03:00
Vinícius Ferrão aa48f53730 fix(build): resolve native Genesis signing paths before changing directories 2026-09-22 03:06:08 -03:00
Vinícius Ferrão 75117dc3cc test(build): cover native dependency signing and Genesis key forwarding 2026-09-22 02:57:09 -03:00
Vinícius Ferrão 4b79eaa463 fix(build): require signing for native openEuler dependency publication 2026-09-22 02:57:09 -03:00
Vinícius Ferrão face1c8328 docs(openeuler): describe native dependency inputs 2026-09-21 18:24:49 -03:00