`makedhcp -q <node>` on Ubuntu's ISC-limited releases answers "no DHCP reservation
found" when it cannot read dhcpd.conf. The operator reads that as a node without a
reservation. An InfiniBand node also gets an answer with no hardware address.
_query_isc_static_host in dhcp.pm read the file with an -r test and dropped a failed
open. It also matched only a "hardware ethernet" line, while _add_isc_static_host
writes "hardware infiniband" for an InfiniBand node and adds a twin declaration
between the same markers.
_read_isc_conf_lines now returns the read error, _query_isc_static_host returns it to
listnode, and listnode answers the caller with an error. The parser accepts any
hardware type and keeps the first declaration of the block. The path of dhcpd.conf and
the distribution name are package variables, so a test can drive the query and
listnode.
dhcp_isc_static_host_query.t covers the InfiniBand address, the twin declaration, the
unreadable file and the listnode answer. It fails without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The static host query reads dhcpd.conf when no configuration is in memory. When the
read fails the query returns nothing, and listnode reports "no DHCP reservation
found" -- the answer for a node that has no reservation. The operator cannot tell
the two apart.
The same query only reads a "hardware ethernet" line. An InfiniBand node declares
"hardware infiniband", so its query answer carries no hardware address. A twin
declaration inside the same markers must not replace the primary one either.
The new assertions drive the writer to build both InfiniBand shapes, set the path of
dhcpd.conf to a file that does not exist, and call listnode. A deletion that names a
hostname is asserted to keep the other declarations of the node.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Fall back to an available DHCP backend on auto-selection. When the request is "auto"
and the backend chosen for this OS is not installed, use the other one if it is,
recording fallback_from so process_request can tell the operator which preferred
backend is missing. A backend the admin forced through site.dhcpbackend still fails
hard when absent, and "neither installed" still errors clearly. Fixes#7710.
Answer makedhcp -q from the static host block on Ubuntu's ISC-limited releases.
listnode now branches on _isc_static_host_fallback() before any omapi work and reads
the node's fixed-address and hardware ethernet straight out of dhcpd.conf, so the
query path never spawns the omshell its own write paths already avoid. A node with no
reservation is now reported rather than answered with silence.
Match the host-block markers exactly. _add_isc_static_host writes a fully determined
pair -- "#xCAT host declaration for <node> aka host <hostname> start" and the "}"
line carrying the matching end -- so both scans anchor on that whole shape through
shared _isc_host_start_re/_isc_host_end_re helpers. The previous /\Q$node\E\b.*/ also
matched at a hyphen, letting node "compute" act on "compute-01"'s block: the query
could return another node's address and the delete could remove another node's
reservation. _delete_isc_static_host also accepts an explicit line list now, so the
scan is unit testable without file-scoped state.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
DHCP backend auto-selection fails hard when the backend it prefers is not installed.
The xcat metapackage's "isc-dhcp-server | kea" Depends guarantees isc-dhcp-server,
while auto-selection prefers kea on 22.04+, where kea arrives only through Recommends.
An apt run that skips recommends therefore leaves a node with isc only, auto picks the
absent kea, and makedhcp errors out on a freshly installed management node (#7710).
makedhcp -q re-enters omshell on the very releases whose write paths avoid it. xCAT
already records each reservation as a static host block in dhcpd.conf on Ubuntu's
ISC-limited releases because their ISC DHCP 4.4 omshell can wedge at 100% CPU and
never be reaped, but listnode still called _omshell_query_host unconditionally.
Both host-block scans also match the node name loosely: /\Q$node\E\b.* start$/ matches
at a hyphen, so "compute" matches the "compute-01" marker -- a query can answer with
another node's address and a delete can remove another node's reservation.
Cover all three: the auto-selection fallback, a query answered from dhcpd.conf without
omshell, and exact node-name matching for both the query and the delete.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Replace source-shape extraction with real preprocess and command execution. Model missing table rows with the same return shape as xCAT::Table.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
The module overrides two methods of XML::Simple, one for a recent version and
one for an older version, and each built its own parser and set its own
handlers. The two bodies were the same apart from spacing, so a change to one
refusal had to be repeated in the other, and a reader had to compare them to
see that they agreed.
Build the parser in one routine that both call. Behaviour does not change.
Cover the three shapes a declaration can take: one that names an entity from
element text, one that names it from an attribute value, and one that carries
no entity at all. The parser must refuse each, and nothing may expand.
The attribute case is the one that matters most. The option that stops the
parser expanding an entity leaves an attribute alone, so a test that only used
element text would pass against a parser that still grows a request through an
attribute.
Cover an ordinary request as well, so a refusal that is too wide is visible:
the command, the node range and the argument must still arrive.
Run all of it against the parser of a recent XML::Simple and against the parser
this module builds for an older XML::Simple.
The rebase onto master left the repository with two modules named
BuildUtils.pm: the shared build helpers at the root, package BuildUtils, and
the target architecture parser at build-utils/lib/XCAT/BuildUtils.pm, package
XCAT::BuildUtils. buildrpms.pl loaded both, one through `@INC` and one through a
path require. A reader cannot tell which module a BuildUtils reference names,
and the test sandbox staged the wrong one.
Move the shared helpers into build-utils/lib/XCAT/BuildUtils.pm as
XCAT::BuildUtils, and export targetarch_from_target beside them. Both builders
and the four tests now put build-utils/lib on `@INC` and import from the one
module. targetarch_from_target keeps its behaviour: it returns the same
architecture as before for suffixed targets, empty and undefined input, mixed
case and every architecture token.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Drive redact_directory_entry with a password on a continued COMMAND
record, in both spellings. Pin the records that do not end the
continuation: a sequence number, a blank record, a comment, and a record
that reads as another directory statement.
redact_directory_entry masked only the first record of a COMMAND
statement, and did not recognize the CMD spelling. A record ending with
a comma continues on the record below, so a password there reached
syslog and the client.
Mask every record of the statement, in both spellings. Read the
continuation comma in columns 1 to 71, so a sequence number does not
open or close a statement. A blank or a comment record does not end one.
buildrpms_source_only.t runs buildrpms.pl from a copied sandbox, and staged
only BuildUtils.pm. Master added a second module with the same basename,
build-utils/lib/XCAT/BuildUtils.pm, which buildrpms.pl also loads. The
sandboxed run then died with "Can't locate .../build-utils/lib/XCAT/
BuildUtils.pm" before it reached the argument parsing the test asserts on.
Stage each needed file at its own relative path and create the parent
directory, so both modules reach the sandbox.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Every in-tree caller now uses isValidIp. The old helper accepted any
value with a colon as a valid address, so there is no reason to keep
it as a wrapper.
The two builders wrote the same run-a-command-or-stop step in opposite
polarities. builddebs.pl used sh(...) == 0 or die; buildrpms.pl used
sh(...) and die, and also used the == 0 or die form for its sh_retry calls, so
both directions appeared in one file. The 'and die' spelling reads as though
the die is what happens next rather than what happens on failure, which is a
poor thing to have to re-read at every call site.
Add BuildUtils::sh_or_die and convert the fourteen plain sh() call sites to it.
The failure message now also carries the exit code, which every one of the old
spellings discarded -- a build that failed said only that a command failed, not
what it returned. The sh_retry sites keep their own form: retry is a different
operation and sh_retry is local to buildrpms.pl.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The two builders read and wrote files three different ways. buildrpms.pl used
File::Slurper, builddebs.pl opened and closed handles inline, and BuildUtils.pm
carried its own read_file/write_file pair to avoid the dependency. The result
was that the same operation was spelled differently in each place, and the
Version stamp was read by hand in both builders: buildrpms.pl chomped it ten
lines after reading it, which is how a stamp keeps a trailing newline nobody
notices until it reaches a package name.
Drop the hand-rolled pair and use File::Slurper throughout, and add
BuildUtils::read_line for the one-line Version and Release stamps -- it strips
the newline at the point of reading and reports an absent file as undef, which
is what builddebs.pl's fallback to snap_release needs.
File::Slurper is already required by buildrpms.pl and is already in the CI
package list, so this adds no new build-host prerequisite. The lock file still
uses a raw open: flock needs a real handle.
Behaviour note: Version is now read as its first line rather than the whole
file with the final newline removed. For the one-line file it is, these are the
same; a malformed multi-line Version now yields its first line instead of an
embedded newline.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs.pl stages the genesis bmcsetup and getipmi helpers into the xCAT
package by writing the rewritten text and then chmod'ing it, which is the third
copy of the write-then-make-executable pair the previous change collapsed. It
was left out because it uses 0755 while the published repo helper uses 0775.
Give write_script an optional mode, defaulting to the 0775 it already used, and
use it for the postscripts with 0755. The two modes stay exactly as they were
-- the postscripts are deliberately not group-writable -- and the test asserts
that the caller's mode is honoured rather than overwritten by the default.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders ship a mklocalrepo.sh beside the packages they publish, and each
wrote it its own way: builddebs.pl opened, printed and closed the file by hand
and then chmod'ed it, while buildrpms.pl used File::Slurper and a separate
chmod. Writing the text and setting the executable bit are one operation -- a
copy published without 0775 is published broken -- but nothing tied them
together, so each caller had to remember the second step.
Add BuildUtils::write_script, which writes the file and sets the mode, and use
it from both builders. The mode is now asserted in build_utils.t instead of
being left to the callers to repeat.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders wrote the same six fields beside a published repository --
VERSION, RELEASE, BUILD_TIME, BUILD_MACHINE, COMMIT_ID, COMMIT_ID_LONG -- each
assembling them by hand, and each shelling out to hostname to do it.
builddebs.pl also ran git rev-parse a second time to get the commit, although
it had already derived the same value into $GITINFO earlier in the file. It now
uses that.
The filenames and the time formats stay as they were. deploy.sh copies
builddebs.pl's buildinfo verbatim and cluster-test.pl parses buildrpms.pl's
buildinfo.txt, so both are a contract with consumers outside this repository;
the shared helper takes the format from its caller rather than picking one.
Confirmed byte for byte that each builder's stamp is unchanged.
Covered by tests: the field names and their order, the seven-character short
commit, and that the two callers' formats still differ.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs.pl spelled out the same slurp five times, and twice wrapped it in the
identical read, transform, write-back sequence -- once to pin the dependency
versions in debian/control and once to rewrite the changelog header. The two
blocks differed only in the file and the function applied to its contents.
BuildUtils.pm now provides read_file, write_file and rewrite_file. The last
leaves a file that is not there alone rather than creating it, which is what
both call sites guarded for with -f.
These use plain open and close rather than File::Slurper. buildrpms.pl already
depends on that module, but builddebs.pl does not, and putting it in the shared
module would oblige a deb build to install something it otherwise has no need
of.
Reading a missing file now names the file it could not read; one of the call
sites died with no message at all.
Covered by tests: a round trip, that rewrite_file applies its transform and
reports whether it acted, that it neither runs the transform nor creates the
file when there is nothing there, and that a failed read names the path.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders defined their own sh() and usage(). The usage() copies differed
only in spelling, but the sh() copies disagreed about what they return:
buildrpms.pl shifted system()'s result into an exit code, builddebs.pl returned
the raw wait status. The same command therefore reported 1 from one builder and
256 from the other, and a caller comparing sh() against a particular code was
correct in only one of them. Both current call sites only test for zero, so
nothing was broken yet.
BuildUtils.pm now provides both, shifting the status as buildrpms.pl did.
pod2usage reads the POD of the running program, so each builder keeps its own
help text while sharing how it is printed and the status it exits with.
sh() echoed the command under the builder's own --verbose flag, which a shared
function cannot see. $BuildUtils::VERBOSE is set once after option parsing
instead of threading the flag through every call site.
Covered by tests: that sh() returns the exit code rather than the wait status
it is packed into, and that a verbose run still echoes the command.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl derived the git revision and SOURCE_DATE_EPOCH with its own inline
backticks while BuildUtils.pm already provided git_revision() and
source_date_epoch() for builddebs.pl. Both did the same thing, and the copies
in buildrpms.pl were the weaker ones: neither fell back to the tracked Gitinfo
and Gitepoch files, so a build from a source tarball with no git history
recorded the revision as "unknown" and stamped the current time as the build
epoch, losing reproducibility exactly where it matters most.
Verified that the helper returns the same revision and epoch as the code it
replaces on a checkout with history.
buildrpms.pl now loads BuildUtils.pm from its own directory, so the source-only
test stages the module into its sandbox alongside Version. That test copies the
builder rather than running it in the checkout, because it rewrites Gitinfo and
creates $HOME/rpmbuild before it looks at its arguments.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
An xCAT installed from these debs could report no version at all: `lsxcatd -v`
printed a bare "Version" and there was no /opt/xcat/share/xcat/version to
consult.
xCAT::Version ships with #XCATVERSIONSUBHERE / #XCATSVNBUILDSUBHERE
placeholders that perl-xCAT/debian/rules fills by calling modifyUtils with the
version and the commit. It takes the commit from ../Gitinfo, falling back to
`git log`. Neither Debian builder wrote Gitinfo -- buildrpms.pl does, which is
why only the deb side was affected -- so a tree without a readable .git left
that argument empty. modifyUtils then hit `if [ -z "$2" ]` and returned 0
WITHOUT substituting, and neither caller checks the status, so the placeholders
shipped.
Two independent things had to be wrong for this to be silent, and both are
fixed:
modifyUtils now exits non-zero on a missing argument, so a build that cannot
stamp a version fails instead of producing one that cannot identify itself. Its
message was also wrong -- it checked $2 and complained about the version, which
is $1 -- so each argument now names itself.
builddebs.pl writes Gitinfo, as buildrpms.pl already does. BuildUtils::
git_revision prefers the checkout, falls back to an existing Gitinfo (a source
export carries the real revision that way, and overwriting it with a
placeholder would discard the only provenance the tree has), and only then
returns "unknown" -- never the empty string modifyUtils ignores.
"unknown" is now warned about rather than stamped quietly. The cause is usually
not a missing .git but git refusing one it considers dubiously owned: the tree
belongs to another user and the safe.directory exception sits in a config that
the build's own HOME override hides. That is exactly how this went unnoticed.
Not a regression -- build-ubunturepo produces byte-identical unsubstituted
placeholders in the same .git-less tree, verified by running it there. It is a
pre-existing hole that CD masks by writing Gitinfo in the pipeline.
Verified on xcat-master-ub: with a readable checkout the deb carries
'Version ' . '2.19.0' . ' (git commit 0030133c89)';
without one it carries '(git commit unknown)' and the build prints the warning
naming the tree, the command to run and the HOME to run it under. Nine
assertions drive modifyUtils itself -- stamping both forms, and each missing
argument -- and dropping either sed, or restoring the silent exit, reddens it.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A second run in the same checkout died once the release string moved:
dpkg-genbuildinfo: error: cannot fstat file ../xcat_..._ppc64el.deb
debian/files accumulates one line per artifact and survives `dh_clean -d`,
which removes directories only. The next build's dpkg-genchanges reads the
stale entries and fstats artifacts collect_debs already moved away. The old
shell builder deleted debian/files explicitly; that step was not carried
over. --force does not help, since it only wipes the output repository.
Three further ways the build did not put the tree back as it found it:
xCAT/postscripts/{bmcsetup,getipmi} are TRACKED files that the xCAT build
rewrites from the genesis sources. They were recorded as created, so
cleanup deleted them from the checkout. They are claimed now.
Restoring a claimed file lost its mode: File::Copy::copy does not carry
permissions, so an executable came back 100644 with identical content --
visible only as a git mode change. backup_file/restore_file record and
reapply it.
debian/*.substvars are rewritten in place and several are tracked; they are
claimed too. debhelper's .debhelper/ and *.debhelper.log are never tracked
and are removed with the rest of the residue.
clean_debian_residue runs from collect_debs, which is OUTSIDE
with_prepared_tree -- the restore has already happened by then. That is why
it must not remove *.substvars: doing so would delete the tracked ones it
just put back. The claim mechanism handles those instead.
Verified on xcat-master-ub: two consecutive full 14-package builds in one
checkout both succeed, and a build against a git checkout now leaves zero
tracked files modified or deleted (was five, two of them deleted). The
remaining untracked residue -- pods/, share/, pod2htmd.tmp and the
substvars of packages that do not track one -- is inherited from
build-ubunturepo and unchanged here.
Also covers the guard that gives this PR its name: deleting
`return if $opts{source_only}` from buildall left buildrpms_source_only.t
green, so nothing checked that --source-only skips the binary rebuild. It
does now, and the mutation reddens two assertions. Same for the changelog
trailer: making its substitution global left the suite green because only
the older header was asserted, not its author and date.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
ubuntu_2604_pkglist.t matched regexes against Template.pm's `push @lines`
literals, and ubuntu_subiquity_storage.t was source greps end to end. Both
pass when the code is moved somewhere it never runs, and both break on a
reformat that changes nothing: reordering two keys inside the efi-part
stanza -- identical emitted config -- fails the old efi-part match.
The apt config is now rendered by calling ubuntu_subiquity_apt_config and
asserting on its output. Only the two collaborators that read the xCAT
database are stubbed; the release branch is driven for real through the
media directory the renderer parses, so 26.04, 24.04 and pre-Deb822 each
render their own shape. The Release-index requirement is exercised against
a real directory instead of matched as a shell fragment.
The partitioning moves to ubuntu_subiquity_storage.t, which owns that
topic and was asserting the same facts by grep. The script cannot run
here -- it stops syslog and carries xCAT template markers -- so the block
that writes the partition file is lifted out and executed with its one
bracket test shadowed by a shell function, which bash resolves ahead of
the builtin, leaving the script's own condition unmodified. The emitted
curtin config is parsed into id => attributes, so the assertions survive
reindentation and reordering.
Both extractions BAIL_OUT if their anchors stop matching, and the
partition-file redirect is asserted to have been pointed at the scratch
tree twice before anything runs, so a rewrite that stops matching fails
loudly instead of writing to /tmp.
Verified by mutation: dropping grub_device, formatting the ESP ext4,
moving its mount point, dropping the bios_grub flag, indenting the block
off column 0, dropping Enabled: no, rendering a live cdrom source on
26.04, dropping the Check-Date waiver, dropping the offline fallback and
dropping the Release requirement each redden the suite; the pure key
reorder that broke the old assertion leaves it green.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl rewrites Gitinfo in its working directory at every run, and a
verification run left it staged into this branch. It is not tracked upstream
and is not part of the source, so it is removed and ignored.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms_source_only.t's CLI half failed in CI: the runner has no
Parallel::ForkManager, so buildrpms.pl aborted at compile time and never
reached the option check the test is about. The module is needed only by
the test suite -- buildrpms.pl is not a runtime dependency of any package
-- so it goes in the workflow apt list.
The same half also escaped its scratch tree. Before buildrpms.pl looks at
@ARGV it rewrites the tracked Gitinfo in its working directory and creates
$HOME/rpmbuild, so running it from the checkout left the tree dirty and
reached into the developer's home to exercise argument parsing. It now
runs from a staged copy with HOME pointed at the sandbox.
Exit 2 is pinned rather than "non-zero", though perl also exits 2 on a
compile abort -- which is exactly how this assertion stayed green in CI
while the program could not load. The message assertion is what separates
the two, and the comment now says so.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Four fixes from @viniciusferrao's review plus the CI break his review predates.
orig tarball version. dpkg looks for <source>_<upstream>.orig.tar.gz with no
Debian revision, and the call site passed the full Version-Release. The rule now
lives in BuildUtils::upstream_version and orig_tarball_name applies it, so the
call site cannot get it wrong whichever string it is handed. Currently dormant --
every package is Format: 1.0, so the quilt branch does not run, which is why the
differential build did not catch it.
--dest could write to the filesystem root. Cwd::abs_path returns undef when a
PARENT component is missing (a missing leaf is fine), and the caller interpolated
that, so `--dest /no/such/parent/out` became `/debs` and `/xcat-core` at /.
Replaced with BuildUtils::resolve_dest, which is rel2abs and purely lexical --
correct for an output directory that does not exist yet.
Generated debian/control left behind. xCAT-genesis-scripts has no debian/control
of its own; it is generated from control-<arch>. The cleanup restored only files
that already existed, so the generated one stayed. Worse than dirty: ppc64el ran
last, so the restore put back the amd64 BACKUP and the leftover was the wrong
architecture's control, which a later single-arch build would have started from.
with_prepared_tree now records created files and removes them. Verified by a real
build: the checkout is byte-clean afterwards, matching the oracle.
CI install step. build-ubunturepo wrote its repo to $curdir/../../xcat-core,
which under GitHub's work/<repo>/<repo> layout IS $RUNNER_WORKSPACE, so
install_xcat's `./mklocalrepo.sh` happened to be in the directory it chdir'd to.
builddebs.pl writes inside the checkout instead -- that outside-the-checkout path
is what used to rm -rf the tree -- so install_xcat now names the script by its
real location and fails with a clear message if the build produced no repository.
This is what reddened xcat_pr_test at 2m13s; the builder itself was fine (the
exact CI invocation, `./builddebs.pl --force` with no --dest, returns 0 with all
14 packages).
The executable bit was already fixed before the review landed.
Both new helpers are tested and mutation-verified: not stripping the revision
reddens 3 assertions, swapping rel2abs back to abs_path reddens 2. Equivalence
re-measured after these changes -- all 14 packages identical to build-ubunturepo
in control and in every non-changelog file by md5.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs.pl was written against build-ubunturepo but never run beside it. Keeping
the old script in the tree for one more cycle makes the comparison repeatable, and
running the two on xcat-master-ub found four defects that no unit test would have.
* builddebs.pl was not executable. github_action_xcat_test.pl runs
`sudo ./builddebs.pl`, so CI would have failed with Permission denied.
* .deb files were collected after every ARCHITECTURE. dpkg-genbuildinfo reads
the sibling artifacts of the source package it is building, so moving the
amd64 .deb away before ppc64el ran killed the second build with
dpkg-genbuildinfo: error: cannot fstat file ../xcat_..._amd64.deb
Collection now happens once a package's last architecture is done.
* A checkout dirtied by an aborted run poisoned the next one: a leftover
.changes made dpkg-genbuildinfo name an architecture the run had not reached.
The dpkg output is now cleared at start.
* The maintainer was xcat@xcat.org where build-ubunturepo uses
xcat-build@xcat.org, and a Release file in the tree was ignored.
buildrpms.pl WRITES Release, so a pipeline building both would have stamped
its debs with a different release from its rpms. Both corrected.
Equivalence, measured rather than assumed. Both builders were run on
xcat-master-ub from the same tree:
package set identical, 14/14, same names
control metadata identical, 14/14
payload contents identical, 14/14
changelog.Debian differs, 2 packages -- deliberately, see below
Byte-identity is NOT the criterion, because neither implementation has it: two
consecutive runs of build-ubunturepo on the same tree agree on 0 of 14 packages.
A .deb records the build wall-clock time in its ar member mtimes and
SOURCE_DATE_EPOCH does not reach dpkg-deb here, so every run differs from every
other. That is a pre-existing property of the Debian build, not a regression, and
worth fixing separately.
The changelog difference is build-ubunturepo being wrong. Its
sed -i "s/^ -- .*/ -- $DEBFULLNAME <$DEBEMAIL> $deterministic_date/"
carries no line address, so it rewrites EVERY trailer in debian/changelog:
"OCF xCAT <xcat@ocf.co.uk> Mon, 25 Oct 2010" ships as "xCAT Build
<xcat-build@xcat.org> Tue, 01 Sep 2026". It falsifies the authorship and dates of
the 2008 and 2010 releases. builddebs.pl rewrites only the top stanza. Matching
byte-for-byte would mean reproducing the defect, so this difference stays.
(xcat-vlan appeared to differ under `diff -r`; that was diff reporting dangling
symlinks in both trees. Its member listing is identical.)
build-ubunturepo is documented as retained-for-comparison and not to be extended.
It goes once the Ubuntu CD pipelines call builddebs.pl.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
build-ubunturepo was 710 lines of shell doing the Debian half of what
buildrpms.pl does for rpms, with no code in common and a different CLI. It also
carried paths that are dead: GSA uploads, the PROMOTE/PREGA release flows, and a
-d mode that built an xcat-dep repository from a different project's packages.
builddebs.pl replaces it and mirrors buildrpms.pl -- Getopt::Long options, one
package list, build then index then sign -- so the two builders read the same way
and share BuildUtils.pm.
The design rests on one fact: xcat-core debs are Perl. They are byte-identical
for every Ubuntu release, so they are built ONCE and the same files are published
into every codename. Only xCAT, xCATsn and xCAT-genesis-scripts carry an
architecture, and there the difference is packaging metadata, not compiled
output. That is why this needs no sbuild and no per-codename chroot -- unlike
xcat-dep, whose packages are compiled and genuinely differ per release.
BuildUtils.pm holds what both builders need and what was worth making testable:
the Version-Release derivation from the commit time, the xCAT-probe helper
staging, the deb arch and dist tables, the debian/control version pinning, the
changelog rewrite, the reprepro conf generation, and the build lock. Every
function is pure or takes its side effect as an argument, so build_utils.t (45
assertions) drives each one rather than grepping a builder for evidence that it
is called. Verified by mutation: shrinking the arch table reddens 1, dropping
the /g from the control pin reddens 2.
The env-var CLI maps to options: BUILDALL=1 -> --force, GPGSIGN=1 -> --gpg-sign,
GPG_HOME -> --gpg-home, DEST -> --dest, DISTS -> --dist (repeatable). UP=0 has no
equivalent because uploading is gone -- the CD pipeline's deploy step publishes.
Callers updated: github_action_xcat_test.pl and travis.pl. The comment in
github_action_xcat_test.pl explaining why CI copies the tree before building is
corrected -- build-ubunturepo rm -rf'd $curdir/../../xcat-core, which under
GitHub's work/<repo>/<repo> layout is the checkout's own parent; builddebs.pl
writes under dist/debs inside the checkout and restores every file it edits, so
the copy is now only isolating the tests from build residue.
Two tests moved with it. build_ubunturepo_lock.t extracted the lock out of the
shell with a regex and ran that; the lock is now a function, so builddebs_lock.t
calls it -- and asserts what actually matters, that two builds of one checkout
fail fast while two builds of different checkouts run concurrently.
ubuntu_2604_pkglist.t asserted that resolute appeared in a shell fragment of
build-ubunturepo's source; it now asks BuildUtils for the release list and checks
a resolute stanza reaches conf/distributions. That assertion would have passed on
any file containing the fragment and broken on a reflow that changed nothing.
Verified: prove -r xCAT-test/unit fails on 6 files here against 7 on
upstream/master, the difference being apache_config_sources.t, fixed by the
preceding commit. The remaining 6 are missing DB modules on the machine that ran
it and are identical on both.
NOT done here, and required before this can merge: the Ubuntu core CD pipelines
still invoke ./build-ubunturepo (ci/ubuntu/Jenkinsfile.core-ubuntu-{devel,stable}
in VersatusHPC/xcat-core-ci-cd, and the inline script in each live Jenkins job).
Those must be switched to builddebs.pl in the same change window.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl is what actually builds xcat-core now: every core CD pipeline calls
it (ci/{el,suse,ubuntu}/Jenkinsfile.core-*), and it reaches none of these
scripts. What is left of them is a second, diverging build path that nothing
runs -- buildcore.sh has no caller at all (its one mention in
Jenkinsfile.core-el-stable is a comment), makerpm's only caller is
buildlocal.sh, and buildlocal.sh has no caller.
That divergence broke master. #7774 added a source-only mode to these scripts
rather than to buildrpms.pl, giving makerpm an unguarded
. "$(dirname "$0")/build-utils/source-only.sh"
at line 13, used on every build path (xcat_configure_rpm_build_mode,
xcat_announce_build), not only under SRCONLY. apache_config_sources.t stages
makerpm into a fixture that creates build-utils/ but copies only
sync-xcat-apache-configs into it, so makerpm died at line 13 and nothing reached
SOURCES/. Because prove -r xCAT-test/unit runs the whole suite, that reddened
xcat_pr_test on every open PR. It merged on a green run from 2026-08-27;
apache_config_sources.t landed 2026-08-30, and nothing rebuilt it in between.
Removed: makerpm, buildcore.sh, buildlocal.sh, build-utils/source-only.sh,
build-utils/buildcore-source-only.sh, and the tests that covered only them
(makerpm_source_only.t, buildcore_source_only.t). The makerpm half of
apache_config_sources.t goes with makerpm; its sync-xcat-apache-configs half
stays, because buildrpms.pl invokes that helper directly. The source-only
capability is preserved as buildrpms.pl --source-only, in the preceding commit.
build-ubunturepo is deliberately NOT touched. It is not a legacy script: both
Ubuntu core CD pipelines invoke it, github_action_xcat_test.pl:325 invokes it on
every PR, and it was last fixed this week. buildrpms.pl is RPM-only -- mock and
createrepo_c, no dpkg path -- so there is nothing to migrate it to. Giving
Debian builds an equivalent modern builder is separate work.
Verified: prove -r xCAT-test/unit before and after differs by exactly one file
-- apache_config_sources.t fails on upstream/master (4a0d9e0bb, tests 64-65) and
passes here. The other six failures are identical on both and are missing DB
modules on the machine that ran it. No remaining reference to any removed file
except the one comment in buildrpms.pl naming where its templates came from.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst
Vinícius Ferrão