mirror of
https://github.com/xcat2/xcat-core.git
synced 2026-09-05 04:27:55 +00:00
42e4c160d4a838682e4687028ca4d1928f4fb848
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Daniel Hilst
|
42e4c160d4 |
refactor(build): share sh() and usage() between the two builders
Both builders defined their own sh() and usage(). The usage() copies differed only in spelling, but the sh() copies disagreed about what they return: buildrpms.pl shifted system()'s result into an exit code, builddebs.pl returned the raw wait status. The same command therefore reported 1 from one builder and 256 from the other, and a caller comparing sh() against a particular code was correct in only one of them. Both current call sites only test for zero, so nothing was broken yet. BuildUtils.pm now provides both, shifting the status as buildrpms.pl did. pod2usage reads the POD of the running program, so each builder keeps its own help text while sharing how it is printed and the status it exits with. sh() echoed the command under the builder's own --verbose flag, which a shared function cannot see. $BuildUtils::VERBOSE is set once after option parsing instead of threading the flag through every call site. Covered by tests: that sh() returns the exit code rather than the wait status it is packed into, and that a verbose run still echoes the command. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com> |
||
|
Daniel Hilst
|
87c8a7c0dd |
fix(build): stamp the version into the deb packages instead of shipping placeholders
An xCAT installed from these debs could report no version at all: `lsxcatd -v`
printed a bare "Version" and there was no /opt/xcat/share/xcat/version to
consult.
xCAT::Version ships with #XCATVERSIONSUBHERE / #XCATSVNBUILDSUBHERE
placeholders that perl-xCAT/debian/rules fills by calling modifyUtils with the
version and the commit. It takes the commit from ../Gitinfo, falling back to
`git log`. Neither Debian builder wrote Gitinfo -- buildrpms.pl does, which is
why only the deb side was affected -- so a tree without a readable .git left
that argument empty. modifyUtils then hit `if [ -z "$2" ]` and returned 0
WITHOUT substituting, and neither caller checks the status, so the placeholders
shipped.
Two independent things had to be wrong for this to be silent, and both are
fixed:
modifyUtils now exits non-zero on a missing argument, so a build that cannot
stamp a version fails instead of producing one that cannot identify itself. Its
message was also wrong -- it checked $2 and complained about the version, which
is $1 -- so each argument now names itself.
builddebs.pl writes Gitinfo, as buildrpms.pl already does. BuildUtils::
git_revision prefers the checkout, falls back to an existing Gitinfo (a source
export carries the real revision that way, and overwriting it with a
placeholder would discard the only provenance the tree has), and only then
returns "unknown" -- never the empty string modifyUtils ignores.
"unknown" is now warned about rather than stamped quietly. The cause is usually
not a missing .git but git refusing one it considers dubiously owned: the tree
belongs to another user and the safe.directory exception sits in a config that
the build's own HOME override hides. That is exactly how this went unnoticed.
Not a regression -- build-ubunturepo produces byte-identical unsubstituted
placeholders in the same .git-less tree, verified by running it there. It is a
pre-existing hole that CD masks by writing Gitinfo in the pipeline.
Verified on xcat-master-ub: with a readable checkout the deb carries
'Version ' . '2.19.0' . ' (git commit
|
||
|
Daniel Hilst
|
63c9ef8a1f |
fix(build): stop builddebs.pl leaving the checkout unbuildable and dirty
A second run in the same checkout died once the release string moved:
dpkg-genbuildinfo: error: cannot fstat file ../xcat_..._ppc64el.deb
debian/files accumulates one line per artifact and survives `dh_clean -d`,
which removes directories only. The next build's dpkg-genchanges reads the
stale entries and fstats artifacts collect_debs already moved away. The old
shell builder deleted debian/files explicitly; that step was not carried
over. --force does not help, since it only wipes the output repository.
Three further ways the build did not put the tree back as it found it:
xCAT/postscripts/{bmcsetup,getipmi} are TRACKED files that the xCAT build
rewrites from the genesis sources. They were recorded as created, so
cleanup deleted them from the checkout. They are claimed now.
Restoring a claimed file lost its mode: File::Copy::copy does not carry
permissions, so an executable came back 100644 with identical content --
visible only as a git mode change. backup_file/restore_file record and
reapply it.
debian/*.substvars are rewritten in place and several are tracked; they are
claimed too. debhelper's .debhelper/ and *.debhelper.log are never tracked
and are removed with the rest of the residue.
clean_debian_residue runs from collect_debs, which is OUTSIDE
with_prepared_tree -- the restore has already happened by then. That is why
it must not remove *.substvars: doing so would delete the tracked ones it
just put back. The claim mechanism handles those instead.
Verified on xcat-master-ub: two consecutive full 14-package builds in one
checkout both succeed, and a build against a git checkout now leaves zero
tracked files modified or deleted (was five, two of them deleted). The
remaining untracked residue -- pods/, share/, pod2htmd.tmp and the
substvars of packages that do not track one -- is inherited from
build-ubunturepo and unchanged here.
Also covers the guard that gives this PR its name: deleting
`return if $opts{source_only}` from buildall left buildrpms_source_only.t
green, so nothing checked that --source-only skips the binary rebuild. It
does now, and the mutation reddens two assertions. Same for the changelog
trailer: making its substitution global left the suite green because only
the older header was asserted, not its author and date.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
|
||
|
Daniel Hilst
|
2bfad97348 |
fix(build): address review on builddebs.pl, and repoint CI at the new repo
Four fixes from @viniciusferrao's review plus the CI break his review predates. orig tarball version. dpkg looks for <source>_<upstream>.orig.tar.gz with no Debian revision, and the call site passed the full Version-Release. The rule now lives in BuildUtils::upstream_version and orig_tarball_name applies it, so the call site cannot get it wrong whichever string it is handed. Currently dormant -- every package is Format: 1.0, so the quilt branch does not run, which is why the differential build did not catch it. --dest could write to the filesystem root. Cwd::abs_path returns undef when a PARENT component is missing (a missing leaf is fine), and the caller interpolated that, so `--dest /no/such/parent/out` became `/debs` and `/xcat-core` at /. Replaced with BuildUtils::resolve_dest, which is rel2abs and purely lexical -- correct for an output directory that does not exist yet. Generated debian/control left behind. xCAT-genesis-scripts has no debian/control of its own; it is generated from control-<arch>. The cleanup restored only files that already existed, so the generated one stayed. Worse than dirty: ppc64el ran last, so the restore put back the amd64 BACKUP and the leftover was the wrong architecture's control, which a later single-arch build would have started from. with_prepared_tree now records created files and removes them. Verified by a real build: the checkout is byte-clean afterwards, matching the oracle. CI install step. build-ubunturepo wrote its repo to $curdir/../../xcat-core, which under GitHub's work/<repo>/<repo> layout IS $RUNNER_WORKSPACE, so install_xcat's `./mklocalrepo.sh` happened to be in the directory it chdir'd to. builddebs.pl writes inside the checkout instead -- that outside-the-checkout path is what used to rm -rf the tree -- so install_xcat now names the script by its real location and fails with a clear message if the build produced no repository. This is what reddened xcat_pr_test at 2m13s; the builder itself was fine (the exact CI invocation, `./builddebs.pl --force` with no --dest, returns 0 with all 14 packages). The executable bit was already fixed before the review landed. Both new helpers are tested and mutation-verified: not stripping the revision reddens 3 assertions, swapping rel2abs back to abs_path reddens 2. Equivalence re-measured after these changes -- all 14 packages identical to build-ubunturepo in control and in every non-changelog file by md5. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com> |
||
|
Daniel Hilst
|
b8510e1be3 |
refactor(build): replace build-ubunturepo with builddebs.pl and BuildUtils.pm
build-ubunturepo was 710 lines of shell doing the Debian half of what
buildrpms.pl does for rpms, with no code in common and a different CLI. It also
carried paths that are dead: GSA uploads, the PROMOTE/PREGA release flows, and a
-d mode that built an xcat-dep repository from a different project's packages.
builddebs.pl replaces it and mirrors buildrpms.pl -- Getopt::Long options, one
package list, build then index then sign -- so the two builders read the same way
and share BuildUtils.pm.
The design rests on one fact: xcat-core debs are Perl. They are byte-identical
for every Ubuntu release, so they are built ONCE and the same files are published
into every codename. Only xCAT, xCATsn and xCAT-genesis-scripts carry an
architecture, and there the difference is packaging metadata, not compiled
output. That is why this needs no sbuild and no per-codename chroot -- unlike
xcat-dep, whose packages are compiled and genuinely differ per release.
BuildUtils.pm holds what both builders need and what was worth making testable:
the Version-Release derivation from the commit time, the xCAT-probe helper
staging, the deb arch and dist tables, the debian/control version pinning, the
changelog rewrite, the reprepro conf generation, and the build lock. Every
function is pure or takes its side effect as an argument, so build_utils.t (45
assertions) drives each one rather than grepping a builder for evidence that it
is called. Verified by mutation: shrinking the arch table reddens 1, dropping
the /g from the control pin reddens 2.
The env-var CLI maps to options: BUILDALL=1 -> --force, GPGSIGN=1 -> --gpg-sign,
GPG_HOME -> --gpg-home, DEST -> --dest, DISTS -> --dist (repeatable). UP=0 has no
equivalent because uploading is gone -- the CD pipeline's deploy step publishes.
Callers updated: github_action_xcat_test.pl and travis.pl. The comment in
github_action_xcat_test.pl explaining why CI copies the tree before building is
corrected -- build-ubunturepo rm -rf'd $curdir/../../xcat-core, which under
GitHub's work/<repo>/<repo> layout is the checkout's own parent; builddebs.pl
writes under dist/debs inside the checkout and restores every file it edits, so
the copy is now only isolating the tests from build residue.
Two tests moved with it. build_ubunturepo_lock.t extracted the lock out of the
shell with a regex and ran that; the lock is now a function, so builddebs_lock.t
calls it -- and asserts what actually matters, that two builds of one checkout
fail fast while two builds of different checkouts run concurrently.
ubuntu_2604_pkglist.t asserted that resolute appeared in a shell fragment of
build-ubunturepo's source; it now asks BuildUtils for the release list and checks
a resolute stanza reaches conf/distributions. That assertion would have passed on
any file containing the fragment and broken on a reflow that changed nothing.
Verified: prove -r xCAT-test/unit fails on 6 files here against 7 on
upstream/master, the difference being apache_config_sources.t, fixed by the
preceding commit. The remaining 6 are missing DB modules on the machine that ran
it and are identical on both.
NOT done here, and required before this can merge: the Ubuntu core CD pipelines
still invoke ./build-ubunturepo (ci/ubuntu/Jenkinsfile.core-ubuntu-{devel,stable}
in VersatusHPC/xcat-core-ci-cd, and the inline script in each live Jenkins job).
Those must be switched to builddebs.pl in the same change window.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
|