Closes the reproducibility/supply-chain gap left as a TODO: the Ubuntu goconserver
build resolved all transitive modules live from the network via 'go mod tidy', so the
compiled binary was not reproducible and the graph was unverified.
Commit goconserver/gomod/go.{mod,sum}, generated with the build's own pinned toolchain
(go 1.25.12) at REF=6166fe5, with the etcd backend removed (its coreos/bbolt dep, now
go.etcd.io/bbolt, breaks 'go mod tidy') and kr/pty replaced by creack/pty -- exactly the
steps the build performs. sbuild.pl now overlays the committed pair into the cloned tree
and builds with GOFLAGS=-mod=mod, so modules are downloaded but PINNED + integrity-checked
by go.sum; no 'go mod tidy'. Verified on xcat-master-ub: both entrypoints (goconserver.go,
cmd/congo.go) compile CGO-free against the committed go.sum. gomod/README.md documents
regeneration. Mirrors the EL branch's pinning model (its go.sum is generated with a
different Go, so a fresh Ubuntu-toolchain pair is used rather than copying it).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up for the Ubuntu sbuild matrix:
- Add a fail-fast exclusive flock over the whole run (<output-root>/.sbuild-all.lock,
file-scoped handle) so two overlapping runs can't corrupt the shared staging/apt
tree -- this is the root of the observed 'remove_tree .../staging/<cn>/<arch>:
Directory not empty' (an NFS silly-rename from a concurrent run).
- wipe_tree(): remove_tree that captures {error} and dies loud, so an ENOTEMPTY no
longer carps-and-continues leaving stale debs; used for all staging/pool/dists wipes.
- Wire the tested, hash-based cross_copy_genesis_deb into build_genesis (was a naive
glob+copy, so the unit-tested stale-dropping copier was dead code); remove the
genuinely-unused deb_snap_version/rewrite_changelog_top helpers + their subtests
(compiled deps intentionally ship their tracked changelog version).
- Dedupe assemble_apt on binary Package+Architecture (keep highest via
dpkg --compare-versions) so a double-produced genesis can't land two versions in
the pool, independent of the --skip-genesis contract.
- Derive Release Architectures from the arches actually staged (non-empty
binary-<arch>/Packages), not a hard-coded 'amd64 ppc64el'.
- goconserver: guard 'go mod init' when a go.mod exists (+ TODO to commit go.sum for
the pinned SHA). Accept-and-ignore the unused per-package --log-dir/--build-number/
--skip-install flags (documented). Remove orphaned make_deb.sh dispatchers
(build-debs-all, build.sh, ipmitool/build.sh) + update the READMEs.
perl -c clean; prove t/sbuild-all.t: 71/71.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The per-package <dep>/sbuild.pl were generic wrappers that shelled out to each
package's make_deb.sh, so a package's build brain was split across two files and
package-specific fixes (e.g. goconserver's Go toolchain) landed in make_deb.sh
instead of the builder -- inconsistent with the EL side, where <dep>/mockbuild.pl
IS the per-package builder.
Make each <dep>/sbuild.pl own its build (mirroring <dep>/mockbuild.pl), absorbing
its make_deb.sh (source prep, patches, toolchain, dpkg-buildpackage), and remove
all seven make_deb.sh. The common chroot orchestration -- ephemeral schroot session,
apt update, build-dep install, out-of-tree copy, SOURCE_DATE_EPOCH, deb collection +
host-side verification -- moves into BuildUtils::build_deb_in_chroot; each builder
supplies only its package-specific recipe (passed base64-encoded to avoid quoting
interplay through schroot).
Also make the older Ubuntu codenames buildable:
- goconserver: install a pinned modern Go (1.25.12) in the build -- focal/jammy ship
a Go too old to even auto-switch toolchains, and goconserver's pinned deps need
Go >= 1.25 (it is a static CGO-free binary, so the pinned toolchain is portable and
reproducible across codenames).
- ipmitool + goconserver: lower debian/compat 13 -> 12 and Build-Depends debhelper
(>= 12), since Ubuntu 20.04 (focal) ships debhelper 12; compat 12 also builds cleanly
on newer codenames.
Validated: all recipe shapes build green through the new path -- tarball
(ipmitool@focal), git-clone+pinned-Go (goconserver@focal), tarball+patches
(syslinux@noble), in-place (grub2-xcat), in-place+custom-rules (xnba); 80/80 unit
tests still pass.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Each <dep>/sbuild.pl copies the built .deb(s) to --result-dir from INSIDE the
schroot session. That only reaches the host when --result-dir is on a path
bind-mounted into the chroot (the shared /opt/xcat-ci-shared tree, as the CI
uses). A mis-configured result-dir (e.g. a chroot-local /tmp) would let the
build "succeed" yet leave nothing on the host -- a silent no-output. Verify
host-side after the session that the .deb(s) are actually present and fail
loud with a pointed message otherwise (consistent with the fail-hard design).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu/Debian dependency build shipped as three bash scripts
(build-dep-debs.sh, build-apt-repo.sh, mk-dep-chroots.sh) whose review
(PR #63) surfaced correctness problems: partial/stale output could be
published, the rpm->deb genesis conversion dropped the maintained package
semantics (Depends/Breaks/Replaces + maintainer scripts), the arch matrix
was invalid (x86-only syslinux/elilo/xnba treated as ppc64el packages, and
Architecture:all packages with no single producer), several required
failures exited zero, and the build/repo scripts disagreed on their staging
path and codename set (focal missing from the assembler).
Rewrite it as proper, unit-tested Perl mirroring the EL side
(mockbuild-all.pl / MockBuildUtils.pm / <dep>/mockbuild.pl / t/*.t /
packages-manifest.conf), sharing one CLI vocabulary:
- BuildUtils.pm: shared, testable helpers + the canonical CLI spec, plus the
Debian-specific helpers (out-of-tree changelog stamping, genesis control
preservation, deb inspection, cross-arch genesis provisioning).
- sbuild-all.pl: the orchestrator, absorbing all three shell scripts. Builds
+ validates into a fresh per-arch staging tree and only (re)assembles the
published apt repo from validated staging -- so partial/failed output never
ships and stale debs never accumulate. Auto-initializes the per-codename
sbuild chroots on first run. Fails the whole run non-zero on any missing
chroot/package/artifact or version-pin mismatch.
- <dep>/sbuild.pl x7: per-package builders that drive each package's
MAINTAINED debian/ in the matching chroot (never re-implemented), so the
converted/built packages keep their control metadata and maintainer scripts.
- debs-manifest.conf: per-[<codename>-<arch>] required set + version pins,
encoding the per-arch package sets (x86 boot components built once on amd64
as the single producer; ppc64el builds only the arch-specific compiled deps).
- t/sbuild-all.t: fixture tests for every pure helper.
- goconserver/make_deb.sh: pin the upstream SHA instead of cloning a moving
branch, so every matrix cell builds the same source (reproducible).
Codename set unified across build, assembly, chroots and docs (focal IS
supported). BUILD.md documents the new flow.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
mockbuild-all.pl treated xnba-undi/grub2-xcat as x86-only @SHARED_NOARCH imported
onto ppc, so an x86 build failure rippled into a fatal "grub2-xcat missing" on
ppc; both actually build on any arch (noarch repackaging of committed artifacts)
and xCAT requires them on ppc too. Un-gate xnba-undi and drop the import so each
host builds a complete, self-sufficient dep repo. Add a single --output that
re-roots all NFS-shared output plus a fail-fast lock at <output>/.lock (owner-pid
guarded so forked build children do not delete it). Default gpg-key-name to the
real "xCAT Signing Key" and --xcat-source to ../xcat-core.
Make concurrent builds safe: give the genesis buildrpms.pl its own HOME/rpmbuild
tree (a shared /root/rpmbuild raced across parallel targets), and nest the
xnba/goconserver rpmbuild dirs under the run-scoped --work-dir (they hard-coded
/var/tmp/xcat-rpmbuild-* and wiped each other). Add --parallel-targets/--max-parallel,
defaulting to serial since the per-package scripts still share repo tarballs under
full parallelism.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Commit f6f8640 added the /var/lib/goconserver datadir so goconserver can create
its nodes.json store and makegocons can register consoles, but kept Release: 2.
The fixed build then shared an identical NVR (goconserver-0.3.3-2) with the
pre-fix build, so a stale pre-fix -2 could be (and was) served to management
nodes where makegocons still failed ("open /var/lib/goconserver/nodes.json: no
such file or directory"). Bump Release to 3 so the datadir fix is a distinct,
upgradeable version that cannot collide with the pre-fix -2.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
goconserver imports the abandoned kr/pty, whose pty.Start sets the controlling
tty in a way Go >= 1.15 rejects ("fork/exec ...: Setctty set but Ctty not valid
in child"). On current toolchains this breaks the console backend: rcons
connects but the session produces no output. Replace kr/pty with the
maintained, API-compatible creack/pty fork at build time.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The goconserver deb shipped no systemd unit, so enabling or starting the
goconserver service after installation had nothing to start (makegocons failed
with "Failed to start goconserver service"). Install the provided
goconserver.service unit during packaging.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Port SOURCE_DATE_EPOCH patterns from xcat-core to all xcat-dep build
scripts. RPM and Debian packages now produce identical output given
the same Gitepoch timestamp, regardless of build host or time.
EL (RPM) changes:
- mockbuild-all.pl: --build-timestamp flag, deterministic run_id,
tar --sort/--owner/--group/--mtime, createrepo --revision
- All mockbuild.pl: SOURCE_DATE_EPOCH cascade (CLI > Gitepoch > git > time),
deterministic mock config with SOURCE_DATE_EPOCH in chroot env,
RPM macros for timestamp clamping and fixed buildhost
- goconserver: -trimpath -buildvcs=false, canonical rpmbuild path
- xnba: canonical rpmbuild path
- syslinux: ZERO_AR_DATE=1 in mock env
Ubuntu (Debian) changes:
- All make_deb.sh: SOURCE_DATE_EPOCH fallback from Gitepoch
- goconserver: deterministic SNAP_TS and changelog from epoch,
-trimpath -buildvcs=false in debian/rules
- syslinux: -fdebug-prefix-map for path-independent debug info
- build-apt-repo.sh: gzip -n, Release Date: from SOURCE_DATE_EPOCH
Verified: 31/33 package artifacts produce identical SHA256 hashes
across independent builds. 2 syslinux sub-packages (devel,
debugsource) have known limitations from upstream build system.
Create debian/ directory and make_deb.sh for building goconserver
as a .deb package. Clones upstream, removes etcd backend, builds
with CGO_ENABLED=0 for static linking, packages via dpkg-buildpackage.
Tested on Ubuntu 22.04, 24.04, and 26.04.
Auto-detection used OS ID directly (e.g. almalinux+epel-10-ppc64le)
but mock configs use short forms (alma+epel-10-ppc64le). Add
resolve_mock_cfg helper that tries the full ID first, then falls
back to known short forms.
The smoke test previously required the IPMI probe to fail, which
breaks on any build host with a BMC. Allow rc=0 when hardware is
present; only fail on unexpected error output.
Clone goconserver from upstream master (v0.3.3), initialize Go
modules, and compile both server and client binaries with
CGO_ENABLED=0 for static linking. The etcd storage backend is
removed before building because its dependencies are incompatible
with modern Go modules; xCAT only uses file-based console storage.
v0.3.3 removes RC4 and 3DES ciphers from the TLS configuration.
Daniel Hilst
Vinícius Ferrão
GONG Jie