The elilo, ipmitool and syslinux builders each fetched their upstream
source and rewrote the tracked source tarball in place, inside the
package source directory that both arch builds share. When the two
per-arch builds run in parallel they were racing to fetch the source:
one build truncated and rewrote the tarball while the other read it, so
the reader got a truncated archive and failed intermittently with
"missing top-level tree" errors.
The correct, normalized source is already tracked in the repository and
is what mock consumes, so the fetch is redundant as well as unsafe. Drop
the download/normalize entirely and verify the tracked source read-only
(it exists and has the expected top-level tree). With no writer, the
shared source is only ever read, so parallel per-arch builds can no
longer race on it. Also removes the now-dead --source-url /
--skip-upstream-download options, the normalize helper, and the wget
dependency check.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
elilo/mockbuild.pl wget'd the upstream tarball over $source_path and then
normalized it in place, rewriting the tracked source file. Parallel builds share
that same source file, so builds running concurrently raced to fetch and rewrite
it -- one build could read the file mid-rewrite and get a truncated archive, an
intermittent "Normalized source archive still missing elilo top-level tree"
failure. Because elilo is a required dep, one such flake failed the whole run.
The tracked tarball is already normalized (elilo/ top-level), so use it read-only
when normalized and only fetch upstream when it is absent or not yet normalized.
This removes the in-place rewrite (hence the race) and the flaky sourceforge
dependency, and is more reproducible.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Per review on xcat2/xcat-dep#62, narrow this PR to the EL matrix only
(rh8/rh9/rh10 x x86_64/ppc64le). The SUSE and Ubuntu work is reverted out of the
PR's net diff and will land in its own PR, so the reviewer's SUSE/Ubuntu points
(SuSE breakage, ubuntu20.04/focal in the default set, the xcat@megware.com key
default) are moot here -- those targets are no longer part of this change.
Reverted (net-zero vs master):
- SUSE target support in mockbuild-all.pl (opensuse-leap -> sles<N>).
- SUSE perl BuildRequires compat + elilo suse_version hunk.
- build-apt-repo.sh Ubuntu changes (focal + key default).
EL-relevant review fixes kept:
- Rename the finalize options to precise arch names: --x86-repo/--ppc-repo ->
--x86_64-repo/--ppc64le-repo (and matching vars/labels). "x86"/"ppc" was
ambiguous, especially since the genesis package is named -ppc64 via tarch yet
carries no big-endian code.
- Clarify the "tolerated build" comment (reviewer #2): builder failures are
tolerated only so one flaky builder cannot abort the others; correctness is
enforced by RESULT via assert_required_deps (a missing REQUIRED rpm still fails
the run), not by exit code. Toleration is load-bearing -- perl-Sys-Virt fails
on el8 by design, and genesis "fails" cosmetically while still producing its rpm.
- BUILD.md: drop all references to the removed --skip-xcat flag and the stale
"unified xCAT repository" framing (the core is built by the xcat-core
pipeline), and document --finalize-xcat-dep.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
- elilo: add %if 0%{?suse_version} to use_prebuilt (openSUSE lacks the gnu-efi
linker inputs elilo compiles against, same as EL8/ppc).
- mockbuild-perl-packages: on openSUSE mock, inject a tiny local repo whose one
noarch rpm Provides perl-generators + perl-interpreter (Requires perl), so
'dnf builddep' on the Fedora perl srpms resolves those (SUSE-absent) names.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The mock chroot's %{_host_cpu} is 'powerpc64le' on AlmaLinux ppc chroots but
'ppc64le' on Rocky, so the single ppc64le compare left use_prebuilt unset on
alma and pulled in the ppc-absent gnu-efi BuildRequires. Match both spellings.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
EL8/EL9 mock chroots run an older rpm that does not evaluate an || between a
string compare and an arithmetic test like EL10's rpm, so use_prebuilt stayed
unset on el9-ppc and the gnu-efi BuildRequires (absent on ppc) broke the build.
Use two independent %ifarch ppc64le / %if 0%{?rhel}==8 blocks instead.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
EL8's gnu-efi-devel places elf_x86_64_efi.lds where elilo's Makefile can't find
it, so elilo failed to compile on EL8 -- yet xCAT hard-requires elilo-xcat on
every arch, so the whole dep repo became uninstallable. Reuse the same tracked
prebuilt elilo-x64.efi (SOURCE4) that ppc64le already uses (elilo-x64.efi is a
noarch artifact). Also add elilo-xcat to assert_required_deps so a missing elilo
fails the build loudly instead of surfacing later as a dnf depsolve error.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Port SOURCE_DATE_EPOCH patterns from xcat-core to all xcat-dep build
scripts. RPM and Debian packages now produce identical output given
the same Gitepoch timestamp, regardless of build host or time.
EL (RPM) changes:
- mockbuild-all.pl: --build-timestamp flag, deterministic run_id,
tar --sort/--owner/--group/--mtime, createrepo --revision
- All mockbuild.pl: SOURCE_DATE_EPOCH cascade (CLI > Gitepoch > git > time),
deterministic mock config with SOURCE_DATE_EPOCH in chroot env,
RPM macros for timestamp clamping and fixed buildhost
- goconserver: -trimpath -buildvcs=false, canonical rpmbuild path
- xnba: canonical rpmbuild path
- syslinux: ZERO_AR_DATE=1 in mock env
Ubuntu (Debian) changes:
- All make_deb.sh: SOURCE_DATE_EPOCH fallback from Gitepoch
- goconserver: deterministic SNAP_TS and changelog from epoch,
-trimpath -buildvcs=false in debian/rules
- syslinux: -fdebug-prefix-map for path-independent debug info
- build-apt-repo.sh: gzip -n, Release Date: from SOURCE_DATE_EPOCH
Verified: 31/33 package artifacts produce identical SHA256 hashes
across independent builds. 2 syslinux sub-packages (devel,
debugsource) have known limitations from upstream build system.
Daniel Hilst
Mark Gurevich
cxhong
immarvin
xq2005
arifali
jbjohnso