CI: make the artifact checks capable of failing, and of passing

The rugnux tarball built and packaged correctly; the step that verifies it did
not. Two shell mistakes, both mine, both of the same family.

`! find /tmp/rgx -name 'libcufft*'` can never pass: find exits 0 whether or not
it matched anything, so the negation always fails. That is what broke the x86_64
job after a successful build.

The aarch64 checks were worse, in the way that matters. Piping a large producer
into `grep -q` kills the producer with SIGPIPE as soon as grep exits on its first
match, and `set -o pipefail` promotes that to the pipeline's status: on
`cuobjdump --list-elf | grep -q sm_90` over a 225 MB binary it fails a check that
should pass, and on `! ... | grep -q "x86-64"` the leading `!` inverts it into a
PASS -- so an x86 file leaking into an aarch64 tarball, the exact thing that check
exists to catch, would have been reported as clean.

Both now capture each producer's output to a file and grep the file, with explicit
if/exit rather than exit-status negation.

Verified by running the checks verbatim against the real artifacts rather than
only reading them: both tarballs pass, and the checks were confirmed to FAIL when
they should -- an x86_64 file planted in the extracted tree is caught, and an
architecture absent from the fatbin is reported missing.

The toolchain file now comes from the checkout rather than /opt/cross in the
image. It describes how to build this source, so it belongs with the source:
baked into the image, the Eigen fix in the previous commit could not reach CI
without rebuilding and re-pushing the image, and appeared to have no effect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQjneRUssfhi1k9rq8Ts3h

.gitea/workflows/build_and_test.yml

@@ -212,9 +212,15 @@ jobs:
           rm -rf /tmp/rgx && mkdir -p /tmp/rgx
           tar xf build-rugnux/rugnux-*.tar.gz -C /tmp/rgx
           bin=$(find /tmp/rgx -name rugnux -type f)
-          # cuFFT is static now: no libcufft beside the binary and none needed at run time.
-          ! ldd "$bin" | grep -q cufft
-          ! find /tmp/rgx -name 'libcufft*'
+          # Each producer writes to a file and the greps read files, rather than piping into
+          # `grep -q`: grep exits at the first match, the producer then dies of SIGPIPE, and under
+          # `set -o pipefail` that becomes the pipeline's status -- which a leading `!` would turn
+          # into a pass, hiding the very thing being tested.
+          ldd "$bin" > /tmp/rgx.ldd 2>&1 || true
+          if grep -q cufft /tmp/rgx.ldd; then echo "cuFFT is linked dynamically"; exit 1; fi
+          find /tmp/rgx -name 'libcufft*' > /tmp/rgx.cufft
+          if [ -s /tmp/rgx.cufft ]; then echo "a libcufft is shipped in the tarball"; exit 1; fi
+          echo "self-contained: static cuFFT, nothing to ship beside it"
       - name: Upload rugnux tgz to release
         if: github.ref_type == 'tag'
         shell: bash
@@ -284,6 +290,10 @@ jobs:
     # Cross-compiled on an x86_64 runner: nvcc, gcc and cmake all run native and only emit aarch64,
     # so this costs about what a normal build costs. Uses the ordinary ubuntu2404 image, which now
     # also carries the aarch64 toolchain and the CUDA sbsa cross tree.
+    #
+    # The toolchain file comes from the CHECKOUT, not from the image. It describes how to build this
+    # source, so it belongs with the source: baked into the image it could only be changed by
+    # rebuilding and re-pushing the image, and a fix to it would appear to have no effect.
     runs-on: jfjoch_ubuntu2404
     container:
       image: gitea.psi.ch/leonarski_f/jfjoch_ubuntu2404:2508
@@ -300,7 +310,7 @@ jobs:
         # sm_90 = GH200 (Grace Hopper, e.g. ALPS), sm_121 = GB10 (DGX Spark). One tarball, both.
         run: |
           cmake -G Ninja -S . -B build-aarch64 \
-            -DCMAKE_TOOLCHAIN_FILE=/opt/cross/aarch64-sbsa.cmake \
+            -DCMAKE_TOOLCHAIN_FILE="$PWD/docker/ubuntu2404/aarch64-sbsa.cmake" \
             -DCMAKE_BUILD_TYPE=Release \
             -DJFJOCH_RUGNUX_ONLY=ON \
             -DJFJOCH_USE_CUDA=ON \
@@ -313,20 +323,29 @@ jobs:
         shell: bash
         run: |
           set -euo pipefail
-          tgz=$(ls build-aarch64/*.tar.gz | head -1)
+          tgz=$(ls build-aarch64/rugnux-*.tar.gz | head -1)
           echo "checking $tgz"
           rm -rf /tmp/tgzcheck && mkdir -p /tmp/tgzcheck
           tar xf "$tgz" -C /tmp/tgzcheck
           bin=$(find /tmp/tgzcheck -name rugnux -type f | head -1)
-          file "$bin"
-          file "$bin" | grep -q "ARM aarch64"
+          # Everything is captured to a file first and the greps read files. Piping a large
+          # producer into `grep -q` makes it die of SIGPIPE once grep exits, which `set -o pipefail`
+          # turns into a failure -- and on a negated check into a spurious PASS.
+          file "$bin" > /tmp/tc.bin
+          cat /tmp/tc.bin
+          if ! grep -q "ARM aarch64" /tmp/tc.bin; then echo "not an aarch64 binary"; exit 1; fi
           # nothing x86 may have leaked in (libjpeg-turbo's ExternalProject is the classic culprit)
-          ! find /tmp/tgzcheck -type f -exec file {} + | grep -q "x86-64"
-          # cuFFT must be linked statically: no libcufft beside the binary, none needed at run time
-          ! ldd "$bin" 2>/dev/null | grep -q cufft
-          # both GPU targets present in the fatbin
-          cuobjdump --list-elf "$bin" | grep -q sm_90
-          cuobjdump --list-elf "$bin" | grep -q sm_121
+          find /tmp/tgzcheck -type f -exec file {} + > /tmp/tc.arch
+          if grep -q "x86-64" /tmp/tc.arch; then echo "an x86-64 file leaked into the tarball"; exit 1; fi
+          # cuFFT must be linked statically: nothing to ship beside the binary, nothing to find
+          ldd "$bin" > /tmp/tc.ldd 2>&1 || true
+          if grep -q cufft /tmp/tc.ldd; then echo "cuFFT is linked dynamically"; exit 1; fi
+          # both GPU targets present in the fatbin: sm_90 = GH200, sm_121 = GB10
+          cuobjdump --list-elf "$bin" > /tmp/tc.elf
+          for arch in sm_90 sm_121; do
+            if ! grep -q "$arch" /tmp/tc.elf; then echo "fatbin is missing $arch"; exit 1; fi
+          done
+          echo "aarch64, self-contained, both GPU targets present"
       - name: Upload tarball to release
         if: github.ref_type == 'tag'
         shell: bash

docker/ubuntu2404/Dockerfile

@@ -239,7 +239,8 @@ RUN set -eux; \
     apt-get clean; \
     rm -rf /var/lib/apt/lists/*
 
-COPY aarch64-sbsa.cmake /opt/cross/aarch64-sbsa.cmake
+# aarch64-sbsa.cmake is deliberately NOT copied in: the build passes it from the checkout, so the
+# toolchain is versioned with the source it configures and needs no image rebuild to change.
 
 # Set workdir for your project
 WORKDIR /workspace