Skip to content

CI: make sure GNU is the system coreutils in the jobs that compare against it - #15024

Open
abendrothj wants to merge 1 commit into
uutils:mainfrom
abendrothj:ci-runner-gnu-coreutils
Open

abendrothj wants to merge 1 commit into
uutils:mainfrom
abendrothj:ci-runner-gnu-coreutils

Conversation

@abendrothj

@abendrothj abendrothj commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Newer Ubuntu images ship uutils as the system coreutils, so the GNU test job and the two fuzz jobs that compare against the system coreutils could end up comparing uutils with itself.

Each of those jobs now has the same step: if coreutils-from-uutils is installed, it installs coreutils-from-gnu in its place, and the job fails if that install does. Runner images are unchanged.

Should land after #15005: once the system coreutils are GNU, the check added there is what stops the root tests from passing on GNU's binaries.

Closes #15023

Copilot AI balanced review requested due to automatic review settings October 2, 2026 02:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/touch/now-owned-by-other is no longer failing!

@oech3

oech3 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Would you change runner of fuzzing.yml to ubuntu-26.04? Then we can use newer GNU coreutils at fuzzing.

@abendrothj
abendrothj force-pushed the ci-runner-gnu-coreutils branch from bf81051 to 23333fd Compare October 4, 2026 00:23
Copilot AI balanced review requested due to automatic review settings October 4, 2026 00:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@abendrothj

Copy link
Copy Markdown
Contributor Author

Done in 23333fd. uufuzz-examples and fuzz-run now run on ubuntu-26.04, since those are the jobs that compare against the system coreutils. Because coreutils-from-gnu is always available there, the GNU install no longer needs the apt-cache show check. detect, fuzz-build and fuzz-summary don't run the system coreutils, so I left them on ubuntu-latest.

@xtqqczze xtqqczze left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't change runner image in same PR, instead detect whether coreutils-from-gnu is available.

Comment thread .github/workflows/fuzzing.yml Outdated
needs: detect
if: needs.detect.outputs.all == 'true'
runs-on: ubuntu-latest
runs-on: ubuntu-26.04

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runs-on: ubuntu-26.04
runs-on: ubuntu-latest

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 47face8: back on ubuntu-latest, and the PR doesn't change any runner now.

Comment thread .github/workflows/fuzzing.yml Outdated
## The examples compare against the system coreutils, which must be GNU.
## ubuntu-26.04 ships uutils there.
sudo apt-get update
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
- name: Use GNU as the system coreutils
shell: bash
run: |
## The examples compare against the system coreutils, which must be GNU.
## ubuntu-26.04 ships uutils there; older images have no such switch.
sudo apt-get update
if apt-cache show coreutils-from-gnu >/dev/null 2>&1; then
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to introduce such complex detection instead of using host that we are 100% sure that the package is available?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package being available on a specific host doesn’t necessarily justify coupling the CI job to that host.

We’re interested in whether coreutils-from-gnu is available, not specifically whether we’re on Ubuntu 26.04. The check expresses that requirement directly and keeps the workflow resilient to runner changes.

@oech3 oech3 Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just change the job when we lost it. No need to detect dynamically.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we should catch failure instead of silently fallbacking to uutils (when package name was changed by some reason).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's outside of the scope of the linked issue #15023 to change the runner image to ubuntu-26.04.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair, a renamed package is the only case where they differ, and it's unlikely. two smaller differences: where GNU is already the system coreutils (ubuntu-latest today) the step skips apt-get update, and the check after the install fails the job if the switch didn't happen, which oech3 asked for. if you'd still rather have apt-cache show, I can switch.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a much higher likelihood that utilities are added to or removed from the coreutils-from-uutils package than that the package itself is renamed. For example, ls could be removed from the package while coreutils-from-uutils remains installed. In that case, relying on ls --version to detect the switch would be unreliable.

@oech3 oech3 Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is hard to imagine, but we can

for _b in $(target/release/coreutils --list);do
  sudo cp -sf /usr/bin/gnu${_b} /usr/bin/${_b}
done

cp -sf fails at missing source binary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, switched to the apt-cache show step. After it, the step fails if coreutils-from-uutils is still installed, which covers every utility, not just ls: on 26.04 cp, mv, rm, df and true already point at GNU while ls is uutils. I didn't use the cp -sf loop: on 26.04 it stops at the first utility without a GNU twin (cp: cannot stat '/usr/bin/gnuchroot'), and the fuzz jobs don't build coreutils before this step.

Comment thread .github/workflows/fuzzing.yml Outdated
needs: detect
if: needs.detect.outputs.targets != '[]'
runs-on: ubuntu-latest
runs-on: ubuntu-26.04

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runs-on: ubuntu-26.04
runs-on: ubuntu-latest

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, ubuntu-latest again.

Comment thread .github/workflows/GnuTests.yml Outdated
Comment on lines +62 to +64
## ubuntu-26.04 ships uutils as the system coreutils; give configure and the
## test harness GNU's tools instead of whichever rust-coreutils the image has.
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## ubuntu-26.04 ships uutils as the system coreutils; give configure and the
## test harness GNU's tools instead of whichever rust-coreutils the image has.
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
- name: Use GNU as the system coreutils
shell: bash
run: |
## The examples compare against the system coreutils, which must be GNU.
## ubuntu-26.04 ships uutils there; older images have no such switch.
sudo apt-get update
if apt-cache show coreutils-from-gnu >/dev/null 2>&1; then
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
fi

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd keep this one unconditional. This job is already pinned to ubuntu-26.04 on main (# some strace? test needs newer glibc), so the package is always there, and if it ever isn't, a failed install is the right outcome for the GNU job.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be great if we weren't as tightly coupled to the runner image in case we want to change it in the future

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check is useful if and only if we know every possible error. But it is very hard to do that. We should just do what we want to do and fail.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f1b4415 does that. There's no availability check anymore, and all three jobs run the same step: if ls --version already says GNU it does nothing, otherwise it installs coreutils-from-gnu and then fails unless ls --version reports GNU. A missing or renamed package fails the apt install, and an install that doesn't switch fails the check, so this job isn't tied to 26.04 either. I ran the step in Ubuntu containers: 24.04 stays on GNU 9.4 without apt, 26.04 switches from uutils 0.8.0 to GNU 9.7, a renamed package fails with 100, and an install that doesn't switch fails the check.

@oech3

oech3 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

This PR has 2 commit to separate things. No worry about confusion.

@abendrothj
abendrothj force-pushed the ci-runner-gnu-coreutils branch from 23333fd to 47face8 Compare October 5, 2026 20:48
Copilot AI balanced review requested due to automatic review settings October 5, 2026 20:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI balanced review requested due to automatic review settings October 6, 2026 07:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@abendrothj abendrothj changed the title CI: use GNU as the system coreutils on ubuntu-26.04 runners CI: make sure GNU is the system coreutils in the jobs that compare against it Oct 6, 2026
@abendrothj

Copy link
Copy Markdown
Contributor Author

The runner changes are gone, and all three jobs run on the image they had on main. Rather than probing for coreutils-from-gnu, the step checks what's actually installed: it does nothing if ls --version is already GNU, otherwise it installs the package and fails unless the result is GNU. That keeps it independent of the Ubuntu version without falling back silently to uutils (f1b4415). This is meant to land after #15005.

@oech3

oech3 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Actually, fuzzer itself is designed to reject non-GNU coreutils on $PATH. So we can wait doing this until ubuntu-latest was updated and we see actual error.

@abendrothj

Copy link
Copy Markdown
Contributor Author

the native GNU job already runs on ubuntu-26.04, and its runner has uutils as the system coreutils: this PR's run removes coreutils-from-uutils there. so that job is affected now, and the fuzz jobs follow when ubuntu-latest moves to 26.04.

Copilot AI balanced review requested due to automatic review settings October 10, 2026 04:26
@abendrothj
abendrothj force-pushed the ci-runner-gnu-coreutils branch from f1b4415 to 44e0404 Compare October 10, 2026 04:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

if apt-cache show coreutils-from-gnu >/dev/null 2>&1; then
sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
fi
if dpkg -s coreutils-from-uutils >/dev/null 2>&1; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do check whether coreutils-from-uutil is installed when we've just uninstalled it? if the previous operation succeeded than this state is impossible

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If coreutils-from-gnu isn't in the index (renamed, say), the install is skipped and uutils stays the system coreutils. This check is what fails the job then.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can check whether coreutils-from-uutils is installed, why not:

        if dpkg -s coreutils-from-uutils >/dev/null 2>&1; then
          sudo apt-get update
          sudo apt-get install -y --allow-remove-essential coreutils-from-gnu coreutils-from-uutils-
        fi

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, changed.

@abendrothj
abendrothj force-pushed the ci-runner-gnu-coreutils branch from 44e0404 to b0003f1 Compare October 11, 2026 21:46
Copilot AI balanced review requested due to automatic review settings October 11, 2026 21:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@xtqqczze

xtqqczze commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

@abendrothj please rewrite the commit description, it reads like AI and doesn't really add anything

Comment thread .github/workflows/fuzzing.yml Outdated
- name: Use GNU as the system coreutils
shell: bash
run: |
## Images such as ubuntu-26.04 ship uutils as the system coreutils; switch to GNU, or fail.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure these comments are adding much value.

Copilot AI balanced review requested due to automatic review settings October 11, 2026 22:40
@abendrothj

Copy link
Copy Markdown
Contributor Author

Fair, dropped the commit body and the comments.

@abendrothj
abendrothj force-pushed the ci-runner-gnu-coreutils branch from b0003f1 to ed8c931 Compare October 11, 2026 22:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: jobs on ubuntu-26.04 run with the runner's rust-coreutils as the system coreutils

4 participants