Skip to content

C# test-project detection has no stated error direction, so reviewers pull it both ways #1270

Open

Description

@ptr727

is_csharp_test_project() in spec/audit.py decides whether a repository owes Codecov coverage. Two reviewers on the promotion #1263 asked for changes to it in opposite directions, and the function has now been revised twice in two days under exactly that pressure. It wants one decision rather than a third revision.

Current behavior

if not path.endswith(".csproj"):
    return False
name = path.rsplit("/", 1)[-1]
segments = name[: -len(".csproj")].lower().split(".")
return (
    "Test" in name
    or any(segment in ("test", "tests") for segment in segments)
    or in_test_directory(path)
)

The two pulls

Widen it. Qodo: src/appunittest.csproj and src/app_unit_test.csproj are valid test projects and are missed, so coverage_claiming_types() omits csharp and the audit skips the Codecov secret, codecov.yml, and the token checks. A missed project is a silent clean report.

Narrow it. CodeRabbit: "Test" in name also matches Testament.csproj and TestUtilities.csproj, so a repository with no test project is told to store a token and ship a codecov.yml. A false positive is a visible wrong finding.

Both are right about their own case, and each proposed fix worsens the other. Lowercasing the whole basename, the widening move, is what makes Contest.csproj and Latest.csproj match, which is why it was rejected during #1255's review. Requiring a PascalCase or segment boundary, the narrowing move, is what drops appunittest.csproj.

The decision this needs

The function's own asymmetry is already written down in coverage_claiming_types(): a wrong claim is a visible finding, a wrong drop is a silent clean report. That argues for tolerating Testament.csproj over missing appunittest.csproj. But the asymmetry has never been stated as a rule for this predicate, so each reviewer applies its own instinct and the function oscillates.

What would settle it:

  1. State the tolerated error direction in the docstring, so a reviewer arguing the other way is answered by the code rather than by a fresh judgment each round.
  2. Decide whether the predicate should read a name at all, or whether the reliable signal is the project's own IsTestProject property or a test-framework package reference. #1259 already records that an IsTestProject-only project is invisible to a tree read, which is the same gap from a third direction.
  3. Add the cases from both findings to coverage_cases, whichever way it goes, so the next revision cannot silently move the boundary back.

Filed rather than fixed on #1263 because that is a promotion of thirteen reviewed commits, and settling a contested predicate inside it would put unreviewed content on the promotion.

Activity

  1. added
    decisionA question waiting on the maintainer, alone or beside whatever else the issue carries
    on Sep 5, 2026
  2. ptr727 commented on Sep 18, 2026

    @ptr727
    OwnerAuthor

    Decision (ptr727, 2026-09-18): read the project rather than the name. is_csharp_test_project() stops matching on the file name and reads the project's own IsTestProject property or a test-framework package reference instead.

    That answers both pulls at once rather than choosing between them: appunittest.csproj and app_unit_test.csproj are found because the project says what it is, and Testament.csproj and TestUtilities.csproj are not found because they do not. Neither reviewer has to be overruled, and the oscillation ends because there is no boundary left to argue about.

    It also settles #1259 from the same side, that issue recording an IsTestProject-only project being invisible to a tree read. The two are the same gap seen from different directions and should land together.

    Scope this brings with it: the audit reads project file contents rather than a path listing, so whatever reads the tree has to fetch the file. Both findings' cases still go into coverage_cases, since the reason for pinning them does not depend on which predicate is used.

  3. removed
    decisionA question waiting on the maintainer, alone or beside whatever else the issue carries
    on Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions