Skip to content

test: enforce test file stems and remove duplicate error tests - #243

Merged
synackd merged 4 commits into
mainfrom
test/reorganize
Nov 10, 2026
Merged

synackd merged 4 commits into
mainfrom
test/reorganize

Conversation

@synackd

@synackd synackd commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Fixes #232
Fixes #235
Fixes #236

Summary

These are the follow-ups to the test layout work in #218. They close a gap in the one-file-per-subject check, add a check that a subject's bucket files share a stem, and remove per-family error tests that repeat a row of a command-wide table. The change touches only tests, test fixtures, and CONTRIBUTING.md, so it needs no changelog entry.

The commits are meant to be reviewed one at a time. Each one passes make lint reuse mod test.

Changes

#232: test: ignore non-platform tags in subject grouping

platform() used the whole //go:build expression as a grouping key, so a file tagged linux && integration got its own group and could split a subject's tests. It now compares the GOOS/GOARCH pairs a file can build for, rather than the text of its constraints. A file name and a constraint that select the same platforms group together, whatever their order. Tags that don't name a platform don't make a file platform-specific: canBuild tries each setting of them, with one value per tag across the file. canBuild, appendOtherTags, and matchesPlatform replace namesPlatform.

This also fixes cases the issue doesn't name: amd64 && linux vs linux && amd64, windows_amd64 in a file name vs windows && amd64, and linux || darwin vs darwin || linux.

#235: test: keep subject bucket files on one stem

Test From To
TestLoad_ClusterShapeBoundaries, TestLoad_AcceptsNullClusterConfig pkg/config/config_boundary_test.go pkg/config/loader_boundary_test.go
TestValidateConfig_NullAndShapeBoundaries pkg/config/config_boundary_test.go pkg/config/validate_boundary_test.go
TestInitLogging_RejectsInvalidConfiguration internal/cli/runtime_errors_test.go internal/cli/runtime_logging_errors_test.go
TestRuntimeHandlePayload_StdinCancelled internal/cli/runtime_errors_test.go internal/cli/runtime_input_environment_errors_test.go

Each moved test is byte-identical to main. TestBSSHistoryGet_* becomes TestBSSHistory_*, since bss history is a leaf command.

The new TestTestFiles_SubjectBucketsShareStem flags a subject whose plain, _errors, and _boundary files have different stems. It compares only files for the same platforms, which covers TestWriteFile and TestSyncParentDirectory, and it leaves out cmd/commands_errors_test.go, which covers TestServiceCommands. CONTRIBUTING.md documents the rule.

#236: test: remove error tests that repeat table rows

This removes 19 tests that run the same command with the same failure as a table row and assert nothing beyond the error and exit code. cmd/bss_boot_script_errors_test.go held only one of them, so it is deleted.

Table Removed
TestServiceCommands_NetworkError TestSMDGroupGet_NetworkError, TestSMDCompepGet_NetworkError, TestSMDComponentGet_NetworkError, TestSMDIfaceGet_NetworkError, TestSMDRFEGet_NetworkError, TestBSSBootParamsGet_AllNetworkError, TestCloudInitGroupGet_NetworkError
TestWriteCommands_NetworkError TestSMDGroupAdd_NetworkError
TestServiceCommands_HTTPError TestSMDGroupGet_HTTPError, TestSMDCompepGet_AllHTTPError, TestSMDComponentGet_AllHTTPError, TestSMDIfaceGet_HTTPError, TestSMDRFEGet_HTTPError, TestBSSBootParamsGet_AllHTTPError, TestBSSBootScriptGet_HTTPError, TestCloudInitGroupGet_HTTPError, TestPCSTransitionList_HTTPError
TestCommands_MalformedPayload TestCloudInitGroupAdd_MalformedPayload, TestCloudInitNodeSet_MalformedPayload

The policy is now in the "Command tests" section of CONTRIBUTING.md: a per-family error test stays when it uses a different selector, flag, or stub response than its row, or asserts more than the exit code.

Beyond the issues: test: cover deprecated status and vendor-data errors

Auditing #236 turned up three related problems, fixed here:

  • TestSMDDeprecatedStatus_HTTPError ran smd service status, not the deprecated command, so it is now TestSMDServiceStatus_HTTPError.
  • smd status and bss status hold their own copy of the status logic, and no test asserted their exit codes. Both now have rows in TestServiceCommands_NetworkError and TestServiceCommands_HTTPError.
  • TestCloudInitNodeGetData_HTTPError repeated TestCloudInitNodeGet_MetadataHTTPError, while cloud-init node get vendor-data had no HTTP error test. It now tests vendor-data, as TestCloudInitNodeGet_VendordataHTTPError.

Review notes

Manual testing

make lint reuse mod test   # passes, 0 lint issues, no go.mod/go.sum changes

# 1024 on main: 2 new tests, 19 removed
go test ./... -list . | grep -c '^Test'   # 1007

The stem script from #235 now prints only the intentional cases:

cmd TestServiceCommands ['commands', 'service_status']
pkg/config TestSyncParentDirectory ['sync', 'sync_windows']
pkg/config TestWriteFile ['owner', 'write']

The rest was checked once by hand:

Checklist

  • My code follows the style guidelines of this project
  • I have added/updated comments where needed
  • I have added tests that prove my fix is effective or my feature works
  • I have run make test (or equivalent) locally and all tests pass
  • I have updated the relevant documentation (CLI examples, man pages, README, other docs, etc.) (CONTRIBUTING.md)
  • DCO Sign-off: All commits are signed off (git commit -s) with my real name and email
  • REUSE Compliance:
    • Each new/modified source file has SPDX copyright and license headers
    • Any non-commentable files include a <filename>.license sidecar
    • All referenced licenses are present in the LICENSES/ directory

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Dependency update
  • Build system/CI

The test file check groups a subject's tests by platform, but platform()
used the whole //go:build expression as the group. A file tagged "linux
&& integration" got its own group, apart from *_linux_test.go, so a
non-platform tag could split a subject's tests without a violation. The
same goes for equivalent constraints written differently, such as
"amd64 && linux" and *_linux_amd64_test.go.

Compare the GOOS/GOARCH pairs a file can build for rather than the text
of its constraints. A file name and a constraint that select the same
platforms now group together, whatever their order. Tags that don't name
a platform don't make a file platform-specific: each setting of them is
tried, with one value per tag across the file, so "(linux || integration)
&& !integration" still counts as Linux. This replaces namesPlatform.

Signed-off-by: Devon Bautista <17506592+synackd@users.noreply.github.com>
CONTRIBUTING.md names a subject's files <subject>_test.go,
<subject>_errors_test.go, and <subject>_boundary_test.go, but nothing
checks that those files share a stem. TestLoad, TestValidateConfig,
TestInitLogging, and TestRuntimeHandlePayload each had a bucket in a file
named after something else, so a reader who opens the plain test file
wouldn't find the rest next to it.

Move those tests to files that match, without changing them. Add a check
that flags a subject's bucket files with different stems, and describe
the rule in CONTRIBUTING.md. The check compares only files for the same
platforms, using platform(), and leaves out the error tables across cmd
families in commands_errors_test.go, whose plain tests are in a file
named after the behavior.

Also rename TestBSSHistoryGet_* to TestBSSHistory_*, since "bss history"
is a leaf command and there is no "bss history get".

Signed-off-by: Devon Bautista <17506592+synackd@users.noreply.github.com>
The tables in commands_errors_test.go guarantee each command's exit code
for a failure. Some per-family error tests ran the same command with the
same failure and asserted nothing more than the row, at the cost of a
second server and run, and had to be kept in step with the table.

Remove the 19 tests that exactly repeat a row, along with
bss_boot_script_errors_test.go, which held only one of them. Coverage of
cmd, internal/cli, and pkg from the cmd tests is unchanged.

Keep the tests that cover more, and record the rule in CONTRIBUTING.md:
those with a different selector, flag, or stub response, and those that
assert the request, message, or output. The tests for commands with no
table row also stay.

Signed-off-by: Devon Bautista <17506592+synackd@users.noreply.github.com>
"smd status" and "bss status" hold their own copy of the status logic,
so tests of "service status" say nothing about them, and no test
asserted their exit codes. Add both to the service command error tables.

TestSMDDeprecatedStatus_HTTPError ran "smd service status" rather than
the deprecated command, so rename it TestSMDServiceStatus_HTTPError.

TestCloudInitNodeGetData_HTTPError repeated
TestCloudInitNodeGet_MetadataHTTPError, while "cloud-init node get
vendor-data" had no HTTP error test. Point it at vendor-data, and rename
it to match the tests of the other data types.

Signed-off-by: Devon Bautista <17506592+synackd@users.noreply.github.com>
@synackd
synackd marked this pull request as ready for review November 5, 2026 17:16
@davidallendj
davidallendj self-requested a review November 10, 2026 19:28

@davidallendj davidallendj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I tested this and all tests passed with the exception of the few that deliberately fail due to the changes.

=== RUN   TestTestFiles_SubjectInOneFile
    conventions_test.go:933: internal/conventions/pr243probe.ZVEw0Z/widget_linux_test.go:11: TestWidget tests (plain) are split between widget_integration_test.go and widget_linux_test.go
    conventions_test.go:933: Move the test into its subject's file for that bucket, or give a too-broad subject a more specific name; see CONTRIBUTING.md.
--- FAIL: TestTestFiles_SubjectInOneFile (0.08s)
FAIL
FAIL    github.com/openchami/ochami/internal/conventions        0.330s
FAIL
=== RUN   TestTestFiles_SubjectBucketsShareStem
    conventions_test.go:941: pkg/config/pr243_wrong_boundary_test.go:17: TestLoad tests are in loader_errors_test.go and pr243_wrong_boundary_test.go, which have different stems
    conventions_test.go:941: Name a subject's bucket files <stem>_test.go, <stem>_errors_test.go, and <stem>_boundary_test.go. See CONTRIBUTING.md.
--- FAIL: TestTestFiles_SubjectBucketsShareStem (0.08s)
FAIL
FAIL    github.com/openchami/ochami/internal/conventions        0.746s
FAIL

@synackd
synackd merged commit 818d7ae into main Nov 10, 2026
19 checks passed
@synackd
synackd deleted the test/reorganize branch November 10, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants