Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/check-action-file.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ on:
- 'scripts/select_coverage_artifacts.py'
- 'scripts/test_select_coverage_artifacts.py'
- 'scripts/test_coverage_native_runtime.py'
- 'scripts/test_coverage_build_mode.py'
- 'scripts/run_coverage_ut.py'
- 'scripts/test_merge_trigger_tke_subject.py'
- 'scripts/compact_coverage_profile.go'
- 'scripts/compact_coverage_profile_test.go'
Expand Down Expand Up @@ -103,6 +105,8 @@ jobs:
run: python3 scripts/test_select_coverage_artifacts.py -v
- name: Test coverage native runtime paths
run: python3 scripts/test_coverage_native_runtime.py -v
- name: Test coverage build mode contracts
run: python3 scripts/test_coverage_build_mode.py -v
- name: Test TKE merge subject identity contract
run: python3 scripts/test_merge_trigger_tke_subject.py -v
- name: Test SCA license scope selection
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/coverage-ut.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ name: MatrixOne UT Coverage
on:
workflow_call:
inputs:
simd_coverage:
description: "Test coverage in the production SIMD build mode"
required: false
type: boolean
default: false
cache_seed:
description: "Opt-in host-ut-v1 cache canary; keep disabled until end-to-end cost is measured"
required: false
Expand Down Expand Up @@ -230,6 +235,7 @@ jobs:
env:
GOPROXY: http://goproxy.goproxy.svc.cluster.local,https://goproxy.cn,https://proxy.golang.org
COVERAGE_UT_TEST_TIMEOUT_SECONDS: "3600"
SIMD_COVERAGE: ${{ inputs.simd_coverage }}
# The go test package deadline below is 20m; it must leave time for
# the runner's 60m whole-suite deadline and diagnostics.
COVERAGE_UT_HEARTBEAT_SECONDS: "15"
Expand Down Expand Up @@ -327,6 +333,13 @@ jobs:
fi
}

# Select coverage build mode before any Go discovery or preparation.
if [ "${SIMD_COVERAGE:-false}" = 'true' ]; then
export GOEXPERIMENT=simd
export GOAMD64=v3
fi
go env GOVERSION GOOS GOARCH GOEXPERIMENT GOAMD64

phase_start discover_packages
package_list="${RUNNER_TEMP}/ut-coverage-packages.txt"
set +e
Expand Down
71 changes: 71 additions & 0 deletions docs/design/simd-ut-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Production-mode SIMD unit-test coverage

Owner: [MatrixOne issue #32325](https://github.com/matrixorigin/matrixone/issues/32325).
The requester approved this two-PR design before implementation: first add the
shared coverage input, then enable it in a separate MatrixOne CI-only PR.

## Problem and contract

MatrixOne's production build enables `GOEXPERIMENT=simd` and `GOAMD64=v3` on
amd64. The dedicated UT coverage producer invokes Go directly, bypassing MO's
test script and excluding SIMD-tagged implementations and tests. Expression
PR [matrixorigin/matrixone#32237](https://github.com/matrixorigin/matrixone/pull/32237)
therefore fails changed-code coverage despite existing backend tests.

Add boolean `simd_coverage`, default `false`, to `coverage-ut.yaml`. Omitted or
false preserves the caller's environment. True selects `GOEXPERIMENT=simd` and
`GOAMD64=v3` before discovery, instrumentation, native preparation, and test
execution. Log compiler, OS, architecture, experiment and architecture level.
The explicit request is never silently retried in scalar mode.

The workflow step owns these process-local environment exports. It does not
write persistent Go configuration. The existing coverage runner inherits them;
its execution, cancellation and cleanup contracts need no changes.

## Decisions and bounds

Keep opt-in local to the coverage workflow. A global SIMD default would change
legacy callers. Changing only MO's test script cannot reach this producer.
Duplicating a coverage lane or adding SQL cases for internal permutations would
increase cost without correcting the build-mode mismatch.

Keep package scope, assertions, coverage thresholds/exclusions, artifact names
and generations, deadlines, permissions, and eligibility checks unchanged.
The ordinary MO UT lane remains scalar. Go validates explicitly requested modes;
unsupported requests fail before producing an eligible profile. Existing Go
cache keys distinguish compiler/experiment/architecture modes; no cache-key or
SDK changes are required. Record actual producer duration/cache observations.

This adds one workflow input, no runtime API, resource owner, background process,
kernel change, database format or migration. The change introduces no additional
all-package test execution.

## Validation

Run fast contract tests for omitted/false/true input, preservation of ambient
values, ordering before discovery and native preparation, inheritance through
the actual runner, and failure propagation. Tests use fake Go commands because
the CI validator runs on arm64 with Go 1.26.4. Enforce them through its existing
workflow validator, alongside schema validation and actionlint.

The MO caller PR adds only `simd_coverage: true` to the main-branch coverage call.
It starts from main independently of #32237. Native/SIMD execution is validated
by MO's supported amd64 coverage runner, not by the CI repository's validator.

## Rollout and completion

1. Merge the shared-CI prerequisite while existing callers retain default-off.
2. Publish and merge the separate MO caller PR after the shared ref accepts the
input. Link both PRs to #32325 using the PR template.
3. Merge newest authoritative MO main normally into #32237 and trigger one
fresh CI generation. `pull_request_target` runs the base workflow, so the
caller PR's own pre-merge CI cannot prove opt-in execution.
4. Verify source/mode identity, SIMD test execution, ordinary scalar UT, and
merged UT+BVT changed-code coverage strictly greater than 75%. Publish the
real result before closing #32325. Local function-only coverage is supporting
evidence, not a replacement for the required verdict.

The MO opt-in can be reverted independently while the shared input remains
default-off. Rollback does not waive #32237's required coverage check. New
performance optimizations and #32266's known regressions remain separate work;
CI-only configuration does not require another TPC-H comparison.
168 changes: 168 additions & 0 deletions scripts/test_coverage_build_mode.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
import json
import os
from pathlib import Path
import re
import shlex
import subprocess
import sys
import tempfile
import textwrap
import unittest


ROOT = Path(__file__).resolve().parents[1]
WORKFLOW = ROOT / ".github/workflows/coverage-ut.yaml"
RUNNER = ROOT / "scripts/run_coverage_ut.py"


class CoverageBuildModeTest(unittest.TestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory()
self.addCleanup(temporary.cleanup)
self.root = Path(temporary.name)
self.events = self.root / "go-events.jsonl"
self.workflow = WORKFLOW.read_text()
start = self.workflow.index(" # Select coverage build mode")
end = self.workflow.index(" phase_start discover_packages", start)
self.mode = textwrap.dedent(self.workflow[start:end])
executable = self.root / "go"
executable.write_text(
f"#!{sys.executable}\n"
+ textwrap.dedent(
"""
import json
import os
from pathlib import Path
import sys

args = sys.argv[1:]
values = {name: os.environ.get(name) for name in ['GOEXPERIMENT', 'GOAMD64']}
with Path(os.environ['FAKE_GO_EVENTS']).open('a') as output:
output.write(json.dumps({'args': args, **values}) + '\\n')
if args[0] == 'env':
if os.environ.get('REJECT_MODE') == 'true':
sys.exit(37)
metadata = {'GOVERSION': 'test-compiler', 'GOOS': 'test-os', 'GOARCH': 'test-arch'}
for name in args[1:]:
print(metadata.get(name, values.get(name) or ''))
elif args[0] == 'list':
print('test/package')
elif args[0] == 'test':
print(json.dumps({'Action': 'pass', 'Package': 'test/package'}))
else:
sys.exit(29)
"""
)
)
executable.chmod(0o755)

def run_mode(self, enabled=None, inherited=None, reject=False, runner=False):
self.events.unlink(missing_ok=True)
env = os.environ.copy()
for name in ["GOEXPERIMENT", "GOAMD64", "SIMD_COVERAGE", "REJECT_MODE"]:
env.pop(name, None)
env.update(inherited or {})
env["PATH"] = str(self.root) + os.pathsep + env["PATH"]
env["FAKE_GO_EVENTS"] = str(self.events)
if enabled is not None:
env["SIMD_COVERAGE"] = enabled
if reject:
env["REJECT_MODE"] = "true"
status = self.root / "status.json"
report = self.root / "report.json"
if runner:
command = shlex.join(
[
sys.executable,
str(RUNNER),
"--report", str(report),
"--progress", str(self.root / "progress.txt"),
"--status", str(status),
"--timeout-diagnostics", str(self.root / "timeout.txt"),
"--timeout-seconds", "5",
"--", "go", "test",
]
)
else:
command = "go test"
result = subprocess.run(
["bash", "-eu", "-c", self.mode + "\ngo list -mod=readonly ./...\n" + command],
cwd=self.root,
env=env,
capture_output=True,
text=True,
check=False,
timeout=15,
)
events = [json.loads(line) for line in self.events.read_text().splitlines()]
return result, events, status, report

def assert_mode(self, events, experiment, architecture):
self.assertEqual([event["args"][0] for event in events], ["env", "list", "test"])
self.assertEqual(
events[0]["args"],
["env", "GOVERSION", "GOOS", "GOARCH", "GOEXPERIMENT", "GOAMD64"],
)
for event in events:
self.assertEqual(event["GOEXPERIMENT"], experiment)
self.assertEqual(event["GOAMD64"], architecture)

def test_input_defaults_off_and_precedes_all_go_phases(self):
declaration = re.search(
r"^ simd_coverage:\n((?:^ .*\n)+)", self.workflow, re.MULTILINE
)
self.assertIsNotNone(declaration)
self.assertRegex(declaration[1], r"(?m)^ type: boolean$")
self.assertRegex(declaration[1], r"(?m)^ default: false$")
self.assertIn("SIMD_COVERAGE: ${{ inputs.simd_coverage }}", self.workflow)
selected = self.workflow.index(self.mode.splitlines()[0].strip())
for marker in [
"phase_start discover_packages",
"phase_start build_coverage_scope",
"phase_start prepare_cgo",
'python "$GITHUB_WORKSPACE/.ci-workflow/scripts/run_coverage_ut.py"',
]:
self.assertLess(selected, self.workflow.index(marker))

def test_omitted_and_disabled_preserve_environment(self):
for enabled in [None, "false"]:
for inherited in [{}, {"GOEXPERIMENT": "nosimd", "GOAMD64": "v2"}]:
with self.subTest(enabled=enabled, inherited=inherited):
result, events, _, _ = self.run_mode(enabled, inherited)
self.assertEqual(result.returncode, 0, result.stderr)
self.assert_mode(events, inherited.get("GOEXPERIMENT"), inherited.get("GOAMD64"))

def test_enabled_selects_production_mode(self):
result, events, _, _ = self.run_mode(
"true", {"GOEXPERIMENT": "nosimd", "GOAMD64": "v1"}
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assert_mode(events, "simd", "v3")

def test_real_runner_inherits_selected_or_preserved_environment(self):
for enabled, experiment, architecture in [("true", "simd", "v3"), ("false", "nosimd", "v2")]:
with self.subTest(enabled=enabled):
result, events, status, report = self.run_mode(
enabled, {"GOEXPERIMENT": "nosimd", "GOAMD64": "v2"}, runner=True
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assert_mode(events, experiment, architecture)
self.assertEqual(json.loads(status.read_text())["result"], "passed")
self.assertEqual(json.loads(report.read_text())["Action"], "pass")

def test_rejected_mode_stops_before_discovery_and_runner(self):
result, events, status, report = self.run_mode("true", reject=True, runner=True)
self.assertEqual(result.returncode, 37)
self.assertEqual([event["args"][0] for event in events], ["env"])
self.assertFalse(status.exists())
self.assertFalse(report.exists())

def test_validator_enforces_build_mode_contracts(self):
validator = (ROOT / ".github/workflows/check-action-file.yaml").read_text()
self.assertIn(" - 'scripts/test_coverage_build_mode.py'", validator)
self.assertIn(" - 'scripts/run_coverage_ut.py'", validator)
self.assertIn("run: python3 scripts/test_coverage_build_mode.py -v", validator)


if __name__ == "__main__":
unittest.main()
Loading