Skip to content

Fix more tests and tag ones requiring a server - #792

Merged
bastelfreak merged 12 commits into
mainfrom
openvox_suite_windows_macos
Nov 14, 2026
Merged

bastelfreak merged 12 commits into
mainfrom
openvox_suite_windows_macos

Conversation

@nmburgan

Copy link
Copy Markdown
Member

This includes more fixes for silently broken tests, marks several tests with the server tag that need one, and fixes up some tests to run serverless where it makes sense to do so so that we can run as many tests under Windows and MacOS as possible.

initialize_temp_dirs failed when there was no server for every test
that uses the temp file helpers, including tests that only touch
agents. parser_validate's default manifest step uses
mk_tmp_environment_with_teardown, which reads the server's
environmentpath, so it now runs only where there is a server.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
The Windows and MacOS cells run the openvox suite without a server and
skip the tests tagged server. These 35 untagged tests use the server
host or run puppet agent against a server, so they need the tag as
well.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Disabling and enabling the agent need no server, since a disabled agent
stops before it contacts one. Without a server the test now runs those
steps alone and skips the final agent run. With a server nothing
changes.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Without a server, each agent builds the code tree and applies it with
puppet apply, then the same checks run on the results. With a server
nothing changes.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@nmburgan

Copy link
Copy Markdown
Member Author

Their file and report checks were blocks passed to Result#stdout, which
ignores blocks, so they never ran. The file checks now allow the
newline the heredoc writes, and the expected content moves out of the
step block that two of the tests set it in, where the checks could not
see it. supports_utf8's asset_equal typo is fixed too.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
The clientcert data file was named after $fqdn, a Ruby global that is
never set, so the first hierarchy level was always empty and the
assertions expected that. Each agent now gets a data file named after
its certname, the assertions expect that level first, and two failure
messages no longer print nil names.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
v3_config_and_data opened the server's whole confdir, SSL private keys
included, with chmod -R 775 and never put it back. It now opens only
the hiera.yaml it writes, as v4_hieradata_with_v5_configs does.
v4_hieradata_with_v5_configs wrote global data next to its global
hiera.yaml, where hiera never reads it and where the teardown left it
behind, so those writes are gone.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
catalog_uuid passed :acceptance_exit_codes, which Beaker ignores.
puppet_lookup_cmd's trusted facts check passed even when lookup fell
back without the CA, since the output is the same, so it now also
checks that the fallback warning is absent.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@nmburgan
nmburgan force-pushed the openvox_suite_windows_macos branch 2 times, most recently from 3d4f917 to 3784b20 Compare November 13, 2026 22:48
GitHub's Windows runners name the built-in administrator (RID 500)
runneradmin, and Beaker connects as it. Use the account Beaker connects
as where the tests act as that user, which stays Administrator under
Beaker's Windows defaults, and find the built-in account by its RID
where should_purge checks that the purge kept it. QA-506's winexitcode
module no longer sets an owner, whose failure skipped every exec the
test runs.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
OpenVox printed the runner's domain in lower case, while the test upper
cased the hostname. Windows account names are case-insensitive.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Beaker's local connection, which the macOS cells use, never fills a
result's output. The errors these tests expect go to stderr, and
calling_all_functions logs at several levels, so it matches on both
streams.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
QA-506 applied its manifests without --detailed-exitcodes, so an exec
that returned an unexpected exit code failed its resource while puppet
apply still exited 0, and the test passed. Now each apply has to exit 2,
which means its exec ran and returned the expected code.

Signed-off-by: nmburgan <13688219+nmburgan@users.noreply.github.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@nmburgan
nmburgan force-pushed the openvox_suite_windows_macos branch 2 times, most recently from 0367ad0 to 50b22b8 Compare November 14, 2026 00:39
@bastelfreak
bastelfreak merged commit 23becb7 into main Nov 14, 2026
22 checks passed
@bastelfreak
bastelfreak deleted the openvox_suite_windows_macos branch November 14, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants