Repository navigation
New test-farm for longer term app testing - #1167
Merged
Merged
Conversation
Onboarding's Navigating and Battery screens skip themselves when there is nothing to grant (since cb3c90d), so tapping their buttons unconditionally failed the flow and every flow after it. Only tap them if the screen shows, and don't expect the Battery screen's Continue after the exemption is granted, since that moves on by itself. Markers.yaml saves C at the current location, where LocationDetails.yaml already saved A, and saveMarker merges a new marker into one at the same coordinates, so C replaces A. Assert that, and build the route in RouteCreation.yaml from B and C. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
test-farm/farm.py runs one worker per phone plugged into the PC. Each loops: upgrade to the newest release (install -r over existing data, then a PostUpgrade check), a monkey soak, the Maestro suite, then collects crashes, ANRs, memory and battery into farm.db and report.html. Releases come from the upstream build-app.yaml release-apk artifact. Release builds are made testable without debug-only hooks: firebase.test.lab turns analytics off, and shell mock location providers replay a GPX track (held at CI's Edinburgh point while the suite runs). The farm only tests a build it installed itself, and dims the screen and mutes media volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
The farm phones' Wi-Fi is unreliable, so every cycle now runs offline. The farm keeps the manifest's current Glasgow extract cached (extractProviderUrl from local.properties), adb-pushes it into the app's files/Download after Onboarding wipes app storage, and flows/OfflineSearch.yaml sets Search mode to Offline. The suite passes with Wi-Fi turned off. USB iPhones are observed through pymobiledevice3, since Maestro can't drive a physical iPhone. --userspace builds the iOS 17+ developer tunnel in-process, so nothing needs root. The worker replays the GPX walk with simulate-location play, keeps a daily Soundscape syslog, and every ios_poll_minutes records TestFlight upgrades, relaunches and restarts, pulls Soundscape crash/hang reports and reads the battery. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
Mock fixes from cmd location providers carry no speed or bearing, so Soundscape treated the phones as standing still and took callout directions from the compass of a phone on a desk. A small mock location helper (test-farm/mock-location, built with plain SDK tools, not part of Soundscape) now sets both on every fix. Its package is deliberately outside org.scottishtecharmy.soundscape: under that prefix Maestro's launchApp stopped granting Soundscape's permissions on the AOSP phone. GPX tracks are thinned to points 15 m apart, because a recording that starts with its owner standing still begins with minutes of on-the-spot GPS jitter, which a constant-speed replay walked in place. The iPhone player now logs to runs/<udid>/player.log. Workers are harder to knock over: the farm stops its child processes on shutdown (an orphaned Maestro run kept the phone's driver and failed every flow after a restart), a retry after a gRPC driver error reinstalls the driver, a phone on a PIN lock screen is reported as locked instead of failing every flow, and the location feeder survives adb timeouts (their partial output arrives as bytes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
Add docs/developers/test-farm.md (what the farm does, how it differs from CI, how release builds are made testable, and the setup traps), link it from the developer index, and point to it from the Maestro docs. Also bring the Maestro flow table up to date: Markers.yaml's C replaces A, and RouteCreation.yaml builds its route from B and C. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
A worker could launch its next Maestro flow after shutdown had already stopped the farm's children, leaving an orphaned run holding the phone's driver. run() now refuses to start anything once the farm is stopping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
When a location's description starts with its name, as for roads and street addresses, the description replaced the whole content description, so TalkBack and VoiceOver users heard neither the type nor the distance and direction that sighted users see. Keep using the description in place of the name, but always add the type and distance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
Compose dialogs are separate windows outside the theme's testTagsAsResourceId, so Maestro couldn't see their tags. Add a common Modifier.testTagsAsResourceId() (a no-op on iOS) and apply it to the AlertDialogs and the search dialog. Tag the Settings sections, the search mode and show map preferences, list preference options and search results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
…nkey - Set up once per release (Onboarding and the suite), then run shuffled roaming flows that navigate the UI while the GPS walks. - Keep the app running across flows (LaunchHome brings it to the front and backs out to Home) and record the app process's uptime. - After the monkey, Restore puts back search mode, the map setting and markers B and C and Test Route without restarting the app; the phone is only set up again if that fails. It also completes a set-up whose suite flows failed. - Per-phone show_map, off for the low-memory Cubot. - Wi-Fi off on every phone; the offline map is pushed, or downloaded in-app through a USB proxy (adb reverse) where the shell can't write the app's files. - Per-phone Maestro cache and state dirs, so concurrent runs don't prune each other's logs and hang. - SwipeAndTap starts its swipe at 60% so small screens don't swipe from under the bottom bar; HomePage scrolls to About. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
dumpsys dropbox only prints entries that match every argument, so asking for four tags in one call printed nothing and the farm never recorded a crash or ANR on any phone. The Cubot's monkey ANRs were in dropbox but never reached the farm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
A new test-farm job compiles farm.py, builds the mock location helper and runs test-farm/test_farm.py, which needs no phone. It checks: - the example config only uses settings farm.py knows - the flows' runFlow paths into maestro/ resolve - the suite location and launch-failure retries still agree with run-maestro-tests.yaml - the dropbox, uptime, meminfo and battery parsers, and GPX track replay Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ex6Fdp5ftuhE1ZEyhZb7WQ
davecraig
had a problem deploying
to
development
October 16, 2026 09:55 — with 
GitHub Actions
Failure
GitHub Actions
Failure
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.