Search by voice in the installed app, recognised on the device #405

What problem would this solve?

Typing a destination on a phone is awkward while walking, cycling or holding a coffee, and the installed app offers no alternative: it hides the voice search button the website shows. In the browser, voice search uses the browser's recognition service, which in Chrome sends the recording to Google. For a privacy-focused map that is the wrong default, and it does not work without a network.

What would you like to be able to do?

In the installed app, I tap a microphone button in the search bar, say a place or an address in German or English, and see the recognised text in the search field, where I can correct it before searching. Recognition runs on the phone: the recording never leaves the device and it works without a network. The app listens only while I can see that it is listening, and stops as soon as I cancel, finish or leave the app.

Verified current state

Checked against main at e68106f on 3 November 2026.

• The web VoiceSearchButton uses browser SpeechRecognition and hides itself in the installed shell, whose feature boundary has no microphone.
• The WebView denies media capture. RECORD_AUDIO is blocked in the app config and forbidden by the compliance list, which also declares no iOS microphone use.
• The app's only audio module, openmapx-navigation-audio, speaks guidance and holds audio focus while it does. There is no speech-input module.

Approach

1. Choose the recogniser (first, short step). Start with the operating systems' on-device recognisers: Android's SpeechRecognizer.createOnDeviceSpeechRecognizer (API 31+, after isOnDeviceRecognitionAvailable) and iOS's SFSpeechRecognizer with requiresOnDeviceRecognition. They need no bundled model. Measure them on a small, consented set of German and English place-name recordings. If they are unavailable on common devices or too inaccurate for place names, evaluate a downloadable model (whisper.cpp or Vosk) with its size and licence. Never fall back to cloud recognition. Record the decision in the developer docs.
2. Native module. Add a local Expo module (Kotlin and Swift) that starts, stops and cancels recognition, reports partial and final text and errors, and releases the microphone on every exit path. It coordinates audio focus with openmapx-navigation-audio, so a voice cue and listening never fight.
3. Bridge. Add reviewed messages, with a protocol bump, to start, stop and cancel listening and to deliver partial text, final text and errors. Report a speech-input capability in the handshake only when on-device recognition is available for the current locale. The WebView itself never gets the microphone.
4. Web. In the installed shell, VoiceSearchButton uses the bridge: a visible listening state, a cancel control, and text placed in the search field without submitting. Browser recognition stays as it is for the website and is described separately.
5. Permissions and compliance. Declare RECORD_AUDIO, NSMicrophoneUsageDescription and NSSpeechRecognitionUsageDescription with reasons. Update the compliance list, the data practices and the store answers. Explain the use before the first OS prompt, and handle denial, permanent denial and revocation with a way to settings.

Out of scope: wake words, background or continuous listening, voice commands that control navigation, starting a route without the existing deliberate tap, and a new text-to-speech system.

Acceptance criteria

• The recogniser decision, supported locales, offline guarantee and licences are documented.
• In the installed Android app, tapping the microphone shows a listening state. A spoken German or English place name appears as editable text in the search field, and search runs on that text as if typed.
• Recognition works in airplane mode. No audio leaves the device, and no audio or transcript is stored beyond the search field.
• Cancel, leaving the app, screen off and app teardown each stop listening and release the microphone. A result that arrives after cancelling is discarded.
• During navigation, listening and voice cues coordinate audio focus without losing or blocking guidance.
• Where on-device recognition is unavailable, the button is hidden or explains what to install. It never silently uses a cloud service.
• Permission rationale, denial, permanent denial and revocation are each handled and tested on a device.
• iOS has the same behaviour once the iOS shell runs on a device; until then, the gap is stated in the docs.
• Accuracy and latency on the recording set are published, and the native module, bridge contract and web button have tests.
• Privacy and capability docs are updated, with screenshots of the listening, result and permission states.

Verification approach

Use consented fixed recordings (quiet and noisy) to compare raw transcripts before search normalisation. On a real device, test permissions and lifecycle with the network blocked. Assert that no audio is persisted or transmitted and that late results cause no action.

Dependencies and decisions

• Connect native navigation in the installed mobile app #398 (native navigation, composed on Android in fix(mobile): start the installed app and keep its bridge on Android #756–feat(mobile): draw the app's own screens in the web UI's colours #759) supplies the audio-focus owner this must coordinate with.
• Search downloaded places and addresses without a network #403 (offline search) is needed only for search results without a network; recognition itself works offline without it.
• Decide during the first step: the recogniser, and whether a downloadable model is worth its size where on-device recognition is missing.

Alternatives or workarounds

Keep typing only (today's app). Give the WebView the microphone and use browser recognition: this would send audio to a cloud service and break the shell's no-microphone boundary for the page. Bundle a model for every device: works everywhere, but adds hundreds of megabytes.

Additional context

Ownership and status

Maintainer-owned backlog. On 3 November 2026 this was rewritten from an evaluation task into the feature itself; choosing the recogniser is its first step.