# STT validation 0.4.1

Firmware **0.4.0** · repository Control Layer **0.4.1** · Protocol **0.4.0**

Validation date: **2026-10-03**

## Result

- Cloudflare authorization: **PASS**
- Workers AI inference: **PASS**
- Configured provider: **cloudflare-workers-ai**
- Model: **`@cf/openai/whisper-large-v3-turbo`**
- REAL STT READY: **YES**
- CHINESE STT READY: **YES**
- ENGLISH STT READY: **YES**
- NATURAL CODE-SWITCHING READY: **YES**
- EXACT MACHINE IDENTIFIER: **BEST EFFORT / CONFIRM BEFORE SEND**
- Production prompt enabled: **NO**
- Product acceptance: exact identifiers are confirmed before sending or selected in
  Device Settings; their transcription is not a production blocker.

This is a synthetic speech/inference smoke, not physical M5Stack microphone
verification. Physical microphone verification remains for a later device flash test.

## Baseline and credential safety

The checkout began clean on `main` at
`74535750670f2f451374b01989342da063da7d6a`; `origin/main` matched. `VERSION`
was `0.4.0` and `CONTROL_VERSION` was `0.4.1`.

The repo-external `0600 ~/.config/esp32-device-gateway/gateway.env` contained all
four required nonempty variables:

- `GATEWAY_STT_PROVIDER`
- `GATEWAY_CF_ACCOUNT_ID`
- `GATEWAY_CF_API_TOKEN`
- `GATEWAY_STT_MODEL`

Provider and model matched the values above. `GATEWAY_STT_INITIAL_PROMPT` was unset.
No Account ID, token, Authorization header, private environment content, or API error
body was printed or recorded. Cloudflare Tunnel, DNS and Access were not touched.

An isolated, non-sensitive MeloTTS inference probe returned HTTP 200 with a valid
result envelope. This proves both HTTP bearer authorization and Workers AI inference
permission with the configured token.

## Fixture generation

Cloudflare's current official schemas were checked before the smoke:

- [MeloTTS](https://developers.cloudflare.com/workers-ai/models/melotts/) takes
  JSON `prompt` and `lang`, and returns generated MP3 audio.
- [Whisper Large v3 Turbo](https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/)
  takes audio plus `task=transcribe`; both `language` and `initial_prompt` are optional.

Cloudflare MeloTTS generated Chinese, English, Chinese-English, model/effort and
technical-vocabulary fixtures. The exact non-sensitive test phrases are shown below.
The returned MP3 files were converted only under `/tmp` with the existing `ffmpeg` to
44.1 kHz, 16-bit, mono PCM WAV. A preinstalled Microsoft Edge TTS Chinese voice was
used once as an independent model/effort pronunciation cross-check. No private
recording was used and no audio fixture was added to Git.

All Whisper requests went through `scripts/smoke-stt.py` and `configured_stt()`.
`language` was unset. Initial primary requests were unprompted.

## Real Whisper smoke

| Case | Expected | Unprompted transcript | Audio | STT call | Result |
| --- | --- | --- | ---: | ---: | --- |
| Chinese | 请打开这个项目，然后运行测试 | 请打开这个项目,然后运行测试。 | 2.720 s | 2.601 s | **PASS** — 项目、测试 and full meaning preserved |
| English | Resume this session, checkout the feature branch, and run pytest | Resume this session, check out the feature branch, and run PyTest. | 3.730 s | 2.086 s | **PASS** — all terms and meaning preserved; `checkout` was split normally |
| Chinese-English | 请帮我 resume 这个 Codex session，然后 checkout feature branch，再跑一下 pytest | 请帮我resume这个codex session 然后checkoutfeaturebranch 再跑一下pytest | 6.559 s | 2.985 s | **PASS** for this command — terms and Chinese meaning preserved; English words were partly joined |
| Model/effort, MeloTTS zh | 把 model 换成 gpt-6-astra，然后 effort 调成 high | 而model换成gets6astra 然后effort调整high | 4.523 s | 3.196 s | **REVIEW** — model, effort and high preserved; model ID failed and opening Chinese changed |
| Model/effort, Edge TTS cross-check | 把 model 换成 gpt-6-astra，然后 effort 调成 high | 把Model 换成 GPT-6-Estra,然后 EFFORT 调成 HIGH | 5.304 s | 1.685 s | **REVIEW** — Chinese meaning passed, but exact model ID still failed |
| Technical vocabulary | Codex, Pi, session, model, branch, commit, pytest, Gateway, Cloudflare, ESP32, GitHub, Docker | CodexPySessionModelBranchCommitPyTestGatewayCloudFluffer, Esfetree2GithubDocker. | 6.110 s | 3.394 s | **REVIEW** — heavy token joining plus Cloudflare/ESP32 errors |

For the primary Chinese-English command:

- Technical term accuracy: **PASS with formatting caveat**. Codex, session, checkout,
  branch and pytest were all recognizable, although checkout/feature/branch were joined.
- Chinese semantic fidelity: **PASS**. The request, sequence and action survived.
- Overall usability: **PASS** for this command.

Exact identifiers remain best effort. `gpt-6-astra` must not be silently treated as
correct when the transcript says `GPT-6-Estra`. This accepted limitation does not
block natural mixed-language STT readiness.

## Technical-term observations

| Term | Result |
| --- | --- |
| Codex | PASS in the mixed command; present but joined in the vocabulary list |
| Pi | REVIEW — rendered as `Py` in the unprompted vocabulary list |
| session | PASS |
| model | PASS |
| branch | PASS in the mixed command; joined in the vocabulary list |
| commit | PASS semantically, but joined in the vocabulary list |
| pytest | PASS |
| Gateway | PASS semantically, but joined in the vocabulary list |
| Cloudflare | REVIEW — `CloudFluffer` in the unprompted list |
| ESP32 | REVIEW — `Esfetree2` in the unprompted list |
| GitHub | PASS semantically, but joined in the vocabulary list |
| Docker | PASS semantically, but joined in the vocabulary list |
| gpt-6-astra | REVIEW — `GPT-6-Estra`, `gets6astra`, or `get6Astra` |

## Initial-prompt A/B

The candidate was:

```text
Codex, Pi, session, model, branch, commit, pytest, Gateway, approval, GitHub, Docker, Cloudflare, ESP32, gpt-6-astra
```

Results on identical audio:

- Main Chinese-English command: 3/5 strict token matches both without and with prompt.
  The two apparent misses were joined words; the prompt did not fix them.
- MeloTTS model/effort phrase: 3/4 both without and with prompt. The prompt did not
  recover `gpt-6-astra`.
- Edge TTS model/effort phrase: 3/4 both without and with prompt. The prompt did not
  recover `gpt-6-astra`; latency increased from 1.857 s to 3.548 s in this A/B.
- Vocabulary-list fixture: 0/12 strict matches without prompt and 12/12 with prompt,
  but its spoken content closely mirrors the prompt itself. This isolated result risks
  prompt copying/overfitting and did not generalize to either practical mixed phrase.

Prompt A/B result: **NO GENERAL IMPROVEMENT**.
Prompt enabled in production: **NO**.

## Latency

Representative unprompted STT calls were **1.685–3.394 s**, averaging approximately
**2.55 s** across the five table rows selected for the primary/cross-check matrix.
Most repeated A/B calls were in the same range. The slowest was the prompted vocabulary
list at **5.820 s**, within the documented 5–8 second review band and not representative
of ordinary short commands. Durations include upload, network wait and inference; no
server-only timing is claimed.

## Production Gateway

The user explicitly accepted the exact-identifier limitation on 2026-10-03.
Control 0.4.1 was deployed: service **active**, `NRestarts=0`, `/health`
`control_version=0.4.1`, zero online devices. The running process environment loads
`CloudflareWorkersAISttProvider`. `GATEWAY_STT_INITIAL_PROMPT` remains unset.
No physical device commands are used for this deployment.

## Scope and integrity

No firmware source, firmware version, protocol, production audio pipeline, transcript
post-processing, Tunnel, DNS, Access or Agent behavior was changed. No device command
was sent. No firmware build, package or flash was run. `dist/m5stack-stopwatch` remained
untouched during this validation.