     1→# Milestone: Real STT + TTS Integration
     2→
     3→> **日期：** 2026-03-16
     4→> **阶段：** Phase 2 + Phase 4（合并完成）
     5→> **状态：** ✅ 单元测试全部通过，等待实机验证
     6→
     7→---
     8→
     9→## 1. 概述
    10→
    11→本次 milestone 将 openclaw-proxy 的 STT（语音识别）和 TTS（语音合成）从 mock 替换为真实实现，打通完整的语音链路：
    12→
    13→```
    14→ESP32 麦克风 → Opus 音频 → [STT: Whisper] → 文字
    15→    → [LLM: OpenClaw CLI] → 回复文字
    16→    → [TTS: edge-tts + ffmpeg] → Opus 音频 → ESP32 扬声器
    17→```
    18→
    19→**之前（Phase 0.5）：** STT 固定返回 `"测试语音输入"`，TTS 固定返回 159 字节静音帧。
    20→**现在：** STT 使用 Whisper base 模型做真实中文语音识别，TTS 使用 edge-tts 合成语音并通过 ffmpeg 编码为 Opus。
    21→
    22→---
    23→
    24→## 2. 技术选型与理由
    25→
    26→### 2.1 STT — OpenAI Whisper (base)
    27→
    28→| 项目 | 选择 |
    29→|------|------|
    30→| 引擎 | `openai-whisper` 20250625 |
    31→| 模型 | `base`（74M 参数，~140MB 磁盘） |
    32→| 运行方式 | 本地 CPU 推理（无 GPU） |
    33→| 语言 | 固定 `zh`（中文） |
    34→
    35→**为什么选 Whisper base：**
    36→- 4GB 内存 + 4 核 CPU 环境下，`base` 是精度和速度的最佳平衡点
    37→- `tiny` 太不准确，`small` 在 CPU 上太慢（>10s/句）
    38→- 本地运行，无需外部 API，无网络延迟
    39→- Whisper 内置 ffmpeg 调用，能直接处理 Opus 格式输入
    40→
    41→**不选 FunASR / SenseVoice 的原因：**
    42→- 依赖更重，安装复杂
    43→- Whisper 已满足演示需求，后续可替换
    44→
    45→### 2.2 TTS — edge-tts + ffmpeg
    46→
    47→| 项目 | 选择 |
    48→|------|------|
    49→| 合成引擎 | `edge-tts` 7.2.7（微软 Edge 在线 TTS） |
    50→| 声音 | `zh-CN-XiaoxiaoNeural`（女声，自然度高） |
    51→| 输出格式 | MP3（edge-tts 原生输出） |
    52→| 编码转换 | `ffmpeg` → Opus（16kHz, mono, 24kbps） |
    53→
    54→**为什么选 edge-tts：**
    55→- 零成本、无需 API key
    56→- 异步原生（`async for` 流式获取音频）
    57→- 中文语音质量极佳（微软 Neural TTS）
    58→- 依赖轻量，纯 Python + aiohttp
    59→
    60→**不选 CosyVoice / GPT-SoVITS 的原因：**
    61→- 需要 GPU 或大量 CPU 资源
    62→- 部署复杂度高，不适合当前 4GB 内存环境
    63→
    64→---
    65→
    66→## 3. 架构变更
    67→
    68→### 3.1 变更前后对比
    69→
    70→```
    71→变更前（Phase 0.5）：
    72→┌─────────────┐     ┌──────────────────┐     ┌──────────────┐
    73→│  ESP32      │────►│  process_stt()   │     │  OpenClaw    │
    74→│  Opus bytes │     │  return 固定文字  │────►│  CLI         │
    75→└─────────────┘     └──────────────────┘     └──────┬───────┘
    76→                    ┌──────────────────┐            │
    77→                    │  process_tts()   │◄───────────┘
    78→                    │  return 静音帧   │
    79→                    └──────────────────┘
    80→
    81→变更后（本次 Milestone）：
    82→┌─────────────┐     ┌──────────────────────────────────┐     ┌──────────────┐
    83→│  ESP32      │────►│  process_stt()                   │     │  OpenClaw    │
    84→│  Opus bytes │     │  tmpfile(.opus) → Whisper base   │────►│  CLI         │
    85→└─────────────┘     │  → asyncio.to_thread() → 文本    │     └──────┬───────┘
    86→                    └──────────────────────────────────┘            │
    87→                    ┌──────────────────────────────────┐            │
    88→                    │  process_tts()                   │◄───────────┘
    89→                    │  edge-tts → MP3 → ffmpeg → Opus  │
    90→                    └──────────────────────────────────┘
    91→```
    92→
    93→### 3.2 数据流详细说明
    94→
    95→#### STT 链路：`process_stt(audio_bytes: bytes) -> str`
    96→
    97→```
    98→1. ESP32 发送原始 Opus 音频字节
    99→2. 写入临时文件（NamedTemporaryFile, suffix=".opus"）
   100→3. Whisper load_audio() 内部调用 ffmpeg 将 Opus 解码为 16kHz float32 PCM
   101→4. Whisper base 模型推理（通过 asyncio.to_thread 在线程池执行，不阻塞事件循环）
   102→5. 提取 result["text"]，strip() 去除首尾空白
   103→6. 返回识别文本
   104→```
   105→
   106→**关键设计决策：**
   107→- **Lazy loading：** Whisper 模型不在 `import` 时加载，而是首次调用 `process_stt()` 时通过 `_get_whisper_model()` 延迟加载。这样做的好处：
   108→  - 测试不需要加载真实模型（mock `_get_whisper_model` 即可）
   109→  - 服务启动快，模型加载延迟到第一个请求
   110→  - 模型全局复用，只加载一次
   111→- **asyncio.to_thread：** Whisper 推理是 CPU 密集型操作，直接在事件循环中运行会阻塞所有 WebSocket 连接。用 `to_thread` 将其放到线程池执行。
   112→- **临时文件：** Whisper 的 `transcribe()` 接受文件路径（内部用 ffmpeg 解码），不接受 bytes。因此需要写临时文件。Linux 上 `NamedTemporaryFile(delete=True)` 在 `with` 块内文件可通过路径访问，退出后自动删除。
   113→
   114→#### TTS 链路：`process_tts(text: str) -> bytes`
   115→
   116→```
   117→1. 创建 edge_tts.Communicate(text=text, voice="zh-CN-XiaoxiaoNeural")
   118→2. async for 遍历 communicate.stream()，收集所有 type="audio" 的 chunk
   119→3. 将 MP3 chunks 拼接为完整 MP3 字节
   120→4. 启动 ffmpeg 子进程进行格式转换：
   121→   ffmpeg -i pipe:0 -c:a libopus -b:a 24k -ar 16000 -ac 1 -f opus pipe:1
   122→   - 输入：stdin 管道接收 MP3 字节
   123→   - 输出：stdout 管道输出 Opus 字节
   124→5. 检查 ffmpeg 返回码，非零则抛出 RuntimeError
   125→6. 返回 Opus 字节
   126→```
   127→
   128→**关键设计决策：**
   129→- **管道传输：** MP3 → Opus 转换通过 stdin/stdout 管道完成，无需写中间文件。
   130→- **ffmpeg 参数：** `libopus` 编码器，24kbps 码率（语音足够），16kHz 采样率，单声道。与 ESP32 硬件解码能力匹配。
   131→- **错误处理：** ffmpeg 失败时读取 stderr 日志并抛出 RuntimeError，上层可捕获处理。
   132→
   133→---
   134→
   135→## 4. 代码变更详情
   136→
   137→### 4.1 修改的文件
   138→
   139→| 文件 | 变更类型 | 说明 |
   140→|------|----------|------|
   141→| `main.py` | 修改 | 替换 mock STT/TTS 为真实实现 |
   142→| `tests/test_main.py` | 修改 | 更新测试用例匹配新实现 |
   143→| `requirements.txt` | 修改 | 新增 whisper 和 edge-tts 依赖 |
   144→
   145→### 4.2 `main.py` 变更明细
   146→
   147→#### 新增导入（第 1-14 行）
   148→
   149→```python
   150→import tempfile          # STT 临时文件
   151→import edge_tts          # TTS 引擎
   152→import whisper           # STT 引擎
   153→```
   154→
   155→#### 新增配置常量（第 29-31 行）
   156→
   157→```python
   158→WHISPER_MODEL_SIZE = "base"                    # Whisper 模型大小
   159→EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"       # TTS 声音
   160→FFMPEG_BIN = which("ffmpeg") or "ffmpeg"       # ffmpeg 可执行文件路径
   161→```
   162→
   163→#### 新增 Whisper 模型懒加载（第 66-77 行）
   164→
   165→```python
   166→_whisper_model: whisper.Whisper | None = None
   167→
   168→def _get_whisper_model() -> whisper.Whisper:
   169→    """Return the Whisper model, loading it on first call."""
   170→    global _whisper_model
   171→    if _whisper_model is None:
   172→        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
   173→        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
   174→    return _whisper_model
   175→```
   176→
   177→#### 替换 `process_stt`（第 83-102 行）
   178→
   179→**之前：** 固定返回 `"测试语音输入"`（2 行逻辑）
   180→**之后：** 完整 Whisper 推理流程（12 行逻辑）
   181→
   182→```python
   183→async def process_stt(audio_bytes: bytes) -> str:
   184→    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
   185→    model = _get_whisper_model()
   186→    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
   187→        f.write(audio_bytes)
   188→        f.flush()
   189→        result = await asyncio.to_thread(
   190→            model.transcribe, f.name, language="zh"
   191→        )
   192→    text = result["text"].strip()
   193→    logger.info("STT: transcribed → %s", text)
   194→    return text
   195→```
   196→
   197→#### 替换 `process_tts`（第 147-187 行）
   198→
   199→**之前：** 固定返回 `b"\xfc\xff\xfe" * 53` 静音帧（3 行逻辑）
   200→**之后：** edge-tts 合成 + ffmpeg Opus 编码（25 行逻辑）
   201→
   202→```python
   203→async def process_tts(text: str) -> bytes:
   204→    logger.info("TTS: synthesising → %s", text)
   205→
   206→    # 1. edge-tts → MP3
   207→    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
   208→    mp3_chunks: list[bytes] = []
   209→    async for chunk in communicate.stream():
   210→        if chunk["type"] == "audio":
   211→            mp3_chunks.append(chunk["data"])
   212→    mp3_bytes = b"".join(mp3_chunks)
   213→
   214→    # 2. MP3 → Opus via ffmpeg
   215→    proc = await asyncio.create_subprocess_exec(
   216→        FFMPEG_BIN, "-i", "pipe:0",
   217→        "-c:a", "libopus", "-b:a", "24k",
   218→        "-ar", "16000", "-ac", "1",
   219→        "-f", "opus", "pipe:1",
   220→        stdin=asyncio.subprocess.PIPE,
   221→        stdout=asyncio.subprocess.PIPE,
   222→        stderr=asyncio.subprocess.PIPE,
   223→    )
   224→    opus_bytes, stderr = await proc.communicate(input=mp3_bytes)
   225→
   226→    if proc.returncode != 0:
   227→        err_msg = stderr.decode(errors="replace").strip()
   228→        logger.error("ffmpeg failed (rc=%d): %s", proc.returncode, err_msg)
   229→        raise RuntimeError(f"ffmpeg TTS conversion error: {err_msg}")
   230→
   231→    logger.info("TTS: generated %d bytes of Opus audio", len(opus_bytes))
   232→    return opus_bytes
   233→```
   234→
   235→#### 未变更的部分
   236→
   237→以下代码**完全未修改**：
   238→- Pydantic 模型（`HardwareAction`, `HardwareEvent`, `StateNotification`, `TextReply`）
   239→- `call_openclaw()` — 仍通过 CLI 调用 OpenClaw
   240→- `handle_text_frame()` — 文本帧路由
   241→- `handle_binary_frame()` — 二进制帧管线编排（STT → LLM → TTS 时序不变）
   242→- `websocket_chat()` — WebSocket 端点
   243→- `/health` — 健康检查
   244→- 入口点 — uvicorn 启动
   245→
   246→### 4.3 `tests/test_main.py` 变更明细
   247→
   248→#### 新增导入
   249→
   250→```python
   251→from unittest.mock import MagicMock  # 用于 mock Whisper 模型（同步对象）
   252→```
   253→
   254→#### 替换的测试
   255→
   256→| 旧测试 | 新测试 | 变更说明 |
   257→|--------|--------|----------|
   258→| `test_process_stt_returns_mock_text` | `test_process_stt_calls_whisper` | mock `_get_whisper_model`，验证临时文件、语言参数、文本 strip |
   259→| `test_process_tts_returns_bytes` | `test_process_tts_returns_opus_bytes` | mock edge-tts stream + ffmpeg subprocess，验证 MP3 拼接和 Opus 输出 |
   260→
   261→#### 新增的测试
   262→
   263→| 测试 | 说明 |
   264→|------|------|
   265→| `test_process_tts_raises_on_ffmpeg_failure` | ffmpeg 返回非零退出码时应抛出 `RuntimeError` |
   266→
   267→#### 修改的测试
   268→
   269→| 测试 | 变更说明 |
   270→|------|----------|
   271→| `test_websocket_binary_frame_full_pipeline` | 从 mock subprocess 改为 mock 三个管线函数（`process_stt`, `call_openclaw`, `process_tts`），更好地隔离 WebSocket 集成测试 |
   272→
   273→#### 未变更的测试（6 个）
   274→
   275→- `test_call_openclaw_parses_cli_output`
   276→- `test_call_openclaw_raises_on_cli_failure`
   277→- `test_call_openclaw_raises_on_empty_payloads`
   278→- `test_health_endpoint`
   279→- `test_websocket_connect_and_disconnect`
   280→- `test_websocket_text_frame_wake_word`
   281→- `test_websocket_text_frame_malformed_json`
   282→
   283→### 4.4 `requirements.txt` 变更
   284→
   285→```diff
   286→ fastapi>=0.115.0
   287→ uvicorn[standard]>=0.34.0
   288→ websockets>=14.0
   289→ pydantic>=2.0
   290→+openai-whisper>=20250625
   291→+edge-tts>=7.0
   292→ pytest>=8.0
   293→ httpx>=0.28.0
   294→ pytest-asyncio>=0.25.0
   295→```
   296→
   297→---
   298→
   299→## 5. 测试结果
   300→
   301→```
   302→$ python -m pytest tests/ -v
   303→
   304→tests/test_main.py::test_process_stt_calls_whisper PASSED            [  9%]
   305→tests/test_main.py::test_call_openclaw_parses_cli_output PASSED      [ 18%]
   306→tests/test_main.py::test_call_openclaw_raises_on_cli_failure PASSED  [ 27%]
   307→tests/test_main.py::test_call_openclaw_raises_on_empty_payloads PASSED [ 36%]
   308→tests/test_main.py::test_process_tts_returns_opus_bytes PASSED       [ 45%]
   309→tests/test_main.py::test_process_tts_raises_on_ffmpeg_failure PASSED [ 54%]
   310→tests/test_main.py::test_health_endpoint PASSED                      [ 63%]
   311→tests/test_main.py::test_websocket_connect_and_disconnect PASSED     [ 72%]
   312→tests/test_main.py::test_websocket_text_frame_wake_word PASSED       [ 81%]
   313→tests/test_main.py::test_websocket_text_frame_malformed_json PASSED  [ 90%]
   314→tests/test_main.py::test_websocket_binary_frame_full_pipeline PASSED [100%]
   315→
   316→============================== 11 passed in 2.11s ==============================
   317→```
   318→
   319→**全部 11 个测试通过**（原 10 个 + 新增 1 个 ffmpeg 错误处理测试）。
   320→
   321→---
   322→
   323→## 6. 依赖清单
   324→
   325→### 运行时依赖
   326→
   327→| 包 | 版本 | 用途 | 安装大小 |
   328→|----|------|------|----------|
   329→| `openai-whisper` | 20250625 | 语音识别引擎 | ~10MB（代码） |
   330→| `torch` | 2.10.0 | Whisper 推理后端 | ~2GB（含 CUDA stubs） |
   331→| `edge-tts` | 7.2.7 | 微软 TTS 合成 | ~50KB |
   332→| `aiohttp` | 3.13+ | edge-tts 的 HTTP 传输层 | ~1MB |
   333→
   334→### 系统依赖
   335→
   336→| 工具 | 版本 | 用途 |
   337→|------|------|------|
   338→| `ffmpeg` | 7.0.2 | Opus ↔ WAV/MP3 音频转码 |
   339→| `libopus` | （含于 ffmpeg） | Opus 编解码器 |
   340→
   341→### Whisper 模型文件
   342→
   343→| 模型 | 参数量 | 磁盘 | 内存 | 首次加载 |
   344→|------|--------|------|------|----------|
   345→| `base` | 74M | ~140MB | ~300MB | 需联网下载到 `~/.cache/whisper/` |
   346→
   347→---
   348→
   349→## 7. 配置参数
   350→
   351→所有配置集中在 `main.py` 顶部 Configuration 区域：
   352→
   353→```python
   354→# ── Configuration ────────────────────────────────────────────────────
   355→OPENCLAW_CLI = which("openclaw") or "openclaw"    # OpenClaw CLI 路径
   356→OPENCLAW_AGENT_ID = "main"                         # 使用的 agent ID
   357→OPENCLAW_TIMEOUT = 120                             # CLI 超时（秒）
   358→WHISPER_MODEL_SIZE = "base"                        # Whisper 模型：tiny/base/small/medium/large
   359→EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"           # TTS 声音选择
   360→FFMPEG_BIN = which("ffmpeg") or "ffmpeg"           # ffmpeg 路径
   361→```
   362→
   363→**可调参数说明：**
   364→
   365→| 参数 | 可选值 | 影响 |
   366→|------|--------|------|
   367→| `WHISPER_MODEL_SIZE` | `tiny` / `base` / `small` / `medium` / `large` | 模型越大越准但越慢。`base` 在 4 核 CPU 上约 2-5 秒/句 |
   368→| `EDGE_TTS_VOICE` | 见 `edge-tts --list-voices` | 可选其他中文声音如 `zh-CN-YunxiNeural`（男声）|
   369→| ffmpeg `-b:a` | `16k` / `24k` / `32k` / `64k` | Opus 码率，`24k` 适合语音 |
   370→
   371→---
   372→
   373→## 8. 性能预估
   374→
   375→基于 4 核 CPU / 4GB 内存 / 无 GPU 环境：
   376→
   377→| 阶段 | 预估延迟 | 瓶颈 |
   378→|------|----------|------|
   379→| **首次 STT 调用** | 10-30 秒 | 下载 Whisper base 模型（~140MB，仅一次） |
   380→| **STT 推理** | 2-5 秒/句 | CPU 推理，取决于音频长度 |
   381→| **LLM（OpenClaw）** | 5-30 秒 | 取决于 agent 复杂度和外部 API |
   382→| **TTS 合成** | 1-3 秒 | edge-tts 网络请求 + ffmpeg 转码 |
   383→| **总延迟** | 8-38 秒 | 首次更长，后续稳定在 8-15 秒 |
   384→
   385→**内存占用估算：**
   386→- Whisper base 模型：~300MB
   387→- FastAPI + Python 运行时：~200MB
   388→- 峰值（推理中）：~500-600MB
   389→- 剩余可用：~3.4GB（充裕）
   390→
   391→---
   392→
   393→## 9. 已知限制与风险
   394→
   395→### 9.1 当前限制
   396→
   397→| 限制 | 说明 | 影响 |
   398→|------|------|------|
   399→| **无音频缓冲** | 当前每个二进制帧独立触发完整管线 | ESP32 按住说话时，每个音频片段会分别处理而非合并。需要 Phase 1 的 `AudioBuffer` 实现 |
   400→| **无分块推流** | TTS 音频一次性整包发送 | 长句子的音频可能有数十 KB，ESP32 需要一次性接收全部数据后才开始播放 |
   401→| **单并发限制** | Whisper 推理会占用 CPU，多个同时请求会排队 | 当前单用户场景下不是问题 |
   402→| **网络依赖** | edge-tts 需要访问微软服务器 | 服务器需有外网访问能力 |
   403→
   404→### 9.2 风险项
   405→
   406→| 风险 | 概率 | 缓解措施 |
   407→|------|------|----------|
   408→| Whisper 模型首次下载失败 | 中 | 可手动下载放到 `~/.cache/whisper/`，或预热 |
   409→| edge-tts 微软服务不稳定 | 低 | 可降级为本地 TTS 或重试 |
   410→| ESP32 发送的 Opus 格式不兼容 | 中 | ffmpeg 通常能处理，如不行需检查 ESP32 编码参数 |
   411→| 长音频导致 Whisper OOM | 低 | base 模型在 4GB 环境处理 <30 秒音频没问题 |
   412→
   413→---
   414→
   415→## 10. 实机测试计划
   416→
   417→### 10.1 前置准备
   418→
   419→```bash
   420→# 1. 确保服务器环境就绪
   421→source venv/bin/activate
   422→python -m pytest tests/ -v          # 所有测试通过
   423→ffmpeg -version                      # 确认 ffmpeg 可用
   424→python -c "import whisper; print(whisper.__version__)"   # 确认 whisper 可导入
   425→
   426→# 2. 预热 Whisper 模型（避免首次调用等待下载）
   427→python -c "import whisper; whisper.load_model('base')"
   428→
   429→# 3. 确认 OpenClaw gateway 运行中
   430→openclaw gateway status              # 或检查 :18789 端口
   431→
   432→# 4. 启动 proxy
   433→python main.py
   434→```
   435→
   436→### 10.2 模拟测试（无需 ESP32）
   437→
   438→```bash
   439→# 用 test_client.py 进行基本连通测试
   440→python test_client.py
   441→```
   442→
   443→预期输出：
   444→```
   445→=== Connected ===
   446→
   447→[TX] wake_word
   448→[RX] {'type': 'state', 'status': 'listening'}
   449→
   450→[TX] binary audio (80 bytes of mock Opus)
   451→[RX] {'type': 'state', 'status': 'thinking'}
   452→[RX] {'type': 'text_reply', 'text': '...（OpenClaw 回复）'}
   453→[RX] binary audio: XXXX bytes
   454→
   455→=== Done ===
   456→```
   457→
   458→> **注意：** `test_client.py` 发送的是 80 字节零填充 mock 数据，Whisper 可能识别为空或乱码。这是正常的——真实 Opus 音频才能产生有意义的识别结果。
   459→
   460→### 10.3 ESP32 实机测试检查清单
   461→
   462→- [ ] ESP32 小智固件已配置 WebSocket 地址为 `ws://<服务器IP>:8000/chat`
   463→- [ ] 服务器防火墙已开放 8000 端口
   464→- [ ] 按下按钮 → 服务器日志显示 `Hardware event: key_down`
   465→- [ ] 对着麦克风说中文 → 服务器日志显示 `STT: transcribed → <识别的文字>`
   466→- [ ] 松开按钮 → 服务器日志显示 `STT: received N bytes` 和 `LLM: sending to OpenClaw`
   467→- [ ] ESP32 扬声器播放 TTS 合成的语音回复
   468→- [ ] 多轮对话：连续提问，agent 能记住上下文（同一 session_id）
   469→
   470→### 10.4 日志关键字速查
   471→
   472→正常流程的日志序列：
   473→```
   474→STT: received XXXX bytes of Opus audio
   475→Loading Whisper 'base' model…              ← 仅首次出现
   476→STT: transcribed → 你好                    ← 识别结果
   477→LLM: sending to OpenClaw → 你好
   478→LLM: reply ← 你好！有什么我可以帮忙的？
   479→TTS: synthesising → 你好！有什么我可以帮忙的？
   480→TTS: generated XXXX bytes of Opus audio
   481→Pushing XXXX bytes of audio to client
   482→```
   483→
   484→异常情况日志：
   485→```
   486→ffmpeg failed (rc=1): ...                  ← TTS ffmpeg 转码失败
   487→OpenClaw CLI failed (rc=N): ...            ← LLM CLI 调用失败
   488→```
   489→
   490→---
   491→
   492→## 11. 文件最终状态
   493→
   494→### `main.py`（309 行）
   495→
   496→```
   497→第 1-14 行    — 导入（新增 tempfile, whisper, edge_tts）
   498→第 16-22 行   — 日志配置
   499→第 24-31 行   — 配置常量（新增 WHISPER_MODEL_SIZE, EDGE_TTS_VOICE, FFMPEG_BIN）
   500→第 34-63 行   — Pydantic 模型（未变）
   501→第 66-77 行   — Whisper 模型懒加载（新增）
   502→第 80-102 行  — process_stt()（真实 Whisper 实现）
   503→第 105-144 行 — call_openclaw()（未变）
   504→第 147-187 行 — process_tts()（真实 edge-tts + ffmpeg 实现）
   505→第 190-261 行 — 帧路由 + 管线编排（未变）
   506→第 264-309 行 — FastAPI 应用 + 入口点（未变）
   507→```
   508→
   509→### `tests/test_main.py`（212 行）
   510→
   511→```
   512→11 个测试用例：
   513→  - 1 个 STT 单元测试（重写）
   514→  - 3 个 LLM 单元测试（未变）
   515→  - 2 个 TTS 单元测试（1 个重写 + 1 个新增）
   516→  - 5 个集成测试（1 个修改 mock 策略，4 个未变）
   517→```
   518→
   519→### `requirements.txt`（9 行）
   520→
   521→```
   522→新增 2 个依赖：openai-whisper, edge-tts
   523→```
   524→
   525→---
   526→
   527→## 12. 与 PLAN.md 阶段映射
   528→
   529→| PLAN.md 阶段 | 状态 | 本次完成内容 |
   530→|-------------|------|-------------|
   531→| Phase 0 — Mock 骨架 | ✅ 已完成 | — |
   532→| Phase 0.5 — OpenClaw 对接 | ✅ 已完成 | — |
   533→| Phase 1 — 音频缓冲 | ⏳ 待做 | — |
   534→| **Phase 2 — 真实 STT** | **✅ 本次完成** | **Whisper base 模型接入** |
   535→| Phase 3 — OpenClaw 对接 | ✅ 已在 0.5 完成 | — |
   536→| **Phase 4 — 真实 TTS** | **✅ 本次完成** | **edge-tts + ffmpeg Opus 编码** |
   537→| Phase 5 — 生产加固 | ⏳ 待做 | — |
   538→
   539→**下一步：** Phase 1（音频缓冲），支持 ESP32 的 press-to-talk 交互模式——用户按住按键持续说话，松开后将完整音频合并送入管线。
   540→
   541→---
   542→
   543→## 13. 快速验证命令
   544→
   545→```bash
   546→# 进入虚拟环境
   547→source venv/bin/activate
   548→
   549→# 跑单元测试（应全部通过）
   550→python -m pytest tests/ -v
   551→
   552→# 启动服务（另一个终端）
   553→python main.py
   554→
   555→# 模拟客户端测试
   556→python test_client.py
   557→```
   558→


     1→fastapi>=0.115.0
     2→uvicorn[standard]>=0.34.0
     3→websockets>=14.0
     4→pydantic>=2.0
     5→openai-whisper>=20250625
     6→edge-tts>=7.0
     7→pytest>=8.0
     8→httpx>=0.28.0
     9→pytest-asyncio>=0.25.0
    10→


Bash

find . -name "*.py" -not -path "./venv/*" -not -path "./.pytest_cache/*" -type f

List all Python files excluding venv and cache


./tests/__init__.py
./tests/test_main.py
./main.py
./test_client.py


Now I have all the information I need. Let me compile a comprehensive report of the codebase exploration.

---

## COMPREHENSIVE CODEBASE EXPLORATION REPORT

I have thoroughly explored the openclaw-proxy codebase. Here is the complete analysis:

---

## 1. MAIN.PY - COMPLETE FLOW ANALYSIS

### Current Architecture (Lines 1-309)

**File Location:** `/root/projects/openclaw-proxy/main.py`

#### A. Imports & Configuration (Lines 1-31)

```python
Lines 1-14:  Core imports
  - asyncio, json, logging, tempfile, uuid
  - enum.Enum
  - shutil.which
  - edge_tts (TTS engine)
  - whisper (STT engine)
  - FastAPI, WebSocket, WebSocketDisconnect

Lines 16-22:  Logging setup
  - Standard format: timestamp [LEVEL] name: message

Lines 24-31:  Configuration constants
  OPENCLAW_CLI = which("openclaw") or "openclaw"
  OPENCLAW_AGENT_ID = "main"
  OPENCLAW_TIMEOUT = 120 seconds
  WHISPER_MODEL_SIZE = "base"
  EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
  FFMPEG_BIN = which("ffmpeg") or "ffmpeg"
```

#### B. Pydantic Models (Lines 34-64)

```python
class HardwareAction(str, Enum):
  - KEY_DOWN = "key_down"
  - KEY_UP = "key_up"
  - WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
  - type: str
  - action: HardwareAction
  - Used for incoming JSON frames from ESP32

class StateNotification(BaseModel):
  - type: str = "state"
  - status: str
  - Pushes LED state to hardware (listening/thinking/speaking)

class TextReply(BaseModel):
  - type: str = "text_reply"
  - text: str
  - Carries LLM response text
```

#### C. Whisper Lazy Singleton (Lines 66-78)

```python
_whisper_model: whisper.Whisper | None = None

def _get_whisper_model() -> whisper.Whisper:
  - Global singleton pattern
  - Loads model on first call only
  - Logs "Loading Whisper 'base' model…" on init
  - Returns cached model on subsequent calls
```

**Key Design:** Lazy loading avoids startup delay and allows test mocking.

#### D. Process STT (Lines 83-102)

```python
async def process_stt(audio_bytes: bytes) -> str:
  Line 92:  Log "STT: received %d bytes of Opus audio"
  Line 93:  model = _get_whisper_model()
  Lines 94-96:  Create temp file with suffix=".opus"
    - Write audio_bytes
    - Flush to disk
  Lines 97-99:  asyncio.to_thread call to Whisper transcribe
    - model.transcribe(f.name, language="zh")
    - Non-blocking thread pool execution (prevents event loop block)
  Line 100:  Strip whitespace from result["text"]
  Line 101:  Log "STT: transcribed → %s"
  Returns:  Recognized text string
```

**Current State:** Real Whisper implementation (since Phase 2 completion)
**Audio Format:** Accepts Opus bytes, Whisper internally calls ffmpeg to decode

#### E. Call OpenClaw (Lines 105-144)

```python
async def call_openclaw(text: str, session_id: str) -> str:
  Line 118:  Log "LLM: sending to OpenClaw → %s"
  Lines 120-129:  Create subprocess for CLI call:
    - Command: openclaw agent --agent main --session-id {id} -m {text} --json
    - Capture stdout/stderr via pipes
    - 120 second timeout
  Lines 130:  Await process.communicate() for completion
  Lines 132-135:  Check return code, raise RuntimeError on non-zero
  Line 137:  Parse JSON from stdout
  Lines 138-140:  Extract payloads, raise on empty list
  Line 142:  Get first payload's "text" field
  Line 143:  Log "LLM: reply ← %s"
  Returns:  Agent reply string

Per-Connection State:
  - session_id is generated per WebSocket connection (Line 274)
  - Maintains conversation context across multiple turns
```

#### F. Process TTS (Lines 147-187)

```python
async def process_tts(text: str) -> bytes:
  Line 159:  Log "TTS: synthesising → %s"

  Stage 1: edge-tts to MP3 (Lines 161-167)
    - Communicate(text=text, voice="zh-CN-XiaoxiaoNeural")
    - async for chunk in communicate.stream()
    - Collect all type="audio" chunks
    - Join into mp3_bytes

  Stage 2: ffmpeg MP3→Opus conversion (Lines 170-178)
    - ffmpeg -i pipe:0 (input from stdin)
    - -c:a libopus (Opus codec)
    - -b:a 24k (24 kbps bitrate, suitable for speech)
    - -ar 16000 (16 kHz sample rate)
    - -ac 1 (mono)
    - -f opus (output format)
    - pipe:1 (output to stdout)

  Lines 179:  opus_bytes, stderr = await proc.communicate(input=mp3_bytes)
  Lines 181-184:  Check ffmpeg return code, raise RuntimeError on failure
  Line 186:  Log "TTS: generated %d bytes of Opus audio"
  Returns:  Opus-encoded audio bytes
```

**Current State:** Real implementation with edge-tts + ffmpeg (Phase 4 complete)

#### G. Frame Router: handle_text_frame (Lines 193-216)

```python
async def handle_text_frame(ws: WebSocket, raw: str) -> None:
  Line 201:  Parse JSON payload
  Line 207:  Validate with HardwareEvent model
  Line 212:  Log action value

  Lines 214-216:  If action == WAKE_WORD:
    - Create StateNotification(status="listening")
    - Send via ws.send_text(json.dumps(...))

  KEY_DOWN, KEY_UP handling:
    - Currently: IGNORED (no explicit handling)
    - Need for Phase 1: Would trigger audio buffer start/stop
```

**Current Limitation:** KEY_DOWN/KEY_UP events are parsed but not acted upon. No audio buffering mechanism exists.

#### H. Frame Router: handle_binary_frame (Lines 219-261)

```python
async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
  Line 233:  Stage 1 — STT: text = await process_stt(data)
  Line 236:  Send thinking state notification immediately

  Lines 239-248:  Heartbeat coroutine
    - Sends {"type": "state", "status": "thinking"} every 3 seconds
    - Runs while LLM is processing
    - Breaks on exception

  Line 250:  heartbeat_task = asyncio.create_task(heartbeat())
  Lines 251-254:  Stage 2 — LLM with heartbeat
    - reply = await call_openclaw(text, session_id)
    - heartbeat_task.cancel() in finally block

  Line 256:  Send text reply: {"type": "text_reply", "text": reply}

  Lines 259-261:  Stage 3 — TTS
    - audio = await process_tts(reply)
    - Send binary audio: ws.send_bytes(audio)

  Response Sequence:
    1. {"type": "state", "status": "thinking"}
    2. {"type": "text_reply", "text": "..."}
    3. <binary Opus audio>
```

**Current Behavior:** Each binary frame independently triggers full pipeline
**Phase 1 Problem:** Binary frames arrive as stream fragments when user holds button. Currently each fragment processes separately, not buffered and combined.

#### I. WebSocket Endpoint: websocket_chat (Lines 269-294)

```python
@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
  Line 272:  await ws.accept()
  Line 273:  peer = client host:port or "unknown"
  Line 274:  session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    - Generated fresh for each connection
    - Used for all turns in that connection's conversation
  Line 275:  Log connection

  Lines 278-290:  Main message loop
    - while True:
    - await ws.receive() gets next message
    - Check if disconnect
    - Route to handler:
      - text_data → handle_text_frame(ws, text_data)
      - byte_data → handle_binary_frame(ws, byte_data, session_id)

  Lines 291-294:  Finally block
    - Handle WebSocketDisconnect
    - Log disconnection
```

**Session State Management:** Currently minimal
- session_id stored only in function scope (Line 274)
- No persistent per-connection state structure
- Per-connection data: session_id only
- Multi-turn conversation: Relies on session_id being passed to openclaw

---

## 2. PLAN.MD - PHASE 1 REQUIREMENTS (Lines 77-97)

**File Location:** `/root/projects/openclaw-proxy/PLAN.md`

### Phase 1 Target: Audio Buffering & Press-to-Talk

**Goal (Line 79):** Support ESP32's real interaction mode—user presses button to start recording, holds button while speaking, releases to trigger pipeline.

**Exact Tasks (Lines 81-89):**

```markdown
Lines 82-85:
  [ ] Implement AudioBuffer class per connection
    - key_down/wake_word → start buffering, enter "listening" state
    - subsequent binary frames → append to buffer
    - key_up → stop buffering, pass complete audio to pipeline

Lines 86:
  [ ] Refactor handle_binary_frame to buffer append logic

Lines 87:
  [ ] New handle_pipeline(ws, audio_buffer) for unified pipeline orchestration

Lines 88:
  [ ] Handle edge cases: empty audio, oversized audio, repeated key_down

Lines 89:
  [ ] Add corresponding test cases
```

**Key Data Structure Required (Lines 92-96):**

```python
class SessionState(BaseModel):
    status: Literal["idle", "listening", "processing"] = "idle"
    audio_chunks: list[bytes] = []
```

**Status Machine Implied:**

```
idle ─(key_down/wake_word)─> listening ─(key_up)─> processing ─(done)─> idle
      audio binary frames appended while in "listening"
```

---

## 3. TEST_MAIN.PY - EXISTING TEST PATTERNS (Lines 1-212)

**File Location:** `/root/projects/openclaw-proxy/tests/test_main.py`

### Unit Tests (Lines 36-124)

#### Test 1: STT Whisper Integration (Lines 36-47)

```python
@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
  Mock _get_whisper_model to return mock model
  Call process_stt(b"\x00" * 80)
  Verify:
    - Returns "你好世界" (stripped)
    - model.transcribe called once
    - Args: temp file path ending in ".opus"
    - Kwargs: {"language": "zh"}
```

**Test Pattern:** Mocks expensive Whisper model, focuses on wrapper logic

#### Tests 2-4: OpenClaw CLI (Lines 50-76)

```python
test_call_openclaw_parses_cli_output()
  - Mocks subprocess, verifies JSON parsing and reply extraction

test_call_openclaw_raises_on_cli_failure()
  - Non-zero return code → RuntimeError

test_call_openclaw_raises_on_empty_payloads()
  - Empty payloads list → RuntimeError
```

**Test Pattern:** Mock subprocess.communicate(), validate CLI integration

#### Tests 5-6: TTS Edge-TTS + ffmpeg (Lines 79-124)

```python
test_process_tts_returns_opus_bytes()
  - Mock edge-tts.Communicate.stream()
  - Mock ffmpeg subprocess
  - Verify MP3 chunks collected and passed to ffmpeg
  - Verify Opus bytes returned

test_process_tts_raises_on_ffmpeg_failure()
  - ffmpeg non-zero exit → RuntimeError
```

**Test Pattern:** Mock external services, verify composition logic

### Integration Tests (Lines 130-211)

#### HTTP Health Check (Lines 130-135)

```python
test_health_endpoint()
  - GET /health → 200 with {"status": "ok"}
```

#### WebSocket Connection (Lines 138-142)

```python
test_websocket_connect_and_disconnect()
  - Client connects to /chat
  - Cleanly disconnects
```

#### Text Frame: Wake Word (Lines 145-155)

```python
test_websocket_text_frame_wake_word()
  - Send: {"type": "hardware_event", "action": "wake_word"}
  - Expect: {"type": "state", "status": "listening"}
```

#### Text Frame: Malformed JSON (Lines 158-168)

```python
test_websocket_text_frame_malformed_json()
  - Send: "not-valid-json{{{"
  - Connection survives (logged, ignored)
  - Verify recovery by sending valid wake_word
```

#### Binary Frame: Full Pipeline (Lines 171-211)

```python
test_websocket_binary_frame_full_pipeline()
  - Mock process_stt → "测试语音输入"
  - Mock call_openclaw → "你好！我在这儿。"
  - Mock process_tts → b"\xfc\xff\xfe"

  - Send: ws.send_bytes(b"\x00" * 80)

  - Expect response sequence:
    1. {"type": "state", "status": "thinking"}
    2. {"type": "text_reply", "text": "你好！我在这儿。"}
    3. binary bytes b"\xfc\xff\xfe"
```

**Test Pattern:** Uses TestClient with WebSocket context manager, sequential frame receive

---

## 4. CURRENT FLOW ANALYSIS: WAKE_WORD vs KEY_DOWN vs BINARY

### Scenario A: Wake Word (PLAN.md Line 28)

```
ESP32 sends:  {"type": "hardware_event", "action": "wake_word"}

Flow:
  1. websocket_chat receives text message
  2. handle_text_frame parses JSON
  3. Validates with HardwareEvent model
  4. Matches action == WAKE_WORD
  5. Sends: StateNotification(status="listening")
  6. ESP32 LED breathes

Proxy state: No persistent changes (ephemeral response)
Next step: ESP32 now ready to send audio
```

### Scenario B: Key Down (PLAN.md Line 28)

```
ESP32 sends:  {"type": "hardware_event", "action": "key_down"}

Current flow:
  1. websocket_chat receives text message
  2. handle_text_frame parses JSON
  3. Validates with HardwareEvent model
  4. No explicit handler for KEY_DOWN (falls through)
  5. No response sent to ESP32

Current state: Ignored (this is the Phase 1 problem)
Expected (Phase 1): Should start audio buffer, send listening state
```

### Scenario C: Binary Audio Frame (Multiple frames during button hold)

```
ESP32 sends:  <binary Opus data, 160 bytes>
ESP32 sends:  <binary Opus data, 160 bytes>
ESP32 sends:  <binary Opus data, 160 bytes>
...
ESP32 sends:  {"type": "hardware_event", "action": "key_up"}

Current flow:
  1. Each binary frame independently triggers handle_binary_frame()
  2. Each frame → complete STT → complete LLM → complete TTS
  3. Results:
     - User says "你好" (takes 3 frames)
     - Frame 1: recognizes as noise/silence → agent responds
     - Frame 2: recognizes partial speech → agent responds again
     - Frame 3: recognizes rest of speech → agent responds yet again
     - 3 concurrent LLM calls with partial/wrong inputs!

Current state: Broken for press-to-talk (each chunk processed separately)
Expected (Phase 1): Accumulate all frames until key_up, then process once
```

### Scenario D: Key Up (Only sent after button released)

```
ESP32 sends:  {"type": "hardware_event", "action": "key_up"}

Current flow:
  1. websocket_chat receives text message
  2. handle_text_frame parses JSON
  3. Validates with HardwareEvent model
  4. No explicit handler for KEY_UP (falls through)
  5. No response sent

Current state: Ignored (no buffer flushing mechanism)
Expected (Phase 1): Trigger pipeline with complete audio buffer
```

---

## 5. SESSION STATE & PERSISTENCE PATTERNS

### Current State Management

```python
main.py, websocket_chat():
  Line 274:  session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"

  Scope: Function local variable
  Lifetime: Duration of WebSocket connection
  Purpose: Pass to call_openclaw for multi-turn conversation

  Per-connection data maintained: ONLY session_id

  No other state persisted:
    - No audio buffer
    - No connection state (listening/processing)
    - No reference to handler functions
```

### What Phase 1 Requires

```python
Need per-connection SessionState:
  - status: Literal["idle", "listening", "processing"]
  - audio_chunks: list[bytes]

Need to store per connection:
  - ws (WebSocket reference) — already have in handlers
  - session_id — already passed around
  - session_state — NEW: dict[WebSocket, SessionState] or similar

Access patterns:
  - handle_text_frame(ws, raw) needs session state lookup
  - handle_binary_frame(ws, data, session_id) needs session state lookup
  - New handle_pipeline(ws, audio_buffer) needs both
```

---

## 6. EXISTING BINARY FRAME PROCESSING: INDEPENDENT

**Key Issue for Phase 1:**

Line 219 of main.py:
```python
async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
```

Each binary frame **autonomously** runs the full 3-stage pipeline:
1. STT on this frame alone
2. LLM on this frame's recognized text
3. TTS response for this frame

**Problem:** No buffering mechanism. When ESP32 sends multiple frames while button held, each frame processes independently, creating:
- Multiple incorrect STT recognitions (incomplete speech)
- Multiple concurrent LLM calls
- Multiple TTS responses (confusing output)

**Solution (Phase 1):**
- Don't trigger pipeline in handle_binary_frame
- Instead: Append to audio buffer
- Only trigger pipeline when key_up arrives

---

## 7. EDGE CASES REQUIRING HANDLING (Phase 1, Line 88)

```
a) Empty Audio
   - User presses key, releases immediately without speaking
   - audio_buffer.audio_chunks = []
   - Need to handle gracefully (skip pipeline or send notification?)

b) Oversized Audio
   - User holds button for >5 minutes
   - audio_chunks accumulate to gigabytes
   - Need max buffer size limit (e.g., 10 MB)
   - On exceed: truncate or reject?

c) Repeated key_down
   - User presses button twice without key_up
   - Current key_down while already listening
   - Should reset buffer or append? (likely reset)

d) Key_up without key_down
   - Malformed event sequence from hardware
   - Should handle gracefully (no-op or error log)

e) Binary frame while idle
   - Audio arrives without key_down/wake_word
   - Should buffer anyway or drop? (likely buffer anyway for robustness)
```

---

## 8. TEST PATTERNS TO FOLLOW

From `tests/test_main.py`:

### Unit Test Pattern
```python
@pytest.mark.asyncio
async def test_xxx() -> None:
  with patch("main.external_function", mock_value):
    result = await function_under_test(...)
  assert result == expected
  mock_value.assert_called_once_with(expected_args)
```

### Integration Test Pattern
```python
def test_websocket_xxx() -> None:
  client = TestClient(app)
  with client.websocket_connect("/chat") as ws:
    ws.send_text(json.dumps({...}))
    resp = json.loads(ws.receive_text())
    assert resp["field"] == expected_value
```

### Binary Frame Test Pattern
```python
def test_websocket_binary_xxx() -> None:
  with patch("main.process_stt", ...):
    with client.websocket_connect("/chat") as ws:
      ws.send_bytes(b"...")
      msg1 = json.loads(ws.receive_text())
      msg2 = json.loads(ws.receive_text())
      audio = ws.receive_bytes()
```

---

## 9. KEY ABSOLUTE FILE PATHS

| File | Path | Lines | Purpose |
|------|------|-------|---------|
| Main App | `/root/projects/openclaw-proxy/main.py` | 1-309 | Frame routing, pipeline orchestration |
| Tests | `/root/projects/openclaw-proxy/tests/test_main.py` | 1-212 | Unit + integration tests |
| Plan | `/root/projects/openclaw-proxy/PLAN.md` | 1-194 | Phase requirements |
| Milestone | `/root/projects/openclaw-proxy/MILESTONE-STT-TTS.md` | 1-558 | Phase 2/4 completion details |
| Requirements | `/root/projects/openclaw-proxy/requirements.txt` | 1-10 | Dependencies |
| Client Test | `/root/projects/openclaw-proxy/test_client.py` | (simple test) | Manual testing |

---

## 10. SUMMARY TABLE: PHASE 1 REQUIREMENTS vs CURRENT STATE

| Requirement | Current State | Gap | Priority |
|-------------|---------------|-----|----------|
| AudioBuffer class | Does not exist | Create from scratch | High |
| SessionState model | Does not exist | Create Pydantic model | High |
| Per-connection state storage | Only session_id | Need dict[WebSocket, SessionState] | High |
| handle_text_frame KEY_DOWN | Ignored (no handler) | Implement buffer start | High |
| handle_text_frame KEY_UP | Ignored (no handler) | Implement buffer flush + pipeline | High |
| handle_binary_frame append logic | Independent pipeline trigger | Become: append to buffer | High |
| handle_pipeline orchestrator | Does not exist | Extract STT→LLM→TTS from handle_binary_frame | High |
| Edge case handling | None | Empty audio, oversized, duplicate key_down | Medium |
| Test coverage | 11 tests (no Phase 1) | Add state machine tests | Medium |

---

This comprehensive exploration shows that Phase 1 requires a fundamental refactoring from **independent frame processing** to **state-driven buffering**. The current architecture routes each frame autonomously, while press-to-talk requires accumulating frames until a release signal, then processing the complete audio once.

agentId: aacf1ab12738baccc (for resuming to continue this agent's work if needed)
<usage>total_tokens: 46591
tool_uses: 10
duration_ms: 114106</usage>

Prompt:
Explore the openclaw-proxy codebase thoroughly to understand what's needed for Phase 1 (Audio Buffering / press-to-talk). Specifically:

1. Read main.py completely - understand the current frame routing in handle_text_frame and handle_binary_frame, and the websocket_chat endpoint. Focus on how session state is managed.

2. Read PLAN.md - specifically Phase 1 requirements for AudioBuffer, SessionState, and the press-to-talk interaction model.

3. Read tests/test_main.py - understand existing test patterns, especially the WebSocket integration tests and how binary frames are tested.

4. Understand the current flow: what happens when a wake_word or key_down event arrives vs when binary audio arrives? How does handle_binary_frame currently process each binary frame independently?

5. Look for any existing state management patterns per WebSocket connection.

Return ALL relevant code sections, line numbers, and the exact Phase 1 requirements from PLAN.md.


Now I have a thorough understanding of the codebase. Let me launch a Plan agent to design the implementation approach.


Now let me design the implementation plan.


Agent

Design Phase 1 implementation

Prompt:
Design the implementation plan for Phase 1 (Audio Buffering / press-to-talk) of the openclaw-proxy project.

## Current Architecture

**main.py** (309 lines) has these key components:

### Pydantic Models (lines 34-64):
- `HardwareAction(str, Enum)`: KEY_DOWN, KEY_UP, WAKE_WORD
- `HardwareEvent(BaseModel)`: type + action
- `StateNotification(BaseModel)`: type="state" + status
- `TextReply(BaseModel)`: type="text_reply" + text

### Frame Routing:
- `handle_text_frame(ws, raw)` (lines 193-216): Parses JSON, validates HardwareEvent. Only handles WAKE_WORD (sends "listening" state). KEY_DOWN and KEY_UP are parsed but **ignored**.
- `handle_binary_frame(ws, data, session_id)` (lines 219-261): Each binary frame **independently** triggers the full STT→LLM→TTS pipeline. This is the core problem.

### WebSocket endpoint (lines 269-294):
```python
async def websocket_chat(ws: WebSocket) -> None:
    await ws.accept()
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    try:
        while True:
            message = await ws.receive()
            if message["type"] == "websocket.disconnect": break
            text_data = message.get("text")
            byte_data = message.get("bytes")
            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect: pass
```

No per-connection state exists beyond `session_id`.

## Phase 1 Requirements (from PLAN.md)

1. Implement `SessionState` per connection:
```python
class SessionState(BaseModel):
    status: Literal["idle", "listening", "processing"] = "idle"
    audio_chunks: list[bytes] = []
```

2. State machine:
```
idle ─(key_down/wake_word)─> listening ─(key_up)─> processing ─(done)─> idle
```

3. Binary frames in "listening" state: append to audio_chunks buffer
4. key_up: join all audio_chunks, pass complete audio to pipeline
5. Refactor handle_binary_frame to just append
6. New handle_pipeline(ws, session_state, session_id) for unified STT→LLM→TTS
7. Edge cases: empty audio, oversized audio (need max buffer), repeated key_down, key_up without key_down, binary while idle

## Design Constraints

- CLAUDE.md rules: TDD, type hints mandatory, Pydantic models, Google-style docstrings
- Keep all changes in main.py (no module split yet per PLAN.md)
- Don't modify process_stt, call_openclaw, process_tts internals
- Don't modify the WebSocket protocol or health endpoint
- Existing 11 tests must continue passing (may need mock updates)

## Questions to Address in the Plan

1. Where to store SessionState? Options:
   a. Local variable in websocket_chat (simplest, passed to handlers)
   b. Module-level dict keyed by ws or session_id

2. How to handle wake_word vs key_down? Both should start listening. PLAN.md says both trigger buffering.

3. What max buffer size? Suggest 10MB or 60 seconds of audio.

4. What to do with empty audio on key_up? Log warning and skip pipeline.

5. What to do with binary frames while in "idle" state? Options:
   a. Drop silently (strict state machine)
   b. Auto-enter listening and buffer (lenient)

   I recommend option (a) - drop with a warning log. The ESP32 firmware should send key_down first.

6. How to handle binary frames during "processing" state? Drop them - can't start new recording while pipeline is running.

7. Test strategy: What new tests are needed?

Please provide a detailed step-by-step implementation plan with code changes needed for each file.


Design the implementation plan for Phase 1 (Audio Buffering / press-to-talk) of the openclaw-proxy project.

## Current Architecture

**main.py** (309 lines) has these key components:

### Pydantic Models (lines 34-64):
- `HardwareAction(str, Enum)`: KEY_DOWN, KEY_UP, WAKE_WORD
- `HardwareEvent(BaseModel)`: type + action
- `StateNotification(BaseModel)`: type="state" + status
- `TextReply(BaseModel)`: type="text_reply" + text

### Frame Routing:
- `handle_text_frame(ws, raw)` (lines 193-216): Parses JSON, validates HardwareEvent. Only handles WAKE_WORD (sends "listening" state). KEY_DOWN and KEY_UP are parsed but **ignored**.
- `handle_binary_frame(ws, data, session_id)` (lines 219-261): Each binary frame **independently** triggers the full STT→LLM→TTS pipeline. This is the core problem.

### WebSocket endpoint (lines 269-294):
```python
async def websocket_chat(ws: WebSocket) -> None:
    await ws.accept()
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    try:
        while True:
            message = await ws.receive()
            if message["type"] == "websocket.disconnect": break
            text_data = message.get("text")
            byte_data = message.get("bytes")
            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect: pass
```

No per-connection state exists beyond `session_id`.

## Phase 1 Requirements (from PLAN.md)

1. Implement `SessionState` per connection:
```python
class SessionState(BaseModel):
    status: Literal["idle", "listening", "processing"] = "idle"
    audio_chunks: list[bytes] = []
```

2. State machine:
```
idle ─(key_down/wake_word)─> listening ─(key_up)─> processing ─(done)─> idle
```

3. Binary frames in "listening" state: append to audio_chunks buffer
4. key_up: join all audio_chunks, pass complete audio to pipeline
5. Refactor handle_binary_frame to just append
6. New handle_pipeline(ws, session_state, session_id) for unified STT→LLM→TTS
7. Edge cases: empty audio, oversized audio (need max buffer), repeated key_down, key_up without key_down, binary while idle

## Design Constraints

- CLAUDE.md rules: TDD, type hints mandatory, Pydantic models, Google-style docstrings
- Keep all changes in main.py (no module split yet per PLAN.md)
- Don't modify process_stt, call_openclaw, process_tts internals
- Don't modify the WebSocket protocol or health endpoint
- Existing 11 tests must continue passing (may need mock updates)

## Questions to Address in the Plan

1. Where to store SessionState? Options:
   a. Local variable in websocket_chat (simplest, passed to handlers)
   b. Module-level dict keyed by ws or session_id

2. How to handle wake_word vs key_down? Both should start listening. PLAN.md says both trigger buffering.

3. What max buffer size? Suggest 10MB or 60 seconds of audio.

4. What to do with empty audio on key_up? Log warning and skip pipeline.

5. What to do with binary frames while in "idle" state? Options:
   a. Drop silently (strict state machine)
   b. Auto-enter listening and buffer (lenient)

   I recommend option (a) - drop with a warning log. The ESP32 firmware should send key_down first.

6. How to handle binary frames during "processing" state? Drop them - can't start new recording while pipeline is running.

7. Test strategy: What new tests are needed?

Please provide a detailed step-by-step implementation plan with code changes needed for each file.

Prompt:
Design the implementation plan for Phase 1 (Audio Buffering / press-to-talk) of the openclaw-proxy project.

## Current Architecture

**main.py** (309 lines) has these key components:

### Pydantic Models (lines 34-64):
- `HardwareAction(str, Enum)`: KEY_DOWN, KEY_UP, WAKE_WORD
- `HardwareEvent(BaseModel)`: type + action
- `StateNotification(BaseModel)`: type="state" + status
- `TextReply(BaseModel)`: type="text_reply" + text

### Frame Routing:
- `handle_text_frame(ws, raw)` (lines 193-216): Parses JSON, validates HardwareEvent. Only handles WAKE_WORD (sends "listening" state). KEY_DOWN and KEY_UP are parsed but **ignored**.
- `handle_binary_frame(ws, data, session_id)` (lines 219-261): Each binary frame **independently** triggers the full STT→LLM→TTS pipeline. This is the core problem.

### WebSocket endpoint (lines 269-294):
```python
async def websocket_chat(ws: WebSocket) -> None:
    await ws.accept()
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    try:
        while True:
            message = await ws.receive()
            if message["type"] == "websocket.disconnect": break
            text_data = message.get("text")
            byte_data = message.get("bytes")
            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect: pass
```

No per-connection state exists beyond `session_id`.

## Phase 1 Requirements (from PLAN.md)

1. Implement `SessionState` per connection:
```python
class SessionState(BaseModel):
    status: Literal["idle", "listening", "processing"] = "idle"
    audio_chunks: list[bytes] = []
```

2. State machine:
```
idle ─(key_down/wake_word)─> listening ─(key_up)─> processing ─(done)─> idle
```

3. Binary frames in "listening" state: append to audio_chunks buffer
4. key_up: join all audio_chunks, pass complete audio to pipeline
5. Refactor handle_binary_frame to just append
6. New handle_pipeline(ws, session_state, session_id) for unified STT→LLM→TTS
7. Edge cases: empty audio, oversized audio (need max buffer), repeated key_down, key_up without key_down, binary while idle

## Design Constraints

- CLAUDE.md rules: TDD, type hints mandatory, Pydantic models, Google-style docstrings
- Keep all changes in main.py (no module split yet per PLAN.md)
- Don't modify process_stt, call_openclaw, process_tts internals
- Don't modify the WebSocket protocol or health endpoint
- Existing 11 tests must continue passing (may need mock updates)

## Questions to Address in the Plan

1. Where to store SessionState? Options:
   a. Local variable in websocket_chat (simplest, passed to handlers)
   b. Module-level dict keyed by ws or session_id

2. How to handle wake_word vs key_down? Both should start listening. PLAN.md says both trigger buffering.

3. What max buffer size? Suggest 10MB or 60 seconds of audio.

4. What to do with empty audio on key_up? Log warning and skip pipeline.

5. What to do with binary frames while in "idle" state? Options:
   a. Drop silently (strict state machine)
   b. Auto-enter listening and buffer (lenient)

   I recommend option (a) - drop with a warning log. The ESP32 firmware should send key_down first.

6. How to handle binary frames during "processing" state? Drop them - can't start new recording while pipeline is running.

7. Test strategy: What new tests are needed?

Please provide a detailed step-by-step implementation plan with code changes needed for each file.


Bash

List project root contents


ls -la /root/projects/openclaw-proxy/tests/ 2>/dev/null || echo "No tests directory"

List test directory contents


total 12
drwxr-xr-x 3 root root   64 Mar 16 00:13 .
drwxr-xr-x 7 root root 4096 Mar 16 00:28 ..
-rw-r--r-- 1 root root    0 Mar 15 21:40 __init__.py
drwxr-xr-x 2 root root   84 Mar 16 00:14 __pycache__
-rw-r--r-- 1 root root 7463 Mar 16 00:13 test_main.py


total 60
drwxr-xr-x 7 root root   4096 Mar 16 00:28 .
drwxr-xr-x 6  501 games    82 Mar 15 21:31 ..
drwxr-xr-x 2 root root     33 Mar 16 00:23 .claude
-rw-r--r-- 1 root root   1392 Mar 15 21:33 CLAUDE.md
-rw-r--r-- 1 root root   9972 Mar 16 00:14 main.py
-rw-r--r-- 1 root root  20939 Mar 16 00:27 MILESTONE-STT-TTS.md
-rw-r--r-- 1 root root   7663 Mar 16 00:28 PLAN.md
drwxr-xr-x 2 root root     34 Mar 16 00:14 __pycache__
drwxr-xr-x 3 root root     70 Mar 15 21:43 .pytest_cache
-rw-r--r-- 1 root root    162 Mar 16 00:14 requirements.txt
-rw-r--r-- 1 root root   1077 Mar 15 23:53 test_client.py
drwxr-xr-x 3 root root     64 Mar 16 00:13 tests
drwxr-xr-x 6 root root     87 Mar 16 00:19 venv


     1→"""Tests for openclaw-proxy WebSocket middleware."""
     2→
     3→import json
     4→from unittest.mock import AsyncMock, MagicMock, patch
     5→
     6→import pytest
     7→from starlette.testclient import TestClient
     8→
     9→from main import app, process_stt, call_openclaw, process_tts
    10→
    11→
    12→# ── Helpers ──────────────────────────────────────────────────────────
    13→
    14→MOCK_OPENCLAW_OUTPUT = json.dumps({
    15→    "status": "ok",
    16→    "result": {
    17→        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    18→    },
    19→}).encode()
    20→
    21→
    22→def _make_mock_process(
    23→    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    24→    returncode: int = 0,
    25→) -> AsyncMock:
    26→    """Build a mock asyncio.Process with preset stdout/stderr."""
    27→    proc = AsyncMock()
    28→    proc.communicate.return_value = (stdout, b"")
    29→    proc.returncode = returncode
    30→    return proc
    31→
    32→
    33→# ── Unit tests: pipeline functions ───────────────────────────────────
    34→
    35→
    36→@pytest.mark.asyncio
    37→async def test_process_stt_calls_whisper() -> None:
    38→    """STT should write audio to temp file, call Whisper, and return stripped text."""
    39→    mock_model = MagicMock()
    40→    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    41→    with patch("main._get_whisper_model", return_value=mock_model):
    42→        result = await process_stt(b"\x00" * 80)
    43→    assert result == "你好世界"
    44→    mock_model.transcribe.assert_called_once()
    45→    args, kwargs = mock_model.transcribe.call_args
    46→    assert args[0].endswith(".opus")
    47→    assert kwargs == {"language": "zh"}
    48→
    49→
    50→@pytest.mark.asyncio
    51→async def test_call_openclaw_parses_cli_output() -> None:
    52→    """call_openclaw should parse the CLI JSON and return reply text."""
    53→    mock_proc = _make_mock_process()
    54→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    55→        result = await call_openclaw("你好", "test-session")
    56→    assert result == "你好！我在这儿。"
    57→
    58→
    59→@pytest.mark.asyncio
    60→async def test_call_openclaw_raises_on_cli_failure() -> None:
    61→    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    62→    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    63→    mock_proc.communicate.return_value = (b"", b"some error")
    64→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    65→        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
    66→            await call_openclaw("你好", "test-session")
    67→
    68→
    69→@pytest.mark.asyncio
    70→async def test_call_openclaw_raises_on_empty_payloads() -> None:
    71→    """call_openclaw should raise RuntimeError on empty payloads."""
    72→    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    73→    mock_proc = _make_mock_process(stdout=empty)
    74→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    75→        with pytest.raises(RuntimeError, match="empty payloads"):
    76→            await call_openclaw("你好", "test-session")
    77→
    78→
    79→@pytest.mark.asyncio
    80→async def test_process_tts_returns_opus_bytes() -> None:
    81→    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""
    82→
    83→    async def mock_stream() -> None:  # type: ignore[override]
    84→        yield {"type": "audio", "data": b"mp3-part1"}
    85→        yield {"type": "audio", "data": b"mp3-part2"}
    86→        yield {"type": "WordBoundary", "data": None}  # non-audio chunk
    87→
    88→    mock_comm = MagicMock()
    89→    mock_comm.stream = mock_stream
    90→
    91→    mock_proc = AsyncMock()
    92→    mock_proc.communicate.return_value = (b"fake-opus", b"")
    93→    mock_proc.returncode = 0
    94→
    95→    with (
    96→        patch("main.edge_tts.Communicate", return_value=mock_comm),
    97→        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    98→    ):
    99→        result = await process_tts("测试文本")
   100→
   101→    assert result == b"fake-opus"
   102→    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")
   103→
   104→
   105→@pytest.mark.asyncio
   106→async def test_process_tts_raises_on_ffmpeg_failure() -> None:
   107→    """TTS should raise RuntimeError when ffmpeg exits non-zero."""
   108→
   109→    async def mock_stream() -> None:  # type: ignore[override]
   110→        yield {"type": "audio", "data": b"mp3-data"}
   111→
   112→    mock_comm = MagicMock()
   113→    mock_comm.stream = mock_stream
   114→
   115→    mock_proc = AsyncMock()
   116→    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
   117→    mock_proc.returncode = 1
   118→
   119→    with (
   120→        patch("main.edge_tts.Communicate", return_value=mock_comm),
   121→        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
   122→    ):
   123→        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
   124→            await process_tts("测试文本")
   125→
   126→
   127→# ── Integration tests: HTTP & WebSocket ──────────────────────────────
   128→
   129→
   130→def test_health_endpoint() -> None:
   131→    """GET /health should return 200 with ok status."""
   132→    client = TestClient(app)
   133→    resp = client.get("/health")
   134→    assert resp.status_code == 200
   135→    assert resp.json() == {"status": "ok"}
   136→
   137→
   138→def test_websocket_connect_and_disconnect() -> None:
   139→    """Client should be able to connect to /chat and cleanly disconnect."""
   140→    client = TestClient(app)
   141→    with client.websocket_connect("/chat"):
   142→        pass
   143→
   144→
   145→def test_websocket_text_frame_wake_word() -> None:
   146→    """Sending a wake_word JSON should get back a 'listening' state."""
   147→    client = TestClient(app)
   148→    with client.websocket_connect("/chat") as ws:
   149→        ws.send_text(json.dumps({
   150→            "type": "hardware_event",
   151→            "action": "wake_word",
   152→        }))
   153→        resp = json.loads(ws.receive_text())
   154→        assert resp["type"] == "state"
   155→        assert resp["status"] == "listening"
   156→
   157→
   158→def test_websocket_text_frame_malformed_json() -> None:
   159→    """Malformed JSON should not crash the connection."""
   160→    client = TestClient(app)
   161→    with client.websocket_connect("/chat") as ws:
   162→        ws.send_text("not-valid-json{{{")
   163→        ws.send_text(json.dumps({
   164→            "type": "hardware_event",
   165→            "action": "wake_word",
   166→        }))
   167→        resp = json.loads(ws.receive_text())
   168→        assert resp["status"] == "listening"
   169→
   170→
   171→def test_websocket_binary_frame_full_pipeline() -> None:
   172→    """Binary audio should trigger STT → LLM → TTS pipeline.
   173→
   174→    Expected response sequence:
   175→      1. {"type": "state", "status": "thinking"}
   176→      2. {"type": "text_reply", "text": "..."}
   177→      3. binary bytes (TTS audio)
   178→    """
   179→    client = TestClient(app)
   180→    with (
   181→        patch(
   182→            "main.process_stt",
   183→            new_callable=AsyncMock,
   184→            return_value="测试语音输入",
   185→        ),
   186→        patch(
   187→            "main.call_openclaw",
   188→            new_callable=AsyncMock,
   189→            return_value="你好！我在这儿。",
   190→        ),
   191→        patch(
   192→            "main.process_tts",
   193→            new_callable=AsyncMock,
   194→            return_value=b"\xfc\xff\xfe",
   195→        ),
   196→    ):
   197→        with client.websocket_connect("/chat") as ws:
   198→            ws.send_bytes(b"\x00" * 80)
   199→
   200→            # 1) thinking state
   201→            msg1 = json.loads(ws.receive_text())
   202→            assert msg1 == {"type": "state", "status": "thinking"}
   203→
   204→            # 2) text reply
   205→            msg2 = json.loads(ws.receive_text())
   206→            assert msg2["type"] == "text_reply"
   207→            assert msg2["text"] == "你好！我在这儿。"
   208→
   209→            # 3) TTS audio bytes
   210→            audio = ws.receive_bytes()
   211→            assert audio == b"\xfc\xff\xfe"
   212→


     1→"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""
     2→
     3→import asyncio
     4→import json
     5→import logging
     6→import tempfile
     7→import uuid
     8→from enum import Enum
     9→from shutil import which
    10→
    11→import edge_tts
    12→import whisper
    13→from fastapi import FastAPI, WebSocket, WebSocketDisconnect
    14→from pydantic import BaseModel, ValidationError
    15→
    16→# ── Logging ──────────────────────────────────────────────────────────
    17→
    18→logging.basicConfig(
    19→    level=logging.INFO,
    20→    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    21→)
    22→logger = logging.getLogger("openclaw-proxy")
    23→
    24→# ── Configuration ────────────────────────────────────────────────────
    25→
    26→OPENCLAW_CLI = which("openclaw") or "openclaw"
    27→OPENCLAW_AGENT_ID = "main"
    28→OPENCLAW_TIMEOUT = 120  # seconds
    29→WHISPER_MODEL_SIZE = "base"
    30→EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
    31→FFMPEG_BIN = which("ffmpeg") or "ffmpeg"
    32→
    33→
    34→# ── Pydantic models ─────────────────────────────────────────────────
    35→
    36→
    37→class HardwareAction(str, Enum):
    38→    """Actions reported by XiaoZhi ESP32 hardware."""
    39→
    40→    KEY_DOWN = "key_down"
    41→    KEY_UP = "key_up"
    42→    WAKE_WORD = "wake_word"
    43→
    44→
    45→class HardwareEvent(BaseModel):
    46→    """Incoming JSON frame from hardware."""
    47→
    48→    type: str
    49→    action: HardwareAction
    50→
    51→
    52→class StateNotification(BaseModel):
    53→    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""
    54→
    55→    type: str = "state"
    56→    status: str
    57→
    58→
    59→class TextReply(BaseModel):
    60→    """Outgoing text reply carrying the LLM response."""
    61→
    62→    type: str = "text_reply"
    63→    text: str
    64→
    65→
    66→# ── Whisper model (lazy singleton) ─────────────────────────────────
    67→
    68→_whisper_model: whisper.Whisper | None = None
    69→
    70→
    71→def _get_whisper_model() -> whisper.Whisper:
    72→    """Return the Whisper model, loading it on first call."""
    73→    global _whisper_model
    74→    if _whisper_model is None:
    75→        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
    76→        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
    77→    return _whisper_model
    78→
    79→
    80→# ── STT / TTS pipeline ────────────────────────────────────────────
    81→
    82→
    83→async def process_stt(audio_bytes: bytes) -> str:
    84→    """Convert Opus audio to text via Whisper.
    85→
    86→    Args:
    87→        audio_bytes: Raw Opus-encoded audio from ESP32.
    88→
    89→    Returns:
    90→        Transcribed text string.
    91→    """
    92→    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    93→    model = _get_whisper_model()
    94→    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
    95→        f.write(audio_bytes)
    96→        f.flush()
    97→        result = await asyncio.to_thread(
    98→            model.transcribe, f.name, language="zh"
    99→        )
   100→    text = result["text"].strip()
   101→    logger.info("STT: transcribed → %s", text)
   102→    return text
   103→
   104→
   105→async def call_openclaw(text: str, session_id: str) -> str:
   106→    """Send text to OpenClaw agent via CLI and return its reply.
   107→
   108→    Args:
   109→        text: User utterance transcribed by STT.
   110→        session_id: Per-connection session ID for multi-turn conversation.
   111→
   112→    Returns:
   113→        Agent reply string.
   114→
   115→    Raises:
   116→        RuntimeError: If the CLI call fails or returns unexpected output.
   117→    """
   118→    logger.info("LLM: sending to OpenClaw → %s", text)
   119→
   120→    proc = await asyncio.create_subprocess_exec(
   121→        OPENCLAW_CLI, "agent",
   122→        "--agent", OPENCLAW_AGENT_ID,
   123→        "--session-id", session_id,
   124→        "-m", text,
   125→        "--json",
   126→        "--timeout", str(OPENCLAW_TIMEOUT),
   127→        stdout=asyncio.subprocess.PIPE,
   128→        stderr=asyncio.subprocess.PIPE,
   129→    )
   130→    stdout, stderr = await proc.communicate()
   131→
   132→    if proc.returncode != 0:
   133→        err_msg = stderr.decode(errors="replace").strip()
   134→        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
   135→        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")
   136→
   137→    data = json.loads(stdout.decode())
   138→    payloads = data.get("result", {}).get("payloads", [])
   139→    if not payloads:
   140→        raise RuntimeError("OpenClaw returned empty payloads")
   141→
   142→    reply = payloads[0].get("text", "")
   143→    logger.info("LLM: reply ← %s", reply)
   144→    return reply
   145→
   146→
   147→async def process_tts(text: str) -> bytes:
   148→    """Convert text to Opus audio via edge-tts and ffmpeg.
   149→
   150→    Args:
   151→        text: Text to synthesize.
   152→
   153→    Returns:
   154→        Opus-encoded audio bytes.
   155→
   156→    Raises:
   157→        RuntimeError: If ffmpeg conversion fails.
   158→    """
   159→    logger.info("TTS: synthesising → %s", text)
   160→
   161→    # 1. edge-tts → MP3
   162→    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
   163→    mp3_chunks: list[bytes] = []
   164→    async for chunk in communicate.stream():
   165→        if chunk["type"] == "audio":
   166→            mp3_chunks.append(chunk["data"])
   167→    mp3_bytes = b"".join(mp3_chunks)
   168→
   169→    # 2. MP3 → Opus via ffmpeg
   170→    proc = await asyncio.create_subprocess_exec(
   171→        FFMPEG_BIN, "-i", "pipe:0",
   172→        "-c:a", "libopus", "-b:a", "24k",
   173→        "-ar", "16000", "-ac", "1",
   174→        "-f", "opus", "pipe:1",
   175→        stdin=asyncio.subprocess.PIPE,
   176→        stdout=asyncio.subprocess.PIPE,
   177→        stderr=asyncio.subprocess.PIPE,
   178→    )
   179→    opus_bytes, stderr = await proc.communicate(input=mp3_bytes)
   180→
   181→    if proc.returncode != 0:
   182→        err_msg = stderr.decode(errors="replace").strip()
   183→        logger.error("ffmpeg failed (rc=%d): %s", proc.returncode, err_msg)
   184→        raise RuntimeError(f"ffmpeg TTS conversion error: {err_msg}")
   185→
   186→    logger.info("TTS: generated %d bytes of Opus audio", len(opus_bytes))
   187→    return opus_bytes
   188→
   189→
   190→# ── Frame router ────────────────────────────────────────────────────
   191→
   192→
   193→async def handle_text_frame(ws: WebSocket, raw: str) -> None:
   194→    """Parse and route an incoming JSON text frame.
   195→
   196→    Args:
   197→        ws: Active WebSocket connection.
   198→        raw: Raw text payload received from client.
   199→    """
   200→    try:
   201→        payload = json.loads(raw)
   202→    except json.JSONDecodeError:
   203→        logger.warning("Malformed JSON (ignored): %.120s", raw)
   204→        return
   205→
   206→    try:
   207→        event = HardwareEvent(**payload)
   208→    except ValidationError as exc:
   209→        logger.warning("Invalid hardware event: %s", exc.errors())
   210→        return
   211→
   212→    logger.info("Hardware event: %s", event.action.value)
   213→
   214→    if event.action == HardwareAction.WAKE_WORD:
   215→        ack = StateNotification(status="listening")
   216→        await ws.send_text(ack.model_dump_json())
   217→
   218→
   219→async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
   220→    """Run the full STT → LLM → TTS pipeline on an audio binary frame.
   221→
   222→    Args:
   223→        ws: Active WebSocket connection.
   224→        data: Raw Opus audio bytes.
   225→        session_id: Per-connection session ID.
   226→
   227→    Response sequence pushed to client:
   228→        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
   229→        2. {"type": "text_reply", "text": "..."}     — agent answer
   230→        3. <binary>                                   — Opus audio of the answer
   231→    """
   232→    # Stage 1 — STT
   233→    text = await process_stt(data)
   234→
   235→    # Notify hardware: entering thinking phase
   236→    await ws.send_text(StateNotification(status="thinking").model_dump_json())
   237→
   238→    # Stage 2 — LLM (with periodic thinking heartbeats)
   239→    async def heartbeat() -> None:
   240→        """Send thinking heartbeat every 3 seconds while LLM is working."""
   241→        while True:
   242→            await asyncio.sleep(3)
   243→            try:
   244→                await ws.send_text(
   245→                    StateNotification(status="thinking").model_dump_json()
   246→                )
   247→            except Exception:
   248→                break
   249→
   250→    heartbeat_task = asyncio.create_task(heartbeat())
   251→    try:
   252→        reply = await call_openclaw(text, session_id)
   253→    finally:
   254→        heartbeat_task.cancel()
   255→
   256→    await ws.send_text(TextReply(text=reply).model_dump_json())
   257→
   258→    # Stage 3 — TTS
   259→    audio = await process_tts(reply)
   260→    logger.info("Pushing %d bytes of audio to client", len(audio))
   261→    await ws.send_bytes(audio)
   262→
   263→
   264→# ── FastAPI application ─────────────────────────────────────────────
   265→
   266→app = FastAPI(title="openclaw-proxy", version="0.1.0")
   267→
   268→
   269→@app.websocket("/chat")
   270→async def websocket_chat(ws: WebSocket) -> None:
   271→    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
   272→    await ws.accept()
   273→    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
   274→    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
   275→    logger.info("Client connected: %s (session=%s)", peer, session_id)
   276→
   277→    try:
   278→        while True:
   279→            message = await ws.receive()
   280→
   281→            if message["type"] == "websocket.disconnect":
   282→                break
   283→
   284→            text_data = message.get("text")
   285→            byte_data = message.get("bytes")
   286→
   287→            if text_data is not None:
   288→                await handle_text_frame(ws, text_data)
   289→            elif byte_data is not None:
   290→                await handle_binary_frame(ws, byte_data, session_id)
   291→    except WebSocketDisconnect:
   292→        pass
   293→    finally:
   294→        logger.info("Client disconnected: %s (session=%s)", peer, session_id)
   295→
   296→
   297→@app.get("/health")
   298→async def health() -> dict[str, str]:
   299→    """Liveness probe."""
   300→    return {"status": "ok"}
   301→
   302→
   303→# ── Entry point ─────────────────────────────────────────────────────
   304→
   305→if __name__ == "__main__":
   306→    import uvicorn
   307→
   308→    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)
   309→


     1→# PLAN.md — openclaw-proxy
     2→
     3→## 项目定位
     4→
     5→轻量级异步 WebSocket 网关，部署在 Linux 云服务器上，负责在 **小智 ESP32 语音硬件** 与 **OpenClaw 智能体** (`/root/.openclaw/`) 之间做协议翻译和管线编排。
     6→
     7→```
     8→┌──────────────┐  Opus/JSON   ┌──────────────────┐   CLI    ┌──────────────┐
     9→│  XiaoZhi     │◄────────────►│  openclaw-proxy  │────────►│  OpenClaw    │
    10→│  ESP32       │  WebSocket   │  (FastAPI:8000)   │  stdout  │  Gateway     │
    11→└──────────────┘              └──────────────────┘◄────────│  (:18789)    │
    12→                                  │        ▲                └──────────────┘
    13→                                  ▼        │
    14→                              ┌──────┐ ┌──────┐
    15→                              │ STT  │ │ TTS  │
    16→                              └──────┘ └──────┘
    17→```
    18→
    19→**对接方式决策（2026-03-15）：** 采用方案 B — 通过 `openclaw agent` CLI 调用 gateway，
    20→不修改 `.openclaw/` 目录，不注册 channel 插件。优先跑通演示。
    21→
    22→## 协议约定
    23→
    24→### 上行（ESP32 → Proxy）
    25→
    26→| 帧类型 | 格式 | 说明 |
    27→|--------|------|------|
    28→| 文本帧 | `{"type": "hardware_event", "action": "wake_word\|key_down\|key_up"}` | 硬件状态事件 |
    29→| 二进制帧 | Raw Opus bytes | 用户语音音频流 |
    30→
    31→### 下行（Proxy → ESP32）
    32→
    33→| 帧类型 | 格式 | 说明 |
    34→|--------|------|------|
    35→| 文本帧 | `{"type": "state", "status": "listening\|thinking\|speaking"}` | 控制硬件 LED 状态 |
    36→| 文本帧 | `{"type": "text_reply", "text": "..."}` | LLM 文本回复 |
    37→| 二进制帧 | Raw Opus bytes | TTS 合成音频流 |
    38→
    39→## 分阶段实施计划
    40→
    41→---
    42→
    43→### Phase 0 — Mock 骨架搭建 ✅ 已完成
    44→
    45→**目标：** 跑通 WebSocket 握手和完整数据通路，用假数据验证时序正确。
    46→
    47→**交付物：**
    48→- [x] `main.py` — FastAPI + WS `/chat` 端点，帧路由，三段 mock 管线
    49→- [x] `tests/test_main.py` — 8 个测试全部通过（3 单元 + 5 集成）
    50→- [x] `requirements.txt` — 依赖声明
    51→
    52→---
    53→
    54→### Phase 0.5 — OpenClaw 真实对接 ✅ 已完成
    55→
    56→**目标：** 将 `call_openclaw` 从 mock 替换为真实 CLI 调用，跑通文本对话。
    57→
    58→**决策：** 采用 `openclaw agent --agent main --session-id {id} -m {text} --json` 命令。
    59→Gateway 运行在 `localhost:18789`，CLI 自动连接。
    60→
    61→**交付物：**
    62→- [x] `call_openclaw` 通过 `asyncio.create_subprocess_exec` 调用 CLI
    63→- [x] 每个 WebSocket 连接分配独立 `session_id`，支持多轮对话
    64→- [x] LLM 推理期间每 3 秒发送 thinking 心跳，保持硬件活跃
    65→- [x] CLI 失败 / 空 payload 的错误处理
    66→- [x] `tests/test_main.py` — 10 个测试全部通过（mock subprocess）
    67→
    68→**管线流程：**
    69→```
    70→Binary frame → process_stt(mock) → send "thinking" state
    71→            → call_openclaw(CLI) → heartbeat every 3s → send text_reply
    72→            → process_tts(mock) → send binary audio
    73→```
    74→
    75→---
    76→
    77→### Phase 1 — 音频缓冲与真实 ESP32 握手
    78→
    79→**目标：** 支持 ESP32 的真实交互模式——用户按住按键持续说话，松开后触发管线。
    80→
    81→**任务：**
    82→- [ ] 实现 `AudioBuffer` 类，按连接维护音频缓冲区
    83→  - `key_down` / `wake_word` → 开始缓冲，进入 "listening" 状态
    84→  - 后续二进制帧 → 追加到缓冲区
    85→  - `key_up` → 停止缓冲，将完整音频交给管线
    86→- [ ] 重构 `handle_binary_frame` 为缓冲追加逻辑
    87→- [ ] 新增 `handle_pipeline(ws, audio_buffer)` 统一编排三段管线
    88→- [ ] 处理边界情况：空音频、超长音频、重复 key_down
    89→- [ ] 补充对应测试用例
    90→
    91→**关键数据结构：**
    92→```python
    93→class SessionState(BaseModel):
    94→    status: Literal["idle", "listening", "processing"] = "idle"
    95→    audio_chunks: list[bytes] = []
    96→```
    97→
    98→---
    99→
   100→### Phase 2 — 真实 STT 接入 ✅ 已完成（2026-03-16）
   101→
   102→**目标：** 将 `process_stt` 从 mock 替换为真实语音识别。
   103→
   104→**决策：** 选用 OpenAI Whisper base 模型，本地 CPU 推理。
   105→
   106→**交付物：**
   107→- [x] Whisper base 模型懒加载（`_get_whisper_model` 单例）
   108→- [x] Opus → tmpfile → Whisper transcribe（`asyncio.to_thread` 非阻塞）
   109→- [x] 固定 `language="zh"` 中文识别
   110→- [x] 测试：mock Whisper 模型验证临时文件创建、语言参数、文本 strip
   111→
   112→---
   113→
   114→### Phase 3 — ~~OpenClaw 智能体对接~~ ✅ 已在 Phase 0.5 完成
   115→
   116→已通过 `openclaw agent` CLI 完成对接，无需额外工作。
   117→后续可考虑升级为 channel 插件获得更紧密集成。
   118→
   119→---
   120→
   121→### Phase 4 — 真实 TTS 接入 ✅ 已完成（2026-03-16）
   122→
   123→**目标：** 将 `process_tts` 从 mock 替换为真实语音合成。
   124→
   125→**决策：** 选用 edge-tts（微软 Neural TTS），MP3 → ffmpeg → Opus 编码。
   126→
   127→**交付物：**
   128→- [x] edge-tts `Communicate.stream()` 异步收集 MP3 音频
   129→- [x] ffmpeg 管道转码 MP3 → Opus（libopus, 24kbps, 16kHz, mono）
   130→- [x] ffmpeg 失败时抛出 RuntimeError
   131→- [x] 测试：mock edge-tts stream + ffmpeg subprocess，验证 MP3 拼接和错误处理
   132→- [x] 11 个测试全部通过
   133→
   134→> **详细记录见：** `MILESTONE-STT-TTS.md`
   135→
   136→**遗留（留给 Phase 5）：**
   137→- [ ] 实现分块推流：将长音频切成 ≤960 字节 Opus 帧逐帧下发
   138→- [ ] 推送 `{"type": "state", "status": "speaking"}` 状态通知
   139→
   140→---
   141→
   142→### Phase 5 — 生产加固
   143→
   144→**目标：** 具备长期稳定运行的能力。
   145→
   146→**任务：**
   147→- [ ] 连接管理：最大并发数限制、心跳超时断开、异常重连
   148→- [ ] 配置外置：`config.yaml` 或环境变量管理端口、模型选择、API 密钥等
   149→- [ ] 结构化日志：JSON 格式日志，接入日志收集
   150→- [ ] 指标监控：`/metrics` 端点暴露连接数、管线延迟、错误率
   151→- [ ] systemd 服务单元文件，支持开机自启和自动重启
   152→- [ ] HTTPS/WSS 支持（通过 nginx 反代或 uvicorn SSL 参数）
   153→- [ ] HTTP 管理接口：`GET /sessions` 查看在线连接、`POST /broadcast` 广播消息
   154→
   155→---
   156→
   157→## 技术栈总览
   158→
   159→| 层 | 当前 | 目标 |
   160→|----|------|------|
   161→| 框架 | FastAPI + uvicorn | 不变 |
   162→| WS 协议 | Starlette WebSocket | 不变 |
   163→| 数据校验 | Pydantic v2 | 不变 |
   164→| STT | Whisper base (本地 CPU) | 不变（可选升级 small/medium） |
   165→| LLM | `openclaw agent` CLI → Gateway :18789 | 同左（可选升级为 channel 插件） |
   166→| TTS | edge-tts → ffmpeg → Opus | 不变（可选 CosyVoice） |
   167→| 音频编解码 | ffmpeg (libopus) | 不变 |
   168→| 部署 | `python main.py` | systemd + nginx |
   169→
   170→## 文件结构规划
   171→
   172→```
   173→openclaw-proxy/
   174→├── main.py              # FastAPI 应用入口、WS 端点、帧路由
   175→├── pipeline/
   176→│   ├── __init__.py
   177→│   ├── stt.py           # process_stt 实现
   178→│   ├── llm.py           # call_openclaw 实现
   179→│   └── tts.py           # process_tts 实现
   180→├── models.py            # Pydantic 模型集中定义
   181→├── session.py           # 连接会话状态管理、音频缓冲
   182→├── config.py            # 配置加载（环境变量 / yaml）
   183→├── tests/
   184→│   ├── test_main.py
   185→│   ├── test_stt.py
   186→│   ├── test_llm.py
   187→│   └── test_tts.py
   188→├── requirements.txt
   189→├── CLAUDE.md
   190→└── PLAN.md
   191→```
   192→
   193→> 注：Phase 0 阶段全部逻辑集中在 `main.py`，从 Phase 1 开始按上述结构拆分模块。
   194→


     1→"""Quick WebSocket client to test the full proxy pipeline."""
     2→
     3→import asyncio
     4→import json
     5→import websockets
     6→
     7→
     8→async def main() -> None:
     9→    uri = "ws://localhost:8000/chat"
    10→    async with websockets.connect(uri) as ws:
    11→        print("=== Connected ===\n")
    12→
    13→        # 1) Test text frame: wake_word
    14→        print("[TX] wake_word")
    15→        await ws.send(json.dumps({"type": "hardware_event", "action": "wake_word"}))
    16→        resp = json.loads(await ws.recv())
    17→        print(f"[RX] {resp}\n")
    18→
    19→        # 2) Test binary frame: triggers STT → LLM → TTS pipeline
    20→        print("[TX] binary audio (80 bytes of mock Opus)")
    21→        await ws.send(b"\x00" * 80)
    22→
    23→        # Collect all responses until we get binary audio back
    24→        while True:
    25→            msg = await ws.recv()
    26→            if isinstance(msg, bytes):
    27→                print(f"[RX] binary audio: {len(msg)} bytes")
    28→                break
    29→            else:
    30→                data = json.loads(msg)
    31→                print(f"[RX] {data}")
    32→
    33→        print("\n=== Done ===")
    34→
    35→
    36→if __name__ == "__main__":
    37→    asyncio.run(main())
    38→


     1→# Milestone: Real STT + TTS Integration
     2→
     3→> **日期：** 2026-03-16
     4→> **阶段：** Phase 2 + Phase 4（合并完成）
     5→> **状态：** ✅ 单元测试全部通过，等待实机验证
     6→
     7→---
     8→
     9→## 1. 概述
    10→
    11→本次 milestone 将 openclaw-proxy 的 STT（语音识别）和 TTS（语音合成）从 mock 替换为真实实现，打通完整的语音链路：
    12→
    13→```
    14→ESP32 麦克风 → Opus 音频 → [STT: Whisper] → 文字
    15→    → [LLM: OpenClaw CLI] → 回复文字
    16→    → [TTS: edge-tts + ffmpeg] → Opus 音频 → ESP32 扬声器
    17→```
    18→
    19→**之前（Phase 0.5）：** STT 固定返回 `"测试语音输入"`，TTS 固定返回 159 字节静音帧。
    20→**现在：** STT 使用 Whisper base 模型做真实中文语音识别，TTS 使用 edge-tts 合成语音并通过 ffmpeg 编码为 Opus。
    21→
    22→---
    23→
    24→## 2. 技术选型与理由
    25→
    26→### 2.1 STT — OpenAI Whisper (base)
    27→
    28→| 项目 | 选择 |
    29→|------|------|
    30→| 引擎 | `openai-whisper` 20250625 |
    31→| 模型 | `base`（74M 参数，~140MB 磁盘） |
    32→| 运行方式 | 本地 CPU 推理（无 GPU） |
    33→| 语言 | 固定 `zh`（中文） |
    34→
    35→**为什么选 Whisper base：**
    36→- 4GB 内存 + 4 核 CPU 环境下，`base` 是精度和速度的最佳平衡点
    37→- `tiny` 太不准确，`small` 在 CPU 上太慢（>10s/句）
    38→- 本地运行，无需外部 API，无网络延迟
    39→- Whisper 内置 ffmpeg 调用，能直接处理 Opus 格式输入
    40→
    41→**不选 FunASR / SenseVoice 的原因：**
    42→- 依赖更重，安装复杂
    43→- Whisper 已满足演示需求，后续可替换
    44→
    45→### 2.2 TTS — edge-tts + ffmpeg
    46→
    47→| 项目 | 选择 |
    48→|------|------|
    49→| 合成引擎 | `edge-tts` 7.2.7（微软 Edge 在线 TTS） |
    50→| 声音 | `zh-CN-XiaoxiaoNeural`（女声，自然度高） |


List .claude directory


python -m pytest tests/test_main.py --co -q 2>/dev/null || echo "Cannot list tests"

List all test names to understand exact test inventory


total 8
drwxr-xr-x 2 root root   33 Mar 16 00:23 .
drwxr-xr-x 7 root root 4096 Mar 16 00:28 ..
-rw-r--r-- 1 root root  452 Mar 16 00:23 settings.local.json


Cannot list tests


python -c "import main; print('import ok')" 2>&1 | head -5

Check if main.py can be imported (dependency availability)


     1→{
     2→  "permissions": {
     3→    "allow": [
     4→      "Bash(python:*)",
     5→      "Bash(python3:*)",
     6→      "Bash(source:*)",
     7→      "Bash(ls:*)",
     8→      "Bash(curl:*)",
     9→      "Bash(which openclaw:*)",
    10→      "Bash(openclaw gateway:*)",
    11→      "Bash(openclaw chat:*)",
    12→      "Bash(openclaw agent:*)",
    13→      "Bash(which wscat:*)",
    14→      "Bash(npm install:*)",
    15→      "Bash(which ffmpeg:*)",
    16→      "Bash(pip3 list:*)",
    17→      "Bash(ffmpeg:*)",
    18→      "Bash(tree:*)"
    19→    ]
    20→  }
    21→}
    22→


/bin/bash: line 1: python: command not found


source venv/bin/activate && python -m pytest tests/test_main.py --co -q 2>&1

List all existing test names using venv


tests/test_main.py::test_process_stt_calls_whisper
tests/test_main.py::test_call_openclaw_parses_cli_output
tests/test_main.py::test_call_openclaw_raises_on_cli_failure
tests/test_main.py::test_call_openclaw_raises_on_empty_payloads
tests/test_main.py::test_process_tts_returns_opus_bytes
tests/test_main.py::test_process_tts_raises_on_ffmpeg_failure
tests/test_main.py::test_health_endpoint
tests/test_main.py::test_websocket_connect_and_disconnect
tests/test_main.py::test_websocket_text_frame_wake_word
tests/test_main.py::test_websocket_text_frame_malformed_json
tests/test_main.py::test_websocket_binary_frame_full_pipeline

11 tests collected in 2.04s


source venv/bin/activate && python -m pytest tests/test_main.py -v 2>&1

Run all tests to confirm current green state


============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-9.0.2, pluggy-1.6.0 -- /root/projects/openclaw-proxy/venv/bin/python
cachedir: .pytest_cache
rootdir: /root/projects/openclaw-proxy
plugins: anyio-4.12.1, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 11 items

tests/test_main.py::test_process_stt_calls_whisper PASSED                [  9%]
tests/test_main.py::test_call_openclaw_parses_cli_output PASSED          [ 18%]
tests/test_main.py::test_call_openclaw_raises_on_cli_failure PASSED      [ 27%]
tests/test_main.py::test_call_openclaw_raises_on_empty_payloads PASSED   [ 36%]
tests/test_main.py::test_process_tts_returns_opus_bytes PASSED           [ 45%]
tests/test_main.py::test_process_tts_raises_on_ffmpeg_failure PASSED     [ 54%]
tests/test_main.py::test_health_endpoint PASSED                          [ 63%]
tests/test_main.py::test_websocket_connect_and_disconnect PASSED         [ 72%]
tests/test_main.py::test_websocket_text_frame_wake_word PASSED           [ 81%]
tests/test_main.py::test_websocket_text_frame_malformed_json PASSED      [ 90%]
tests/test_main.py::test_websocket_binary_frame_full_pipeline PASSED     [100%]

============================== 11 passed in 2.08s ==============================


Grep

Phase 1.*main\.py|Phase 1.*模块|从 Phase 1 开始


193:> 注：Phase 0 阶段全部逻辑集中在 `main.py`，从 Phase 1 开始按上述结构拆分模块。