Error: The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file). STOP what you are doing and wait for the user to tell you how to proceed.


[Request interrupted by user for tool use]


Implement the following plan:

# Plan: Phase 1 — Audio Buffering (Press-to-Talk)

## Context

Phase 0/0.5/2/4 已完成：WebSocket 握手、OpenClaw CLI 对接、真实 STT (Whisper) 和 TTS (edge-tts) 均已跑通，11 个测试全部通过。

**当前问题：** 每个二进制帧独立触发完整 STT→LLM→TTS 管线。ESP32 按住按键说话时会发送多个 Opus 音频帧，当前每帧分别识别和回复，导致乱序错误。

**目标：** 实现 press-to-talk 状态机——`key_down`/`wake_word` 开始缓冲，二进制帧追加到缓冲区，`key_up` 合并音频后触发一次管线。

## 设计决策

| 问题 | 决策 |
|------|------|
| SessionState 存储位置 | `websocket_chat` 局部变量，传引用给 handler（最简单，无需清理） |
| wake_word vs key_down | 行为一致，均触发 idle→listening 转换 |
| 最大缓冲 | 10MB（`MAX_AUDIO_BUFFER_BYTES`），超限 drop 帧并 log warning |
| 空音频 key_up | Log warning，跳过管线，直接回 idle |
| idle 状态收到二进制帧 | Drop + log warning（严格状态机） |
| processing 状态收到二进制帧 | Drop + log warning |
| 管线异常 | `finally` 块保证 state 回 idle |

## 状态机

```
idle ──(key_down/wake_word)──► listening ──(key_up)──► processing ──(done/error)──► idle
         send "listening"       append audio            drain → STT→LLM→TTS
```

## 修改文件

### 1. `tests/test_main.py` — 先写失败测试（TDD）

**新增测试（8 个）：**

| 测试 | 类型 | 验证点 |
|------|------|--------|
| `test_session_state_defaults` | 单元 | status=="idle", audio_chunks==[] |
| `test_key_down_starts_listening` | 集成 | key_down → 返回 listening 状态 |
| `test_key_up_triggers_pipeline` | 集成 | key_down → binary → key_up → thinking + text_reply + audio |
| `test_key_up_empty_buffer_skips_pipeline` | 集成 | key_down → key_up（无音频）→ 不触发管线，连接存活 |
| `test_binary_frame_while_idle_is_dropped` | 集成 | idle 状态发 binary → 不触发管线，连接存活 |
| `test_binary_frames_are_concatenated` | 集成 | 3 个 binary 帧合并后传给 process_stt |
| `test_repeated_key_down_resets_buffer` | 集成 | key_down → binary → key_down → binary → key_up → 只有第二次的音频 |
| `test_key_up_without_key_down_is_ignored` | 集成 | idle 状态发 key_up → 不崩溃，连接存活 |

**修改 1 个现有测试：**
- `test_websocket_binary_frame_full_pipeline` → 改为 key_down → binary → key_up 序列

### 2. `main.py` — 实现变更

#### 2a. 新增导入

```python
from typing import Literal  # 第 1-14 行区域
```

#### 2b. 新增常量（Configuration 区域）

```python
MAX_AUDIO_BUFFER_BYTES: int = 10 * 1024 * 1024  # 10 MB
```

#### 2c. 新增 `SessionState` 模型（Pydantic models 区域后，约第 65 行后）

```python
class SessionState(BaseModel):
    status: Literal["idle", "listening", "processing"] = "idle"
    audio_chunks: list[bytes] = []

    def buffer_size(self) -> int:
        return sum(len(c) for c in self.audio_chunks)

    def drain_audio(self) -> bytes:
        audio = b"".join(self.audio_chunks)
        self.audio_chunks.clear()
        return audio

    def reset(self) -> None:
        self.status = "idle"
        self.audio_chunks.clear()
```

#### 2d. 重构 `handle_text_frame`

签名变更：`(ws, raw)` → `(ws, raw, state, session_id)`

新增逻辑：
- `KEY_DOWN` / `WAKE_WORD`：idle→listening（send "listening"）；listening 时 reset buffer 重新开始
- `KEY_UP`：listening→processing，调用 `handle_pipeline`；idle 时 ignore

#### 2e. 重构 `handle_binary_frame`

签名变更：`(ws, data, session_id)` → `(ws, data, state)`

新逻辑：仅在 listening 状态 append，其余状态 drop。检查 buffer 大小上限。
**不再调用 process_stt/call_openclaw/process_tts。**

#### 2f. 新增 `handle_pipeline(ws, state, session_id)`

从旧 `handle_binary_frame` 提取 STT→LLM→TTS 逻辑：
1. `audio = state.drain_audio()`
2. 空音频 → log + reset + return
3. `try:` 执行 STT → thinking → heartbeat + LLM → text_reply → TTS → binary
4. `finally: state.reset()` 保证回 idle

#### 2g. 更新 `websocket_chat`

```python
state = SessionState()  # 新增
# 传 state 给 handler：
await handle_text_frame(ws, text_data, state, session_id)
await handle_binary_frame(ws, byte_data, state)
```

### 3. `test_client.py` — 更新手动测试脚本

改为 key_down → binary → key_up 序列。

## 不变的部分

- `process_stt`, `call_openclaw`, `process_tts` — 内部不变
- Pydantic 消息模型（HardwareAction/Event, StateNotification, TextReply）— 不变
- `/health` 端点 — 不变
- 6 个管线单元测试 — 不变
- requirements.txt — 不变

## 验证

```bash
source venv/bin/activate && python -m pytest tests/ -v
# 预期：~19 个测试全部通过
```

If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /root/.claude/projects/-root-projects-openclaw-proxy/ec25cb04-1036-4d7f-aa28-2cea626b2974.jsonl


Let me start by understanding the current codebase before implementing the plan. I need to read the existing files to understand the current state.


Let me read the current codebase first.


     1→"""Quick WebSocket client to test the full proxy pipeline."""
     2→
     3→import asyncio
     4→import json
     5→import websockets
     6→
     7→
     8→async def main() -> None:
     9→    uri = "ws://localhost:8000/chat"
    10→    async with websockets.connect(uri) as ws:
    11→        print("=== Connected ===\n")
    12→
    13→        # 1) Test text frame: wake_word
    14→        print("[TX] wake_word")
    15→        await ws.send(json.dumps({"type": "hardware_event", "action": "wake_word"}))
    16→        resp = json.loads(await ws.recv())
    17→        print(f"[RX] {resp}\n")
    18→
    19→        # 2) Test binary frame: triggers STT → LLM → TTS pipeline
    20→        print("[TX] binary audio (80 bytes of mock Opus)")
    21→        await ws.send(b"\x00" * 80)
    22→
    23→        # Collect all responses until we get binary audio back
    24→        while True:
    25→            msg = await ws.recv()
    26→            if isinstance(msg, bytes):
    27→                print(f"[RX] binary audio: {len(msg)} bytes")
    28→                break
    29→            else:
    30→                data = json.loads(msg)
    31→                print(f"[RX] {data}")
    32→
    33→        print("\n=== Done ===")
    34→
    35→
    36→if __name__ == "__main__":
    37→    asyncio.run(main())
    38→

"""Quick WebSocket client to test the full proxy pipeline."""

import asyncio
import json
import websockets

async def main() -> None:
    uri = "ws://localhost:8000/chat"
    async with websockets.connect(uri) as ws:
        print("=== Connected ===\n")

        # 1) Test text frame: wake_word
        print("[TX] wake_word")
        await ws.send(json.dumps({"type": "hardware_event", "action": "wake_word"}))
        resp = json.loads(await ws.recv())
        print(f"[RX] {resp}\n")

        # 2) Test binary frame: triggers STT → LLM → TTS pipeline
        print("[TX] binary audio (80 bytes of mock Opus)")
        await ws.send(b"\x00" * 80)

        # Collect all responses until we get binary audio back
        while True:
            msg = await ws.recv()
            if isinstance(msg, bytes):
                print(f"[RX] binary audio: {len(msg)} bytes")
                break
            else:
                data = json.loads(msg)
                print(f"[RX] {data}")

        print("\n=== Done ===")

if __name__ == "__main__":
    asyncio.run(main())


     1→"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""
     2→
     3→import asyncio
     4→import json
     5→import logging
     6→import tempfile
     7→import uuid
     8→from enum import Enum
     9→from shutil import which
    10→
    11→import edge_tts
    12→import whisper
    13→from fastapi import FastAPI, WebSocket, WebSocketDisconnect
    14→from pydantic import BaseModel, ValidationError
    15→
    16→# ── Logging ──────────────────────────────────────────────────────────
    17→
    18→logging.basicConfig(
    19→    level=logging.INFO,
    20→    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    21→)
    22→logger = logging.getLogger("openclaw-proxy")
    23→
    24→# ── Configuration ────────────────────────────────────────────────────
    25→
    26→OPENCLAW_CLI = which("openclaw") or "openclaw"
    27→OPENCLAW_AGENT_ID = "main"
    28→OPENCLAW_TIMEOUT = 120  # seconds
    29→WHISPER_MODEL_SIZE = "base"
    30→EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
    31→FFMPEG_BIN = which("ffmpeg") or "ffmpeg"
    32→
    33→
    34→# ── Pydantic models ─────────────────────────────────────────────────
    35→
    36→
    37→class HardwareAction(str, Enum):
    38→    """Actions reported by XiaoZhi ESP32 hardware."""
    39→
    40→    KEY_DOWN = "key_down"
    41→    KEY_UP = "key_up"
    42→    WAKE_WORD = "wake_word"
    43→
    44→
    45→class HardwareEvent(BaseModel):
    46→    """Incoming JSON frame from hardware."""
    47→
    48→    type: str
    49→    action: HardwareAction
    50→
    51→
    52→class StateNotification(BaseModel):
    53→    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""
    54→
    55→    type: str = "state"
    56→    status: str
    57→
    58→
    59→class TextReply(BaseModel):
    60→    """Outgoing text reply carrying the LLM response."""
    61→
    62→    type: str = "text_reply"
    63→    text: str
    64→
    65→
    66→# ── Whisper model (lazy singleton) ─────────────────────────────────
    67→
    68→_whisper_model: whisper.Whisper | None = None
    69→
    70→
    71→def _get_whisper_model() -> whisper.Whisper:
    72→    """Return the Whisper model, loading it on first call."""
    73→    global _whisper_model
    74→    if _whisper_model is None:
    75→        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
    76→        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
    77→    return _whisper_model
    78→
    79→
    80→# ── STT / TTS pipeline ────────────────────────────────────────────
    81→
    82→
    83→async def process_stt(audio_bytes: bytes) -> str:
    84→    """Convert Opus audio to text via Whisper.
    85→
    86→    Args:
    87→        audio_bytes: Raw Opus-encoded audio from ESP32.
    88→
    89→    Returns:
    90→        Transcribed text string.
    91→    """
    92→    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    93→    model = _get_whisper_model()
    94→    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
    95→        f.write(audio_bytes)
    96→        f.flush()
    97→        result = await asyncio.to_thread(
    98→            model.transcribe, f.name, language="zh"
    99→        )
   100→    text = result["text"].strip()
   101→    logger.info("STT: transcribed → %s", text)
   102→    return text
   103→
   104→
   105→async def call_openclaw(text: str, session_id: str) -> str:
   106→    """Send text to OpenClaw agent via CLI and return its reply.
   107→
   108→    Args:
   109→        text: User utterance transcribed by STT.
   110→        session_id: Per-connection session ID for multi-turn conversation.
   111→
   112→    Returns:
   113→        Agent reply string.
   114→
   115→    Raises:
   116→        RuntimeError: If the CLI call fails or returns unexpected output.
   117→    """
   118→    logger.info("LLM: sending to OpenClaw → %s", text)
   119→
   120→    proc = await asyncio.create_subprocess_exec(
   121→        OPENCLAW_CLI, "agent",
   122→        "--agent", OPENCLAW_AGENT_ID,
   123→        "--session-id", session_id,
   124→        "-m", text,
   125→        "--json",
   126→        "--timeout", str(OPENCLAW_TIMEOUT),
   127→        stdout=asyncio.subprocess.PIPE,
   128→        stderr=asyncio.subprocess.PIPE,
   129→    )
   130→    stdout, stderr = await proc.communicate()
   131→
   132→    if proc.returncode != 0:
   133→        err_msg = stderr.decode(errors="replace").strip()
   134→        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
   135→        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")
   136→
   137→    data = json.loads(stdout.decode())
   138→    payloads = data.get("result", {}).get("payloads", [])
   139→    if not payloads:
   140→        raise RuntimeError("OpenClaw returned empty payloads")
   141→
   142→    reply = payloads[0].get("text", "")
   143→    logger.info("LLM: reply ← %s", reply)
   144→    return reply
   145→
   146→
   147→async def process_tts(text: str) -> bytes:
   148→    """Convert text to Opus audio via edge-tts and ffmpeg.
   149→
   150→    Args:
   151→        text: Text to synthesize.
   152→
   153→    Returns:
   154→        Opus-encoded audio bytes.
   155→
   156→    Raises:
   157→        RuntimeError: If ffmpeg conversion fails.
   158→    """
   159→    logger.info("TTS: synthesising → %s", text)
   160→
   161→    # 1. edge-tts → MP3
   162→    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
   163→    mp3_chunks: list[bytes] = []
   164→    async for chunk in communicate.stream():
   165→        if chunk["type"] == "audio":
   166→            mp3_chunks.append(chunk["data"])
   167→    mp3_bytes = b"".join(mp3_chunks)
   168→
   169→    # 2. MP3 → Opus via ffmpeg
   170→    proc = await asyncio.create_subprocess_exec(
   171→        FFMPEG_BIN, "-i", "pipe:0",
   172→        "-c:a", "libopus", "-b:a", "24k",
   173→        "-ar", "16000", "-ac", "1",
   174→        "-f", "opus", "pipe:1",
   175→        stdin=asyncio.subprocess.PIPE,
   176→        stdout=asyncio.subprocess.PIPE,
   177→        stderr=asyncio.subprocess.PIPE,
   178→    )
   179→    opus_bytes, stderr = await proc.communicate(input=mp3_bytes)
   180→
   181→    if proc.returncode != 0:
   182→        err_msg = stderr.decode(errors="replace").strip()
   183→        logger.error("ffmpeg failed (rc=%d): %s", proc.returncode, err_msg)
   184→        raise RuntimeError(f"ffmpeg TTS conversion error: {err_msg}")
   185→
   186→    logger.info("TTS: generated %d bytes of Opus audio", len(opus_bytes))
   187→    return opus_bytes
   188→
   189→
   190→# ── Frame router ────────────────────────────────────────────────────
   191→
   192→
   193→async def handle_text_frame(ws: WebSocket, raw: str) -> None:
   194→    """Parse and route an incoming JSON text frame.
   195→
   196→    Args:
   197→        ws: Active WebSocket connection.
   198→        raw: Raw text payload received from client.
   199→    """
   200→    try:
   201→        payload = json.loads(raw)
   202→    except json.JSONDecodeError:
   203→        logger.warning("Malformed JSON (ignored): %.120s", raw)
   204→        return
   205→
   206→    try:
   207→        event = HardwareEvent(**payload)
   208→    except ValidationError as exc:
   209→        logger.warning("Invalid hardware event: %s", exc.errors())
   210→        return
   211→
   212→    logger.info("Hardware event: %s", event.action.value)
   213→
   214→    if event.action == HardwareAction.WAKE_WORD:
   215→        ack = StateNotification(status="listening")
   216→        await ws.send_text(ack.model_dump_json())
   217→
   218→
   219→async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
   220→    """Run the full STT → LLM → TTS pipeline on an audio binary frame.
   221→
   222→    Args:
   223→        ws: Active WebSocket connection.
   224→        data: Raw Opus audio bytes.
   225→        session_id: Per-connection session ID.
   226→
   227→    Response sequence pushed to client:
   228→        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
   229→        2. {"type": "text_reply", "text": "..."}     — agent answer
   230→        3. <binary>                                   — Opus audio of the answer
   231→    """
   232→    # Stage 1 — STT
   233→    text = await process_stt(data)
   234→
   235→    # Notify hardware: entering thinking phase
   236→    await ws.send_text(StateNotification(status="thinking").model_dump_json())
   237→
   238→    # Stage 2 — LLM (with periodic thinking heartbeats)
   239→    async def heartbeat() -> None:
   240→        """Send thinking heartbeat every 3 seconds while LLM is working."""
   241→        while True:
   242→            await asyncio.sleep(3)
   243→            try:
   244→                await ws.send_text(
   245→                    StateNotification(status="thinking").model_dump_json()
   246→                )
   247→            except Exception:
   248→                break
   249→
   250→    heartbeat_task = asyncio.create_task(heartbeat())
   251→    try:
   252→        reply = await call_openclaw(text, session_id)
   253→    finally:
   254→        heartbeat_task.cancel()
   255→
   256→    await ws.send_text(TextReply(text=reply).model_dump_json())
   257→
   258→    # Stage 3 — TTS
   259→    audio = await process_tts(reply)
   260→    logger.info("Pushing %d bytes of audio to client", len(audio))
   261→    await ws.send_bytes(audio)
   262→
   263→
   264→# ── FastAPI application ─────────────────────────────────────────────
   265→
   266→app = FastAPI(title="openclaw-proxy", version="0.1.0")
   267→
   268→
   269→@app.websocket("/chat")
   270→async def websocket_chat(ws: WebSocket) -> None:
   271→    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
   272→    await ws.accept()
   273→    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
   274→    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
   275→    logger.info("Client connected: %s (session=%s)", peer, session_id)
   276→
   277→    try:
   278→        while True:
   279→            message = await ws.receive()
   280→
   281→            if message["type"] == "websocket.disconnect":
   282→                break
   283→
   284→            text_data = message.get("text")
   285→            byte_data = message.get("bytes")
   286→
   287→            if text_data is not None:
   288→                await handle_text_frame(ws, text_data)
   289→            elif byte_data is not None:
   290→                await handle_binary_frame(ws, byte_data, session_id)
   291→    except WebSocketDisconnect:
   292→        pass
   293→    finally:
   294→        logger.info("Client disconnected: %s (session=%s)", peer, session_id)
   295→
   296→
   297→@app.get("/health")
   298→async def health() -> dict[str, str]:
   299→    """Liveness probe."""
   300→    return {"status": "ok"}
   301→
   302→
   303→# ── Entry point ─────────────────────────────────────────────────────
   304→
   305→if __name__ == "__main__":
   306→    import uvicorn
   307→
   308→    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)
   309→

"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import edge_tts
import whisper
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

# ── Logging ──────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("openclaw-proxy")

# ── Configuration ────────────────────────────────────────────────────

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds
WHISPER_MODEL_SIZE = "base"
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"

# ── Pydantic models ─────────────────────────────────────────────────

class HardwareAction(str, Enum):
    """Actions reported by XiaoZhi ESP32 hardware."""

    KEY_DOWN = "key_down"
    KEY_UP = "key_up"
    WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
    """Incoming JSON frame from hardware."""

    type: str
    action: HardwareAction

class StateNotification(BaseModel):
    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""

    type: str = "state"
    status: str

class TextReply(BaseModel):
    """Outgoing text reply carrying the LLM response."""

    type: str = "text_reply"
    text: str

# ── Whisper model (lazy singleton) ─────────────────────────────────

_whisper_model: whisper.Whisper | None = None

def _get_whisper_model() -> whisper.Whisper:
    """Return the Whisper model, loading it on first call."""
    global _whisper_model
    if _whisper_model is None:
        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
    return _whisper_model

# ── STT / TTS pipeline ────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text via Whisper.

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    model = _get_whisper_model()
    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
        f.write(audio_bytes)
        f.flush()
        result = await asyncio.to_thread(
            model.transcribe, f.name, language="zh"
        )
    text = result["text"].strip()
    logger.info("STT: transcribed → %s", text)
    return text

async def call_openclaw(text: str, session_id: str) -> str:
    """Send text to OpenClaw agent via CLI and return its reply.

    Args:
        text: User utterance transcribed by STT.
        session_id: Per-connection session ID for multi-turn conversation.

    Returns:
        Agent reply string.

    Raises:
        RuntimeError: If the CLI call fails or returns unexpected output.
    """
    logger.info("LLM: sending to OpenClaw → %s", text)

    proc = await asyncio.create_subprocess_exec(
        OPENCLAW_CLI, "agent",
        "--agent", OPENCLAW_AGENT_ID,
        "--session-id", session_id,
        "-m", text,
        "--json",
        "--timeout", str(OPENCLAW_TIMEOUT),
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")

    data = json.loads(stdout.decode())
    payloads = data.get("result", {}).get("payloads", [])
    if not payloads:
        raise RuntimeError("OpenClaw returned empty payloads")

    reply = payloads[0].get("text", "")
    logger.info("LLM: reply ← %s", reply)
    return reply

async def process_tts(text: str) -> bytes:
    """Convert text to Opus audio via edge-tts and ffmpeg.

    Args:
        text: Text to synthesize.

    Returns:
        Opus-encoded audio bytes.

    Raises:
        RuntimeError: If ffmpeg conversion fails.
    """
    logger.info("TTS: synthesising → %s", text)

    # 1. edge-tts → MP3
    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
    mp3_chunks: list[bytes] = []
    async for chunk in communicate.stream():
        if chunk["type"] == "audio":
            mp3_chunks.append(chunk["data"])
    mp3_bytes = b"".join(mp3_chunks)

    # 2. MP3 → Opus via ffmpeg
    proc = await asyncio.create_subprocess_exec(
        FFMPEG_BIN, "-i", "pipe:0",
        "-c:a", "libopus", "-b:a", "24k",
        "-ar", "16000", "-ac", "1",
        "-f", "opus", "pipe:1",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    opus_bytes, stderr = await proc.communicate(input=mp3_bytes)

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("ffmpeg failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"ffmpeg TTS conversion error: {err_msg}")

    logger.info("TTS: generated %d bytes of Opus audio", len(opus_bytes))
    return opus_bytes

# ── Frame router ────────────────────────────────────────────────────

async def handle_text_frame(ws: WebSocket, raw: str) -> None:
    """Parse and route an incoming JSON text frame.

    Args:
        ws: Active WebSocket connection.
        raw: Raw text payload received from client.
    """
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        logger.warning("Malformed JSON (ignored): %.120s", raw)
        return

    try:
        event = HardwareEvent(**payload)
    except ValidationError as exc:
        logger.warning("Invalid hardware event: %s", exc.errors())
        return

    logger.info("Hardware event: %s", event.action.value)

    if event.action == HardwareAction.WAKE_WORD:
        ack = StateNotification(status="listening")
        await ws.send_text(ack.model_dump_json())

async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
    """Run the full STT → LLM → TTS pipeline on an audio binary frame.

    Args:
        ws: Active WebSocket connection.
        data: Raw Opus audio bytes.
        session_id: Per-connection session ID.

    Response sequence pushed to client:
        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
        2. {"type": "text_reply", "text": "..."}     — agent answer
        3. <binary>                                   — Opus audio of the answer
    """
    # Stage 1 — STT
    text = await process_stt(data)

    # Notify hardware: entering thinking phase
    await ws.send_text(StateNotification(status="thinking").model_dump_json())

    # Stage 2 — LLM (with periodic thinking heartbeats)
    async def heartbeat() -> None:
        """Send thinking heartbeat every 3 seconds while LLM is working."""
        while True:
            await asyncio.sleep(3)
            try:
                await ws.send_text(
                    StateNotification(status="thinking").model_dump_json()
                )
            except Exception:
                break

    heartbeat_task = asyncio.create_task(heartbeat())
    try:
        reply = await call_openclaw(text, session_id)
    finally:
        heartbeat_task.cancel()

    await ws.send_text(TextReply(text=reply).model_dump_json())

    # Stage 3 — TTS
    audio = await process_tts(reply)
    logger.info("Pushing %d bytes of audio to client", len(audio))
    await ws.send_bytes(audio)

# ── FastAPI application ─────────────────────────────────────────────

app = FastAPI(title="openclaw-proxy", version="0.1.0")

@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
    await ws.accept()
    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    logger.info("Client connected: %s (session=%s)", peer, session_id)

    try:
        while True:
            message = await ws.receive()

            if message["type"] == "websocket.disconnect":
                break

            text_data = message.get("text")
            byte_data = message.get("bytes")

            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect:
        pass
    finally:
        logger.info("Client disconnected: %s (session=%s)", peer, session_id)

@app.get("/health")
async def health() -> dict[str, str]:
    """Liveness probe."""
    return {"status": "ok"}

# ── Entry point ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)


     1→"""Tests for openclaw-proxy WebSocket middleware."""
     2→
     3→import json
     4→from unittest.mock import AsyncMock, MagicMock, patch
     5→
     6→import pytest
     7→from starlette.testclient import TestClient
     8→
     9→from main import app, process_stt, call_openclaw, process_tts
    10→
    11→
    12→# ── Helpers ──────────────────────────────────────────────────────────
    13→
    14→MOCK_OPENCLAW_OUTPUT = json.dumps({
    15→    "status": "ok",
    16→    "result": {
    17→        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    18→    },
    19→}).encode()
    20→
    21→
    22→def _make_mock_process(
    23→    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    24→    returncode: int = 0,
    25→) -> AsyncMock:
    26→    """Build a mock asyncio.Process with preset stdout/stderr."""
    27→    proc = AsyncMock()
    28→    proc.communicate.return_value = (stdout, b"")
    29→    proc.returncode = returncode
    30→    return proc
    31→
    32→
    33→# ── Unit tests: pipeline functions ───────────────────────────────────
    34→
    35→
    36→@pytest.mark.asyncio
    37→async def test_process_stt_calls_whisper() -> None:
    38→    """STT should write audio to temp file, call Whisper, and return stripped text."""
    39→    mock_model = MagicMock()
    40→    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    41→    with patch("main._get_whisper_model", return_value=mock_model):
    42→        result = await process_stt(b"\x00" * 80)
    43→    assert result == "你好世界"
    44→    mock_model.transcribe.assert_called_once()
    45→    args, kwargs = mock_model.transcribe.call_args
    46→    assert args[0].endswith(".opus")
    47→    assert kwargs == {"language": "zh"}
    48→
    49→
    50→@pytest.mark.asyncio
    51→async def test_call_openclaw_parses_cli_output() -> None:
    52→    """call_openclaw should parse the CLI JSON and return reply text."""
    53→    mock_proc = _make_mock_process()
    54→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    55→        result = await call_openclaw("你好", "test-session")
    56→    assert result == "你好！我在这儿。"
    57→
    58→
    59→@pytest.mark.asyncio
    60→async def test_call_openclaw_raises_on_cli_failure() -> None:
    61→    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    62→    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    63→    mock_proc.communicate.return_value = (b"", b"some error")
    64→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    65→        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
    66→            await call_openclaw("你好", "test-session")
    67→
    68→
    69→@pytest.mark.asyncio
    70→async def test_call_openclaw_raises_on_empty_payloads() -> None:
    71→    """call_openclaw should raise RuntimeError on empty payloads."""
    72→    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    73→    mock_proc = _make_mock_process(stdout=empty)
    74→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    75→        with pytest.raises(RuntimeError, match="empty payloads"):
    76→            await call_openclaw("你好", "test-session")
    77→
    78→
    79→@pytest.mark.asyncio
    80→async def test_process_tts_returns_opus_bytes() -> None:
    81→    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""
    82→
    83→    async def mock_stream() -> None:  # type: ignore[override]
    84→        yield {"type": "audio", "data": b"mp3-part1"}
    85→        yield {"type": "audio", "data": b"mp3-part2"}
    86→        yield {"type": "WordBoundary", "data": None}  # non-audio chunk
    87→
    88→    mock_comm = MagicMock()
    89→    mock_comm.stream = mock_stream
    90→
    91→    mock_proc = AsyncMock()
    92→    mock_proc.communicate.return_value = (b"fake-opus", b"")
    93→    mock_proc.returncode = 0
    94→
    95→    with (
    96→        patch("main.edge_tts.Communicate", return_value=mock_comm),
    97→        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    98→    ):
    99→        result = await process_tts("测试文本")
   100→
   101→    assert result == b"fake-opus"
   102→    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")
   103→
   104→
   105→@pytest.mark.asyncio
   106→async def test_process_tts_raises_on_ffmpeg_failure() -> None:
   107→    """TTS should raise RuntimeError when ffmpeg exits non-zero."""
   108→
   109→    async def mock_stream() -> None:  # type: ignore[override]
   110→        yield {"type": "audio", "data": b"mp3-data"}
   111→
   112→    mock_comm = MagicMock()
   113→    mock_comm.stream = mock_stream
   114→
   115→    mock_proc = AsyncMock()
   116→    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
   117→    mock_proc.returncode = 1
   118→
   119→    with (
   120→        patch("main.edge_tts.Communicate", return_value=mock_comm),
   121→        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
   122→    ):
   123→        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
   124→            await process_tts("测试文本")
   125→
   126→
   127→# ── Integration tests: HTTP & WebSocket ──────────────────────────────
   128→
   129→
   130→def test_health_endpoint() -> None:
   131→    """GET /health should return 200 with ok status."""
   132→    client = TestClient(app)
   133→    resp = client.get("/health")
   134→    assert resp.status_code == 200
   135→    assert resp.json() == {"status": "ok"}
   136→
   137→
   138→def test_websocket_connect_and_disconnect() -> None:
   139→    """Client should be able to connect to /chat and cleanly disconnect."""
   140→    client = TestClient(app)
   141→    with client.websocket_connect("/chat"):
   142→        pass
   143→
   144→
   145→def test_websocket_text_frame_wake_word() -> None:
   146→    """Sending a wake_word JSON should get back a 'listening' state."""
   147→    client = TestClient(app)
   148→    with client.websocket_connect("/chat") as ws:
   149→        ws.send_text(json.dumps({
   150→            "type": "hardware_event",
   151→            "action": "wake_word",
   152→        }))
   153→        resp = json.loads(ws.receive_text())
   154→        assert resp["type"] == "state"
   155→        assert resp["status"] == "listening"
   156→
   157→
   158→def test_websocket_text_frame_malformed_json() -> None:
   159→    """Malformed JSON should not crash the connection."""
   160→    client = TestClient(app)
   161→    with client.websocket_connect("/chat") as ws:
   162→        ws.send_text("not-valid-json{{{")
   163→        ws.send_text(json.dumps({
   164→            "type": "hardware_event",
   165→            "action": "wake_word",
   166→        }))
   167→        resp = json.loads(ws.receive_text())
   168→        assert resp["status"] == "listening"
   169→
   170→
   171→def test_websocket_binary_frame_full_pipeline() -> None:
   172→    """Binary audio should trigger STT → LLM → TTS pipeline.
   173→
   174→    Expected response sequence:
   175→      1. {"type": "state", "status": "thinking"}
   176→      2. {"type": "text_reply", "text": "..."}
   177→      3. binary bytes (TTS audio)
   178→    """
   179→    client = TestClient(app)
   180→    with (
   181→        patch(
   182→            "main.process_stt",
   183→            new_callable=AsyncMock,
   184→            return_value="测试语音输入",
   185→        ),
   186→        patch(
   187→            "main.call_openclaw",
   188→            new_callable=AsyncMock,
   189→            return_value="你好！我在这儿。",
   190→        ),
   191→        patch(
   192→            "main.process_tts",
   193→            new_callable=AsyncMock,
   194→            return_value=b"\xfc\xff\xfe",
   195→        ),
   196→    ):
   197→        with client.websocket_connect("/chat") as ws:
   198→            ws.send_bytes(b"\x00" * 80)
   199→
   200→            # 1) thinking state
   201→            msg1 = json.loads(ws.receive_text())
   202→            assert msg1 == {"type": "state", "status": "thinking"}
   203→
   204→            # 2) text reply
   205→            msg2 = json.loads(ws.receive_text())
   206→            assert msg2["type"] == "text_reply"
   207→            assert msg2["text"] == "你好！我在这儿。"
   208→
   209→            # 3) TTS audio bytes
   210→            audio = ws.receive_bytes()
   211→            assert audio == b"\xfc\xff\xfe"
   212→

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    client = TestClient(app)
    with (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    ):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"


Now I have a clear picture of the codebase. Let me implement the plan step by step following TDD:

1. First write the failing tests
2. Then implement the changes in main.py
3. Update test_client.py

Let me create the task list and start implementing.


Good, I have the full picture. Let me implement this following TDD — tests first, then implementation.


TaskCreate

Write failing tests for audio buffering state machine

Add 8 new tests and modify 1 existing test in tests/test_main.py for the press-to-talk state machine.


Task #1 created successfully: Write failing tests for audio buffering state machine


Implementing SessionState and state machine

Add SessionState model, MAX_AUDIO_BUFFER_BYTES constant, refactor handle_text_frame/handle_binary_frame, add handle_pipeline, update websocket_chat.

Implement state machine in main.py


Task #2 created successfully: Implement state machine in main.py

Implement state machine in main.py


Update test_client.py to use key_down → binary → key_up sequence.

Update test_client.py for press-to-talk flow


Task #3 created successfully: Update test_client.py for press-to-talk flow


Run pytest and verify all ~19 tests pass.

Run tests and verify all pass


Task #4 created successfully: Run tests and verify all pass

Run tests and verify all pass


Updated task #1 status


Now writing the tests. I need to import `SessionState` and update/add tests.


Write

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import (
    SessionState,
    app,
    process_stt,
    call_openclaw,
    process_tts,
    MAX_AUDIO_BUFFER_BYTES,
)

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

def _key_down_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "key_down"})

def _key_up_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "key_up"})

def _wake_word_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "wake_word"})

def _pipeline_patches():
    """Context manager that patches STT/LLM/TTS pipeline functions."""
    return (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    )

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

# ── Unit tests: SessionState ─────────────────────────────────────────

def test_session_state_defaults() -> None:
    """SessionState should start idle with empty audio_chunks."""
    state = SessionState()
    assert state.status == "idle"
    assert state.audio_chunks == []
    assert state.buffer_size() == 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(_wake_word_msg())
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(_wake_word_msg())
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_key_down_starts_listening() -> None:
    """key_down should transition to listening and return state notification."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(_key_down_msg())
        resp = json.loads(ws.receive_text())
        assert resp == {"type": "state", "status": "listening"}

def test_key_up_triggers_pipeline() -> None:
    """key_down → binary → key_up should trigger full STT→LLM→TTS pipeline."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x00" * 80)

            ws.send_text(_key_up_msg())

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"

def test_key_up_empty_buffer_skips_pipeline() -> None:
    """key_down → key_up with no audio should skip pipeline, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_text(_key_up_msg())

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive — send another event
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_binary_frame_while_idle_is_dropped() -> None:
    """Binary frame in idle state should be dropped, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_binary_frames_are_concatenated() -> None:
    """Multiple binary frames should be concatenated before STT."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x01" * 10)
            ws.send_bytes(b"\x02" * 20)
            ws.send_bytes(b"\x03" * 30)

            ws.send_text(_key_up_msg())

            # Consume pipeline responses
            ws.receive_text()  # thinking
            ws.receive_text()  # text_reply
            ws.receive_bytes()  # audio

            # STT should have received concatenated audio
            mock_stt.assert_called_once_with(
                b"\x01" * 10 + b"\x02" * 20 + b"\x03" * 30
            )

def test_repeated_key_down_resets_buffer() -> None:
    """Second key_down should reset buffer — only second chunk reaches STT."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"
            ws.send_bytes(b"\xAA" * 50)

            # Second key_down resets
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume second "listening"
            ws.send_bytes(b"\xBB" * 30)

            ws.send_text(_key_up_msg())

            ws.receive_text()  # thinking
            ws.receive_text()  # text_reply
            ws.receive_bytes()  # audio

            # Only the second chunk should be passed to STT
            mock_stt.assert_called_once_with(b"\xBB" * 30)

def test_key_up_without_key_down_is_ignored() -> None:
    """key_up in idle state should be ignored, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_up_msg())

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """key_down → binary → key_up should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x00" * 80)

            ws.send_text(_key_up_msg())

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"


The file /root/projects/openclaw-proxy/tests/test_main.py has been updated successfully.

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import (
    SessionState,
    app,
    process_stt,
    call_openclaw,
    process_tts,
    MAX_AUDIO_BUFFER_BYTES,
)

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

def _key_down_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "key_down"})

def _key_up_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "key_up"})

def _wake_word_msg() -> str:
    return json.dumps({"type": "hardware_event", "action": "wake_word"})

def _pipeline_patches():
    """Context manager that patches STT/LLM/TTS pipeline functions."""
    return (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    )

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

# ── Unit tests: SessionState ─────────────────────────────────────────

def test_session_state_defaults() -> None:
    """SessionState should start idle with empty audio_chunks."""
    state = SessionState()
    assert state.status == "idle"
    assert state.audio_chunks == []
    assert state.buffer_size() == 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(_wake_word_msg())
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(_wake_word_msg())
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_key_down_starts_listening() -> None:
    """key_down should transition to listening and return state notification."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(_key_down_msg())
        resp = json.loads(ws.receive_text())
        assert resp == {"type": "state", "status": "listening"}

def test_key_up_triggers_pipeline() -> None:
    """key_down → binary → key_up should trigger full STT→LLM→TTS pipeline."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x00" * 80)

            ws.send_text(_key_up_msg())

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"

def test_key_up_empty_buffer_skips_pipeline() -> None:
    """key_down → key_up with no audio should skip pipeline, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_text(_key_up_msg())

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive — send another event
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_binary_frame_while_idle_is_dropped() -> None:
    """Binary frame in idle state should be dropped, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_binary_frames_are_concatenated() -> None:
    """Multiple binary frames should be concatenated before STT."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x01" * 10)
            ws.send_bytes(b"\x02" * 20)
            ws.send_bytes(b"\x03" * 30)

            ws.send_text(_key_up_msg())

            # Consume pipeline responses
            ws.receive_text()  # thinking
            ws.receive_text()  # text_reply
            ws.receive_bytes()  # audio

            # STT should have received concatenated audio
            mock_stt.assert_called_once_with(
                b"\x01" * 10 + b"\x02" * 20 + b"\x03" * 30
            )

def test_repeated_key_down_resets_buffer() -> None:
    """Second key_down should reset buffer — only second chunk reaches STT."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"
            ws.send_bytes(b"\xAA" * 50)

            # Second key_down resets
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume second "listening"
            ws.send_bytes(b"\xBB" * 30)

            ws.send_text(_key_up_msg())

            ws.receive_text()  # thinking
            ws.receive_text()  # text_reply
            ws.receive_bytes()  # audio

            # Only the second chunk should be passed to STT
            mock_stt.assert_called_once_with(b"\xBB" * 30)

def test_key_up_without_key_down_is_ignored() -> None:
    """key_up in idle state should be ignored, connection alive."""
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt as mock_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_up_msg())

            # Pipeline should NOT have been called
            mock_stt.assert_not_called()

            # Connection should still be alive
            ws.send_text(_wake_word_msg())
            resp = json.loads(ws.receive_text())
            assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """key_down → binary → key_up should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    p_stt, p_llm, p_tts = _pipeline_patches()
    client = TestClient(app)
    with p_stt, p_llm, p_tts:
        with client.websocket_connect("/chat") as ws:
            ws.send_text(_key_down_msg())
            ws.receive_text()  # consume "listening"

            ws.send_bytes(b"\x00" * 80)

            ws.send_text(_key_up_msg())

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"

-from main import app, process_stt, call_openclaw, process_tts

+    return json.dumps({"type": "hardware_event", "action": "key_down"})

+    return json.dumps({"type": "hardware_event", "action": "key_up"})

+    return json.dumps({"type": "hardware_event", "action": "wake_word"})

+    """Context manager that patches STT/LLM/TTS pipeline functions."""

+            return_value=b"\xfc\xff\xfe",

+# ── Unit tests: SessionState ─────────────────────────────────────────

+def test_session_state_defaults() -> None:

+    """SessionState should start idle with empty audio_chunks."""

+def test_key_down_starts_listening() -> None:

+    """key_down should transition to listening and return state notification."""

+    with client.websocket_connect("/chat") as ws:

+        resp = json.loads(ws.receive_text())

+        assert resp == {"type": "state", "status": "listening"}

+def test_key_up_triggers_pipeline() -> None:

+    """key_down → binary → key_up should trigger full STT→LLM→TTS pipeline."""

+    p_stt, p_llm, p_tts = _pipeline_patches()

+            ws.send_text(_key_down_msg())

+            ws.receive_text()  # consume "listening"

+            ws.send_bytes(b"\x00" * 80)

+            ws.send_text(_key_up_msg())

+            msg1 = json.loads(ws.receive_text())

+            assert msg1 == {"type": "state", "status": "thinking"}

+            msg2 = json.loads(ws.receive_text())

+            assert msg2["type"] == "text_reply"

+            assert msg2["text"] == "你好！我在这儿。"

+            assert audio == b"\xfc\xff\xfe"

+def test_key_up_empty_buffer_skips_pipeline() -> None:

+    """key_down → key_up with no audio should skip pipeline, connection alive."""

+    with p_stt as mock_stt, p_llm, p_tts:

+            # Pipeline should NOT have been called

+            mock_stt.assert_not_called()

+            # Connection should still be alive — send another event

+            ws.send_text(_wake_word_msg())

+            resp = json.loads(ws.receive_text())

+            assert resp["status"] == "listening"

+def test_binary_frame_while_idle_is_dropped() -> None:

+    """Binary frame in idle state should be dropped, connection alive."""

+            ws.send_bytes(b"\x00" * 80)

+            # Connection should still be alive

+def test_binary_frames_are_concatenated() -> None:

+    """Multiple binary frames should be concatenated before STT."""

+            ws.send_text(_key_down_msg())

+            ws.send_bytes(b"\x01" * 10)

+            ws.send_bytes(b"\x02" * 20)

+            ws.send_bytes(b"\x03" * 30)

+            ws.send_text(_key_up_msg())

+            # Consume pipeline responses

+            ws.receive_text()  # thinking

+            ws.receive_text()  # text_reply

+            ws.receive_bytes()  # audio

+            # STT should have received concatenated audio

+            mock_stt.assert_called_once_with(

+def test_repeated_key_down_resets_buffer() -> None:

+    """Second key_down should reset buffer — only second chunk reaches STT."""

+    with p_stt as mock_stt, p_llm, p_tts:

+            ws.send_text(_key_down_msg())

+            ws.send_bytes(b"\xAA" * 50)

+            ws.receive_text()  # consume second "listening"

+            ws.send_bytes(b"\xBB" * 30)

+            ws.send_text(_key_up_msg())

+            # Only the second chunk should be passed to STT

+            mock_stt.assert_called_once_with(b"\xBB" * 30)

+def test_key_up_without_key_down_is_ignored() -> None:

+    """key_up in idle state should be ignored, connection alive."""

+            mock_stt.assert_not_called()

+            # Connection should still be alive

+            ws.send_text(_wake_word_msg())

+            resp = json.loads(ws.receive_text())

+            assert resp["status"] == "listening"

-    """Binary audio should trigger STT → LLM → TTS pipeline.

+    """key_down → binary → key_up should trigger STT → LLM → TTS pipeline.

-            return_value=b"\xfc\xff\xfe",

+            ws.send_text(_key_down_msg())

+            ws.send_text(_key_up_msg())

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    client = TestClient(app)
    with (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    ):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"


Tests written. Now let me verify they fail (since `SessionState` and `MAX_AUDIO_BUFFER_BYTES` don't exist yet):


Bash

source venv/bin/activate && python -m pytest tests/ -v 2>&1 | head -60

Run tests to confirm they fail before implementation


============================= test session starts ==============================
platform linux -- Python 3.11.6, pytest-9.0.2, pluggy-1.6.0 -- /root/projects/openclaw-proxy/venv/bin/python
cachedir: .pytest_cache
rootdir: /root/projects/openclaw-proxy
plugins: anyio-4.12.1, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 0 items / 1 error

==================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_main.py ______________________
ImportError while importing test module '/root/projects/openclaw-proxy/tests/test_main.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_main.py:9: in <module>
    from main import (
E   ImportError: cannot import name 'SessionState' from 'main' (/root/projects/openclaw-proxy/main.py)
=========================== short test summary info ============================
ERROR tests/test_main.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 2.13s ===============================


Good — tests fail on import as expected. Now implement the state machine in `main.py`.


Updated task #2 status


Edit

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which
from typing import Literal


The file /root/projects/openclaw-proxy/main.py has been updated successfully.

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which
from typing import Literal

"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import edge_tts
import whisper
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

# ── Logging ──────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("openclaw-proxy")

# ── Configuration ────────────────────────────────────────────────────

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds
WHISPER_MODEL_SIZE = "base"
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"

# ── Pydantic models ─────────────────────────────────────────────────

class HardwareAction(str, Enum):
    """Actions reported by XiaoZhi ESP32 hardware."""

    KEY_DOWN = "key_down"
    KEY_UP = "key_up"
    WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
    """Incoming JSON frame from hardware."""

    type: str
    action: HardwareAction

class StateNotification(BaseModel):
    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""

    type: str = "state"
    status: str

class TextReply(BaseModel):
    """Outgoing text reply carrying the LLM response."""

    type: str = "text_reply"
    text: str

# ── Whisper model (lazy singleton) ─────────────────────────────────

_whisper_model: whisper.Whisper | None = None

def _get_whisper_model() -> whisper.Whisper:
    """Return the Whisper model, loading it on first call."""
    global _whisper_model
    if _whisper_model is None:
        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
    return _whisper_model

# ── STT / TTS pipeline ────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text via Whisper.

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    model = _get_whisper_model()
    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
        f.write(audio_bytes)
        f.flush()
        result = await asyncio.to_thread(
            model.transcribe, f.name, language="zh"
        )
    text = result["text"].strip()
    logger.info("STT: transcribed → %s", text)
    return text

async def call_openclaw(text: str, session_id: str) -> str:
    """Send text to OpenClaw agent via CLI and return its reply.

    Args:
        text: User utterance transcribed by STT.
        session_id: Per-connection session ID for multi-turn conversation.

    Returns:
        Agent reply string.

    Raises:
        RuntimeError: If the CLI call fails or returns unexpected output.
    """
    logger.info("LLM: sending to OpenClaw → %s", text)

    proc = await asyncio.create_subprocess_exec(
        OPENCLAW_CLI, "agent",
        "--agent", OPENCLAW_AGENT_ID,
        "--session-id", session_id,
        "-m", text,
        "--json",
        "--timeout", str(OPENCLAW_TIMEOUT),
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")

    data = json.loads(stdout.decode())
    payloads = data.get("result", {}).get("payloads", [])
    if not payloads:
        raise RuntimeError("OpenClaw returned empty payloads")

    reply = payloads[0].get("text", "")
    logger.info("LLM: reply ← %s", reply)
    return reply

async def process_tts(text: str) -> bytes:
    """Convert text to Opus audio via edge-tts and ffmpeg.

    Args:
        text: Text to synthesize.

    Returns:
        Opus-encoded audio bytes.

    Raises:
        RuntimeError: If ffmpeg conversion fails.
    """
    logger.info("TTS: synthesising → %s", text)

    # 1. edge-tts → MP3
    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
    mp3_chunks: list[bytes] = []
    async for chunk in communicate.stream():
        if chunk["type"] == "audio":
            mp3_chunks.append(chunk["data"])
    mp3_bytes = b"".join(mp3_chunks)

    # 2. MP3 → Opus via ffmpeg
    proc = await asyncio.create_subprocess_exec(
        FFMPEG_BIN, "-i", "pipe:0",
        "-c:a", "libopus", "-b:a", "24k",
        "-ar", "16000", "-ac", "1",
        "-f", "opus", "pipe:1",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    opus_bytes, stderr = await proc.communicate(input=mp3_bytes)

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("ffmpeg failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"ffmpeg TTS conversion error: {err_msg}")

    logger.info("TTS: generated %d bytes of Opus audio", len(opus_bytes))
    return opus_bytes

# ── Frame router ────────────────────────────────────────────────────

async def handle_text_frame(ws: WebSocket, raw: str) -> None:
    """Parse and route an incoming JSON text frame.

    Args:
        ws: Active WebSocket connection.
        raw: Raw text payload received from client.
    """
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        logger.warning("Malformed JSON (ignored): %.120s", raw)
        return

    try:
        event = HardwareEvent(**payload)
    except ValidationError as exc:
        logger.warning("Invalid hardware event: %s", exc.errors())
        return

    logger.info("Hardware event: %s", event.action.value)

    if event.action == HardwareAction.WAKE_WORD:
        ack = StateNotification(status="listening")
        await ws.send_text(ack.model_dump_json())

async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
    """Run the full STT → LLM → TTS pipeline on an audio binary frame.

    Args:
        ws: Active WebSocket connection.
        data: Raw Opus audio bytes.
        session_id: Per-connection session ID.

    Response sequence pushed to client:
        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
        2. {"type": "text_reply", "text": "..."}     — agent answer
        3. <binary>                                   — Opus audio of the answer
    """
    # Stage 1 — STT
    text = await process_stt(data)

    # Notify hardware: entering thinking phase
    await ws.send_text(StateNotification(status="thinking").model_dump_json())

    # Stage 2 — LLM (with periodic thinking heartbeats)
    async def heartbeat() -> None:
        """Send thinking heartbeat every 3 seconds while LLM is working."""
        while True:
            await asyncio.sleep(3)
            try:
                await ws.send_text(
                    StateNotification(status="thinking").model_dump_json()
                )
            except Exception:
                break

    heartbeat_task = asyncio.create_task(heartbeat())
    try:
        reply = await call_openclaw(text, session_id)
    finally:
        heartbeat_task.cancel()

    await ws.send_text(TextReply(text=reply).model_dump_json())

    # Stage 3 — TTS
    audio = await process_tts(reply)
    logger.info("Pushing %d bytes of audio to client", len(audio))
    await ws.send_bytes(audio)

# ── FastAPI application ─────────────────────────────────────────────

app = FastAPI(title="openclaw-proxy", version="0.1.0")

@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
    await ws.accept()
    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    logger.info("Client connected: %s (session=%s)", peer, session_id)

    try:
        while True:
            message = await ws.receive()

            if message["type"] == "websocket.disconnect":
                break

            text_data = message.get("text")
            byte_data = message.get("bytes")

            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect:
        pass
    finally:
        logger.info("Client disconnected: %s (session=%s)", peer, session_id)

@app.get("/health")
async def health() -> dict[str, str]:
    """Liveness probe."""
    return {"status": "ok"}

# ── Entry point ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

import uuid
from enum import Enum
from shutil import which
+from typing import Literal
import edge_tts
import whisper


EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"

EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"
MAX_AUDIO_BUFFER_BYTES: int = 10 * 1024 * 1024  # 10 MB