Error: The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file). STOP what you are doing and wait for the user to tell you how to proceed.


[Request interrupted by user for tool use]


Implement the following plan:

# Plan: Real STT + TTS Integration

## Context

openclaw-proxy Phase 0.5 已跑通：ESP32 ↔ WebSocket ↔ OpenClaw CLI 对话链路完整，但 STT 和 TTS 仍是 mock。
现在需要接入真实的 Whisper（STT）和 edge-tts（TTS），让语音输入→文字→语音输出全链路打通。

**环境确认（均已安装）：**
- openai-whisper 20250625 + PyTorch 2.10.0 (CPU)
- edge-tts 7.2.7
- ffmpeg 7.0.2 (libopus encoder/decoder)
- 4GB 可用内存，4 核 CPU，无 GPU

## 实现方案

### 1. STT — `process_stt(audio_bytes) -> str`

**链路：** Opus bytes → ffmpeg → 16kHz WAV → Whisper base 模型 → 文本

**实现：**
- 模块级加载 Whisper base 模型（一次加载，全局复用）
- 用 `asyncio.to_thread()` 包装 Whisper 推理（CPU 密集，不能阻塞事件循环）
- Whisper 的 `load_audio()` 内部会调 ffmpeg 处理任意格式，所以只需把 Opus 写入临时文件传路径即可
- 用 `tempfile.NamedTemporaryFile` 写 Opus 字节，传路径给 `model.transcribe()`

**关键代码骨架：**
```python
import whisper
_whisper_model = whisper.load_model("base")

async def process_stt(audio_bytes: bytes) -> str:
    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
        f.write(audio_bytes)
        f.flush()
        result = await asyncio.to_thread(
            _whisper_model.transcribe, f.name, language="zh"
        )
    return result["text"].strip()
```

**修改文件：** `main.py` 第 54-86 行（`process_stt` 函数）+ 模块顶部导入

### 2. TTS — `process_tts(text) -> bytes`

**链路：** 文本 → edge-tts (MP3) → ffmpeg → Opus bytes

**实现：**
- edge-tts 是异步的，用 `Communicate(text, voice).stream()` 获取 MP3 流
- 收集完整 MP3 字节后，用 ffmpeg subprocess 转 Opus：
  `ffmpeg -i pipe:0 -c:a libopus -b:a 24k -ar 16000 -ac 1 -f opus pipe:1`
- 返回完整 Opus 字节

**关键代码骨架：**
```python
import edge_tts
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"

async def process_tts(text: str) -> bytes:
    # 1. edge-tts → MP3
    communicate = edge_tts.Communicate(text=text, voice=EDGE_TTS_VOICE)
    mp3_chunks = []
    async for chunk in communicate.stream():
        if chunk["type"] == "audio":
            mp3_chunks.append(chunk["data"])
    mp3_bytes = b"".join(mp3_chunks)

    # 2. MP3 → Opus via ffmpeg
    proc = await asyncio.create_subprocess_exec(
        "ffmpeg", "-i", "pipe:0", "-c:a", "libopus",
        "-b:a", "24k", "-ar", "16000", "-ac", "1", "-f", "opus", "pipe:1",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    opus_bytes, _ = await proc.communicate(input=mp3_bytes)
    return opus_bytes
```

**修改文件：** `main.py` 第 100-122 行（`process_tts` 函数）+ 模块顶部导入

### 3. 配置常量

在 `main.py` 顶部 Configuration 区域新增：
```python
WHISPER_MODEL_SIZE = "base"
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"
```

### 4. requirements.txt 更新

新增：
```
openai-whisper>=20250625
edge-tts>=7.0
```
（系统已安装，写入 requirements.txt 声明依赖）

### 5. 测试更新

- `test_process_stt` — mock `whisper.load_model` 和 `model.transcribe`，验证 tempfile 和结果解析
- `test_process_tts` — mock `edge_tts.Communicate` 和 ffmpeg subprocess，验证 MP3→Opus 管线
- 保持现有 10 个测试不变（mock subprocess 的 call_openclaw 测试不受影响）

## 不改的东西

- `.openclaw/` 目录 — 不动
- `call_openclaw()` — 已经是真实 CLI 调用，不动
- WebSocket 协议 / 帧路由 / 心跳逻辑 — 不动

## 验证方式

```bash
# 1. 跑单元测试
source venv/bin/activate && python -m pytest tests/ -v

# 2. 启动服务
python main.py

# 3. 用 test_client.py 测试完整链路（mock Opus → real STT → real LLM → real TTS → Opus）
python test_client.py
```

If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /root/.claude/projects/-root-projects-openclaw-proxy/3cc6e826-1199-4350-9754-74df00ea29ab.jsonl


Let me start by understanding the current codebase structure, then implement the plan step by step. I need to:

1. Read the current `main.py` to understand the existing mock STT/TTS
2. Read the current tests
3. Read `requirements.txt`
4. Implement the changes following TDD (write tests first, then implement)

Let me start by reading the relevant files.


Let me start by reading the current codebase to understand what needs to change.


     1→"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""
     2→
     3→import asyncio
     4→import json
     5→import logging
     6→import uuid
     7→from enum import Enum
     8→from shutil import which
     9→
    10→from fastapi import FastAPI, WebSocket, WebSocketDisconnect
    11→from pydantic import BaseModel, ValidationError
    12→
    13→# ── Logging ──────────────────────────────────────────────────────────
    14→
    15→logging.basicConfig(
    16→    level=logging.INFO,
    17→    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    18→)
    19→logger = logging.getLogger("openclaw-proxy")
    20→
    21→# ── Configuration ────────────────────────────────────────────────────
    22→
    23→OPENCLAW_CLI = which("openclaw") or "openclaw"
    24→OPENCLAW_AGENT_ID = "main"
    25→OPENCLAW_TIMEOUT = 120  # seconds
    26→
    27→
    28→# ── Pydantic models ─────────────────────────────────────────────────
    29→
    30→
    31→class HardwareAction(str, Enum):
    32→    """Actions reported by XiaoZhi ESP32 hardware."""
    33→
    34→    KEY_DOWN = "key_down"
    35→    KEY_UP = "key_up"
    36→    WAKE_WORD = "wake_word"
    37→
    38→
    39→class HardwareEvent(BaseModel):
    40→    """Incoming JSON frame from hardware."""
    41→
    42→    type: str
    43→    action: HardwareAction
    44→
    45→
    46→class StateNotification(BaseModel):
    47→    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""
    48→
    49→    type: str = "state"
    50→    status: str
    51→
    52→
    53→class TextReply(BaseModel):
    54→    """Outgoing text reply carrying the LLM response."""
    55→
    56→    type: str = "text_reply"
    57→    text: str
    58→
    59→
    60→# ── Mock pipeline ───────────────────────────────────────────────────
    61→
    62→
    63→async def process_stt(audio_bytes: bytes) -> str:
    64→    """Convert Opus audio to text (mock).
    65→
    66→    Args:
    67→        audio_bytes: Raw Opus-encoded audio from ESP32.
    68→
    69→    Returns:
    70→        Transcribed text string.
    71→
    72→    TODO (real implementation):
    73→        1. Decode Opus → WAV with ffmpeg-python:
    74→             import ffmpeg
    75→             wav, _ = (ffmpeg
    76→                 .input("pipe:", format="opus")
    77→                 .output("pipe:", format="wav", ar=16000, ac=1)
    78→                 .run(input=audio_bytes, capture_stdout=True, capture_stderr=True))
    79→        2. Or use pydub:
    80→             from pydub import AudioSegment
    81→             seg = AudioSegment.from_file(io.BytesIO(audio_bytes), codec="opus")
    82→             wav = seg.set_frame_rate(16000).set_channels(1).raw_data
    83→        3. Feed WAV bytes to STT (Whisper / FunASR / SenseVoice).
    84→    """
    85→    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    86→    return "测试语音输入"
    87→
    88→
    89→async def call_openclaw(text: str, session_id: str) -> str:
    90→    """Send text to OpenClaw agent via CLI and return its reply.
    91→
    92→    Args:
    93→        text: User utterance transcribed by STT.
    94→        session_id: Per-connection session ID for multi-turn conversation.
    95→
    96→    Returns:
    97→        Agent reply string.
    98→
    99→    Raises:
   100→        RuntimeError: If the CLI call fails or returns unexpected output.
   101→    """
   102→    logger.info("LLM: sending to OpenClaw → %s", text)
   103→
   104→    proc = await asyncio.create_subprocess_exec(
   105→        OPENCLAW_CLI, "agent",
   106→        "--agent", OPENCLAW_AGENT_ID,
   107→        "--session-id", session_id,
   108→        "-m", text,
   109→        "--json",
   110→        "--timeout", str(OPENCLAW_TIMEOUT),
   111→        stdout=asyncio.subprocess.PIPE,
   112→        stderr=asyncio.subprocess.PIPE,
   113→    )
   114→    stdout, stderr = await proc.communicate()
   115→
   116→    if proc.returncode != 0:
   117→        err_msg = stderr.decode(errors="replace").strip()
   118→        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
   119→        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")
   120→
   121→    data = json.loads(stdout.decode())
   122→    payloads = data.get("result", {}).get("payloads", [])
   123→    if not payloads:
   124→        raise RuntimeError("OpenClaw returned empty payloads")
   125→
   126→    reply = payloads[0].get("text", "")
   127→    logger.info("LLM: reply ← %s", reply)
   128→    return reply
   129→
   130→
   131→async def process_tts(text: str) -> bytes:
   132→    """Convert text to Opus audio (mock).
   133→
   134→    Args:
   135→        text: Text to synthesize.
   136→
   137→    Returns:
   138→        Opus-encoded audio bytes.
   139→
   140→    TODO (real implementation):
   141→        1. Call TTS engine (edge-tts / CosyVoice / GPT-SoVITS).
   142→        2. Encode output to Opus:
   143→             import ffmpeg
   144→             opus, _ = (ffmpeg
   145→                 .input("pipe:", format="wav")
   146→                 .output("pipe:", format="opus", ar=16000, ac=1)
   147→                 .run(input=wav_bytes, capture_stdout=True))
   148→        3. Stream Opus frames back to ESP32 in ≤960-byte chunks.
   149→    """
   150→    logger.info("TTS: synthesising → %s", text)
   151→    mock_opus = b"\xfc\xff\xfe" * 53  # 159 bytes, resembles Opus silence
   152→    logger.info("TTS: generated %d bytes of audio", len(mock_opus))
   153→    return mock_opus
   154→
   155→
   156→# ── Frame router ────────────────────────────────────────────────────
   157→
   158→
   159→async def handle_text_frame(ws: WebSocket, raw: str) -> None:
   160→    """Parse and route an incoming JSON text frame.
   161→
   162→    Args:
   163→        ws: Active WebSocket connection.
   164→        raw: Raw text payload received from client.
   165→    """
   166→    try:
   167→        payload = json.loads(raw)
   168→    except json.JSONDecodeError:
   169→        logger.warning("Malformed JSON (ignored): %.120s", raw)
   170→        return
   171→
   172→    try:
   173→        event = HardwareEvent(**payload)
   174→    except ValidationError as exc:
   175→        logger.warning("Invalid hardware event: %s", exc.errors())
   176→        return
   177→
   178→    logger.info("Hardware event: %s", event.action.value)
   179→
   180→    if event.action == HardwareAction.WAKE_WORD:
   181→        ack = StateNotification(status="listening")
   182→        await ws.send_text(ack.model_dump_json())
   183→
   184→
   185→async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
   186→    """Run the full STT → LLM → TTS pipeline on an audio binary frame.
   187→
   188→    Args:
   189→        ws: Active WebSocket connection.
   190→        data: Raw Opus audio bytes.
   191→        session_id: Per-connection session ID.
   192→
   193→    Response sequence pushed to client:
   194→        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
   195→        2. {"type": "text_reply", "text": "..."}     — agent answer
   196→        3. <binary>                                   — Opus audio of the answer
   197→    """
   198→    # Stage 1 — STT
   199→    text = await process_stt(data)
   200→
   201→    # Notify hardware: entering thinking phase
   202→    await ws.send_text(StateNotification(status="thinking").model_dump_json())
   203→
   204→    # Stage 2 — LLM (with periodic thinking heartbeats)
   205→    async def heartbeat() -> None:
   206→        """Send thinking heartbeat every 3 seconds while LLM is working."""
   207→        while True:
   208→            await asyncio.sleep(3)
   209→            try:
   210→                await ws.send_text(
   211→                    StateNotification(status="thinking").model_dump_json()
   212→                )
   213→            except Exception:
   214→                break
   215→
   216→    heartbeat_task = asyncio.create_task(heartbeat())
   217→    try:
   218→        reply = await call_openclaw(text, session_id)
   219→    finally:
   220→        heartbeat_task.cancel()
   221→
   222→    await ws.send_text(TextReply(text=reply).model_dump_json())
   223→
   224→    # Stage 3 — TTS
   225→    audio = await process_tts(reply)
   226→    logger.info("Pushing %d bytes of audio to client", len(audio))
   227→    await ws.send_bytes(audio)
   228→
   229→
   230→# ── FastAPI application ─────────────────────────────────────────────
   231→
   232→app = FastAPI(title="openclaw-proxy", version="0.1.0")
   233→
   234→
   235→@app.websocket("/chat")
   236→async def websocket_chat(ws: WebSocket) -> None:
   237→    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
   238→    await ws.accept()
   239→    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
   240→    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
   241→    logger.info("Client connected: %s (session=%s)", peer, session_id)
   242→
   243→    try:
   244→        while True:
   245→            message = await ws.receive()
   246→
   247→            if message["type"] == "websocket.disconnect":
   248→                break
   249→
   250→            text_data = message.get("text")
   251→            byte_data = message.get("bytes")
   252→
   253→            if text_data is not None:
   254→                await handle_text_frame(ws, text_data)
   255→            elif byte_data is not None:
   256→                await handle_binary_frame(ws, byte_data, session_id)
   257→    except WebSocketDisconnect:
   258→        pass
   259→    finally:
   260→        logger.info("Client disconnected: %s (session=%s)", peer, session_id)
   261→
   262→
   263→@app.get("/health")
   264→async def health() -> dict[str, str]:
   265→    """Liveness probe."""
   266→    return {"status": "ok"}
   267→
   268→
   269→# ── Entry point ─────────────────────────────────────────────────────
   270→
   271→if __name__ == "__main__":
   272→    import uvicorn
   273→
   274→    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)
   275→

"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""

import asyncio
import json
import logging
import uuid
from enum import Enum
from shutil import which

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

# ── Logging ──────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("openclaw-proxy")

# ── Configuration ────────────────────────────────────────────────────

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds

# ── Pydantic models ─────────────────────────────────────────────────

class HardwareAction(str, Enum):
    """Actions reported by XiaoZhi ESP32 hardware."""

    KEY_DOWN = "key_down"
    KEY_UP = "key_up"
    WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
    """Incoming JSON frame from hardware."""

    type: str
    action: HardwareAction

class StateNotification(BaseModel):
    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""

    type: str = "state"
    status: str

class TextReply(BaseModel):
    """Outgoing text reply carrying the LLM response."""

    type: str = "text_reply"
    text: str

# ── Mock pipeline ───────────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text (mock).

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.

    TODO (real implementation):
        1. Decode Opus → WAV with ffmpeg-python:
             import ffmpeg
             wav, _ = (ffmpeg
                 .input("pipe:", format="opus")
                 .output("pipe:", format="wav", ar=16000, ac=1)
                 .run(input=audio_bytes, capture_stdout=True, capture_stderr=True))
        2. Or use pydub:
             from pydub import AudioSegment
             seg = AudioSegment.from_file(io.BytesIO(audio_bytes), codec="opus")
             wav = seg.set_frame_rate(16000).set_channels(1).raw_data
        3. Feed WAV bytes to STT (Whisper / FunASR / SenseVoice).
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    return "测试语音输入"

async def call_openclaw(text: str, session_id: str) -> str:
    """Send text to OpenClaw agent via CLI and return its reply.

    Args:
        text: User utterance transcribed by STT.
        session_id: Per-connection session ID for multi-turn conversation.

    Returns:
        Agent reply string.

    Raises:
        RuntimeError: If the CLI call fails or returns unexpected output.
    """
    logger.info("LLM: sending to OpenClaw → %s", text)

    proc = await asyncio.create_subprocess_exec(
        OPENCLAW_CLI, "agent",
        "--agent", OPENCLAW_AGENT_ID,
        "--session-id", session_id,
        "-m", text,
        "--json",
        "--timeout", str(OPENCLAW_TIMEOUT),
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")

    data = json.loads(stdout.decode())
    payloads = data.get("result", {}).get("payloads", [])
    if not payloads:
        raise RuntimeError("OpenClaw returned empty payloads")

    reply = payloads[0].get("text", "")
    logger.info("LLM: reply ← %s", reply)
    return reply

async def process_tts(text: str) -> bytes:
    """Convert text to Opus audio (mock).

    Args:
        text: Text to synthesize.

    Returns:
        Opus-encoded audio bytes.

    TODO (real implementation):
        1. Call TTS engine (edge-tts / CosyVoice / GPT-SoVITS).
        2. Encode output to Opus:
             import ffmpeg
             opus, _ = (ffmpeg
                 .input("pipe:", format="wav")
                 .output("pipe:", format="opus", ar=16000, ac=1)
                 .run(input=wav_bytes, capture_stdout=True))
        3. Stream Opus frames back to ESP32 in ≤960-byte chunks.
    """
    logger.info("TTS: synthesising → %s", text)
    mock_opus = b"\xfc\xff\xfe" * 53  # 159 bytes, resembles Opus silence
    logger.info("TTS: generated %d bytes of audio", len(mock_opus))
    return mock_opus

# ── Frame router ────────────────────────────────────────────────────

async def handle_text_frame(ws: WebSocket, raw: str) -> None:
    """Parse and route an incoming JSON text frame.

    Args:
        ws: Active WebSocket connection.
        raw: Raw text payload received from client.
    """
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        logger.warning("Malformed JSON (ignored): %.120s", raw)
        return

    try:
        event = HardwareEvent(**payload)
    except ValidationError as exc:
        logger.warning("Invalid hardware event: %s", exc.errors())
        return

    logger.info("Hardware event: %s", event.action.value)

    if event.action == HardwareAction.WAKE_WORD:
        ack = StateNotification(status="listening")
        await ws.send_text(ack.model_dump_json())

async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
    """Run the full STT → LLM → TTS pipeline on an audio binary frame.

    Args:
        ws: Active WebSocket connection.
        data: Raw Opus audio bytes.
        session_id: Per-connection session ID.

    Response sequence pushed to client:
        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
        2. {"type": "text_reply", "text": "..."}     — agent answer
        3. <binary>                                   — Opus audio of the answer
    """
    # Stage 1 — STT
    text = await process_stt(data)

    # Notify hardware: entering thinking phase
    await ws.send_text(StateNotification(status="thinking").model_dump_json())

    # Stage 2 — LLM (with periodic thinking heartbeats)
    async def heartbeat() -> None:
        """Send thinking heartbeat every 3 seconds while LLM is working."""
        while True:
            await asyncio.sleep(3)
            try:
                await ws.send_text(
                    StateNotification(status="thinking").model_dump_json()
                )
            except Exception:
                break

    heartbeat_task = asyncio.create_task(heartbeat())
    try:
        reply = await call_openclaw(text, session_id)
    finally:
        heartbeat_task.cancel()

    await ws.send_text(TextReply(text=reply).model_dump_json())

    # Stage 3 — TTS
    audio = await process_tts(reply)
    logger.info("Pushing %d bytes of audio to client", len(audio))
    await ws.send_bytes(audio)

# ── FastAPI application ─────────────────────────────────────────────

app = FastAPI(title="openclaw-proxy", version="0.1.0")

@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
    await ws.accept()
    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    logger.info("Client connected: %s (session=%s)", peer, session_id)

    try:
        while True:
            message = await ws.receive()

            if message["type"] == "websocket.disconnect":
                break

            text_data = message.get("text")
            byte_data = message.get("bytes")

            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect:
        pass
    finally:
        logger.info("Client disconnected: %s (session=%s)", peer, session_id)

@app.get("/health")
async def health() -> dict[str, str]:
    """Liveness probe."""
    return {"status": "ok"}

# ── Entry point ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)


     1→fastapi>=0.115.0
     2→uvicorn[standard]>=0.34.0
     3→websockets>=14.0
     4→pydantic>=2.0
     5→pytest>=8.0
     6→httpx>=0.28.0
     7→pytest-asyncio>=0.25.0
     8→

fastapi>=0.115.0
uvicorn[standard]>=0.34.0
websockets>=14.0
pydantic>=2.0
pytest>=8.0
httpx>=0.28.0
pytest-asyncio>=0.25.0


/root/projects/openclaw-proxy/tests/__init__.py
/root/projects/openclaw-proxy/tests/test_main.py


Error: File does not exist. Note: your current working directory is /root/projects/openclaw-proxy.


     1→"""Tests for openclaw-proxy WebSocket middleware."""
     2→
     3→import json
     4→from unittest.mock import AsyncMock, patch
     5→
     6→import pytest
     7→from starlette.testclient import TestClient
     8→
     9→from main import app, process_stt, call_openclaw, process_tts
    10→
    11→
    12→# ── Helpers ──────────────────────────────────────────────────────────
    13→
    14→MOCK_OPENCLAW_OUTPUT = json.dumps({
    15→    "status": "ok",
    16→    "result": {
    17→        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    18→    },
    19→}).encode()
    20→
    21→
    22→def _make_mock_process(
    23→    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    24→    returncode: int = 0,
    25→) -> AsyncMock:
    26→    """Build a mock asyncio.Process with preset stdout/stderr."""
    27→    proc = AsyncMock()
    28→    proc.communicate.return_value = (stdout, b"")
    29→    proc.returncode = returncode
    30→    return proc
    31→
    32→
    33→# ── Unit tests: pipeline functions ───────────────────────────────────
    34→
    35→
    36→@pytest.mark.asyncio
    37→async def test_process_stt_returns_mock_text() -> None:
    38→    """STT should return fixed mock string for any input bytes."""
    39→    result = await process_stt(b"\x00" * 80)
    40→    assert result == "测试语音输入"
    41→
    42→
    43→@pytest.mark.asyncio
    44→async def test_call_openclaw_parses_cli_output() -> None:
    45→    """call_openclaw should parse the CLI JSON and return reply text."""
    46→    mock_proc = _make_mock_process()
    47→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    48→        result = await call_openclaw("你好", "test-session")
    49→    assert result == "你好！我在这儿。"
    50→
    51→
    52→@pytest.mark.asyncio
    53→async def test_call_openclaw_raises_on_cli_failure() -> None:
    54→    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    55→    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    56→    mock_proc.communicate.return_value = (b"", b"some error")
    57→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    58→        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
    59→            await call_openclaw("你好", "test-session")
    60→
    61→
    62→@pytest.mark.asyncio
    63→async def test_call_openclaw_raises_on_empty_payloads() -> None:
    64→    """call_openclaw should raise RuntimeError on empty payloads."""
    65→    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    66→    mock_proc = _make_mock_process(stdout=empty)
    67→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
    68→        with pytest.raises(RuntimeError, match="empty payloads"):
    69→            await call_openclaw("你好", "test-session")
    70→
    71→
    72→@pytest.mark.asyncio
    73→async def test_process_tts_returns_bytes() -> None:
    74→    """TTS mock should return non-empty bytes."""
    75→    result = await process_tts("测试")
    76→    assert isinstance(result, bytes)
    77→    assert len(result) > 0
    78→
    79→
    80→# ── Integration tests: HTTP & WebSocket ──────────────────────────────
    81→
    82→
    83→def test_health_endpoint() -> None:
    84→    """GET /health should return 200 with ok status."""
    85→    client = TestClient(app)
    86→    resp = client.get("/health")
    87→    assert resp.status_code == 200
    88→    assert resp.json() == {"status": "ok"}
    89→
    90→
    91→def test_websocket_connect_and_disconnect() -> None:
    92→    """Client should be able to connect to /chat and cleanly disconnect."""
    93→    client = TestClient(app)
    94→    with client.websocket_connect("/chat"):
    95→        pass
    96→
    97→
    98→def test_websocket_text_frame_wake_word() -> None:
    99→    """Sending a wake_word JSON should get back a 'listening' state."""
   100→    client = TestClient(app)
   101→    with client.websocket_connect("/chat") as ws:
   102→        ws.send_text(json.dumps({
   103→            "type": "hardware_event",
   104→            "action": "wake_word",
   105→        }))
   106→        resp = json.loads(ws.receive_text())
   107→        assert resp["type"] == "state"
   108→        assert resp["status"] == "listening"
   109→
   110→
   111→def test_websocket_text_frame_malformed_json() -> None:
   112→    """Malformed JSON should not crash the connection."""
   113→    client = TestClient(app)
   114→    with client.websocket_connect("/chat") as ws:
   115→        ws.send_text("not-valid-json{{{")
   116→        ws.send_text(json.dumps({
   117→            "type": "hardware_event",
   118→            "action": "wake_word",
   119→        }))
   120→        resp = json.loads(ws.receive_text())
   121→        assert resp["status"] == "listening"
   122→
   123→
   124→def test_websocket_binary_frame_full_pipeline() -> None:
   125→    """Binary audio should trigger STT → LLM → TTS pipeline.
   126→
   127→    Expected response sequence:
   128→      1. {"type": "state", "status": "thinking"}
   129→      2. {"type": "text_reply", "text": "..."}
   130→      3. binary bytes (TTS audio)
   131→    """
   132→    mock_proc = _make_mock_process()
   133→    client = TestClient(app)
   134→    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
   135→        with client.websocket_connect("/chat") as ws:
   136→            ws.send_bytes(b"\x00" * 80)
   137→
   138→            # 1) thinking state
   139→            msg1 = json.loads(ws.receive_text())
   140→            assert msg1 == {"type": "state", "status": "thinking"}
   141→
   142→            # 2) text reply
   143→            msg2 = json.loads(ws.receive_text())
   144→            assert msg2["type"] == "text_reply"
   145→            assert msg2["text"] == "你好！我在这儿。"
   146→
   147→            # 3) TTS audio bytes
   148→            audio = ws.receive_bytes()
   149→            assert isinstance(audio, bytes)
   150→            assert len(audio) > 0
   151→

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_returns_mock_text() -> None:
    """STT should return fixed mock string for any input bytes."""
    result = await process_stt(b"\x00" * 80)
    assert result == "测试语音输入"

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_bytes() -> None:
    """TTS mock should return non-empty bytes."""
    result = await process_tts("测试")
    assert isinstance(result, bytes)
    assert len(result) > 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0


Now I have a clear picture. Let me create the task list and implement the changes following TDD:

1. Update tests first (TDD) - replace mock STT/TTS tests with real-implementation mocked tests
2. Update `main.py` - replace mock `process_stt` and `process_tts` with real implementations
3. Update `requirements.txt`

Let me plan the implementation:

### Tests to update:
- `test_process_stt_returns_mock_text` → mock whisper model, verify tempfile + transcribe call
- `test_process_tts_returns_bytes` → mock edge_tts + ffmpeg subprocess, verify pipeline

- `test_websocket_binary_frame_full_pipeline` → needs to mock both whisper and edge_tts now

### main.py changes:
- Add imports: `whisper`, `edge_tts`, `tempfile`
- Add config constants: `WHISPER_MODEL_SIZE`, `EDGE_TTS_VOICE`, `FFMPEG_BIN`
- Load whisper model at module level
- Replace `process_stt` with real implementation
- Replace `process_tts` with real implementation

I need to be careful about the whisper model loading at module level since that's a heavy operation, so I should think through the performance implications there.

The issue is that importing main in tests will trigger the model load. I could either mock the load before importing, or switch to lazy loading where the model only gets initialized when actually needed. Lazy loading seems cleaner for testing—I'd wrap the model in a getter function that initializes it on first call.

Implementing lazy loading pattern... Writing test mocks... Writing test mocks... I need to write the tests first, then implement. I'm also checking whether `asyncio.to_thread` will work properly in the test context with mocked models, and verifying that the `NamedTemporaryFile` stays open long enough for the thread to access it by name since the `await asyncio.to_thread` call happens within the context manager. The file stays open during the transcription since we're still inside the context manager, so that works out. Now I need to implement everything using test-driven development, starting with the tests. For the integration test, I realize the current mock on `asyncio.create_subprocess_exec` would catch both the subprocess call and the ffmpeg conversion, so instead I'll mock the three pipeline functions directly—`process_stt`, `call_openclaw`, and `process_tts`—to return fixed values and keep the test focused on integration logic rather than implementation details.

Now I'm working through how to mock `edge_tts.Communicate` since it needs to return an object with an async iterator method...

I'll create a mock class that implements the `.stream()` method as an async generator, then patch it in the test. Let me start implementing the test cases. Setting up a lazy loader for the Whisper model so it only initializes once when first needed, then replacing the STT processing function to write incoming audio to a temporary file...

Now I'm implementing the TTS function using edge-tts to convert text to MP3 audio...

Converting MP3 to Opus format via ffmpeg... Writing test mocks... Setting up the mock subprocess to return fake Opus bytes and verifying that ffmpeg receives the concatenated MP3 data from the stream. Now moving to the full pipeline test to check the complete WebSocket flow. Testing the assertions...

Now I'm thinking about what other test cases would be valuable—I should add coverage for error scenarios like when ffmpeg fails, and I want to verify that whitespace stripping actually works as expected since the implementation calls `strip()` on the transcription result. The existing test already has spaces around the text that should be trimmed, so that should catch it. I could test the lazy loading of the Whisper model, but that might be unnecessary complexity for now.

One thing I need to be careful about: `asyncio.to_thread` runs synchronous functions in a thread pool and requires an active event loop, which pytest-asyncio provides. Since my mock `model.transcribe` is a regular callable (not async), it should work fine with `to_thread`. I'm just making sure the arguments passed to the mock match what the actual implementation expects. I'm refining the assertion logic to check that the temp file path was passed as a positional argument and the language parameter was set correctly, knowing that the string value persists in the mock's call args even after the file is deleted. Now I need to verify the imports and type hints work properly, particularly around the whisper module dependency. I'm being cautious about error handling, but the plan doesn't specify these edge cases and the guidelines say to avoid over-engineering. I'll include the ffmpeg return code check since it's a boundary with an external process, similar to existing patterns. Now I need to write the tests first, then update main.py, though I'm reconsidering how to mock the subprocess calls in the integration test to avoid affecting all three pipeline stages.

Setting up mocks at the function level... I need to be careful about how the imports work—when `process_stt` is imported directly, it still resolves `_get_whisper_model` through the main module's namespace at runtime, so mocking it at the module level should work correctly. Now I'm laying out the implementation sequence: update the tests first, then main.py, then requirements.txt, and finally run everything. I'm also thinking through a potential race condition with the temporary file in the STT function—the file gets created and opened in the context manager, then passed to the transcription model running in a thread, which should be safe on Linux since the file descriptor stays open throughout. I'm confirming that `asyncio.to_thread` will execute `model.transcribe(f.name, language="zh")` in a separate thread, which is the right approach here. I'm going to deviate slightly from the original plan by using lazy loading instead of module-level loading for the model—it's better for testability and achieves the same result of loading once and reusing globally. Now I'll start writing the implementation, though I need to handle the type ignore annotations for Whisper's type hints.

Now I have a clear picture of the codebase. Let me create tasks and implement TDD-style.

TaskCreate

Write failing tests for real STT + TTS

Update tests/test_main.py: replace mock STT/TTS tests with tests that mock whisper model + edge_tts + ffmpeg subprocess. Add test for ffmpeg failure in TTS. Update full pipeline integration test to mock at function level.


Task #1 created successfully: Write failing tests for real STT + TTS

Write failing tests for real STT + TTS


Replace mock process_stt and process_tts in main.py with real implementations using whisper and edge-tts. Add lazy whisper model loading, config constants, and imports.

Implement real STT + TTS in main.py


Task #2 created successfully: Implement real STT + TTS in main.py

Implement real STT + TTS in main.py


Add openai-whisper>=20250625 and edge-tts>=7.0 to requirements.txt

Update requirements.txt


Task #3 created successfully: Update requirements.txt

Update requirements.txt


Run pytest to verify all tests pass, including old ones that should remain unchanged.

Run tests and verify


Task #4 created successfully: Run tests and verify

Run tests and verify


Updated task #2 blockedBy


Updated task #4 blockedBy


## Task 1: Write failing tests first (TDD)


Updated task #1 status


Edit

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts


The file /root/projects/openclaw-proxy/tests/test_main.py has been updated successfully.

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_returns_mock_text() -> None:
    """STT should return fixed mock string for any input bytes."""
    result = await process_stt(b"\x00" * 80)
    assert result == "测试语音输入"

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_bytes() -> None:
    """TTS mock should return non-empty bytes."""
    result = await process_tts("测试")
    assert isinstance(result, bytes)
    assert len(result) > 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0

-from unittest.mock import AsyncMock, patch

+from unittest.mock import AsyncMock, MagicMock, patch


Now replace the STT unit test:


@pytest.mark.asyncio
async def test_process_stt_returns_mock_text() -> None:
    """STT should return fixed mock string for any input bytes."""
    result = await process_stt(b"\x00" * 80)
    assert result == "测试语音输入"

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}


@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_returns_mock_text() -> None:
    """STT should return fixed mock string for any input bytes."""
    result = await process_stt(b"\x00" * 80)
    assert result == "测试语音输入"

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_bytes() -> None:
    """TTS mock should return non-empty bytes."""
    result = await process_tts("测试")
    assert isinstance(result, bytes)
    assert len(result) > 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0

-async def test_process_stt_returns_mock_text() -> None:

-    """STT should return fixed mock string for any input bytes."""

-    result = await process_stt(b"\x00" * 80)

+async def test_process_stt_calls_whisper() -> None:

+    """STT should write audio to temp file, call Whisper, and return stripped text."""

+    mock_model.transcribe.return_value = {"text": " 你好世界 "}

+    with patch("main._get_whisper_model", return_value=mock_model):

+        result = await process_stt(b"\x00" * 80)

+    mock_model.transcribe.assert_called_once()

+    args, kwargs = mock_model.transcribe.call_args

+    assert kwargs == {"language": "zh"}


Now replace the TTS unit test and add an error case:


@pytest.mark.asyncio
async def test_process_tts_returns_bytes() -> None:
    """TTS mock should return non-empty bytes."""
    result = await process_tts("测试")
    assert isinstance(result, bytes)
    assert len(result) > 0

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")


@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_bytes() -> None:
    """TTS mock should return non-empty bytes."""
    result = await process_tts("测试")
    assert isinstance(result, bytes)
    assert len(result) > 0

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0

-async def test_process_tts_returns_bytes() -> None:

-    """TTS mock should return non-empty bytes."""

+async def test_process_tts_returns_opus_bytes() -> None:

+    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

+    async def mock_stream() -> None:  # type: ignore[override]

+        yield {"type": "audio", "data": b"mp3-part1"}

+        yield {"type": "audio", "data": b"mp3-part2"}

+        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

+    mock_proc.communicate.return_value = (b"fake-opus", b"")

+        patch("main.edge_tts.Communicate", return_value=mock_comm),

+        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),

+        result = await process_tts("测试文本")

+    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

+async def test_process_tts_raises_on_ffmpeg_failure() -> None:

+    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

+        yield {"type": "audio", "data": b"mp3-data"}

+    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")

+        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):


Now update the full pipeline integration test to mock at function level:


def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    client = TestClient(app)
    with (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    ):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"


def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    client = TestClient(app)
    with (
        patch(
            "main.process_stt",
            new_callable=AsyncMock,
            return_value="测试语音输入",
        ),
        patch(
            "main.call_openclaw",
            new_callable=AsyncMock,
            return_value="你好！我在这儿。",
        ),
        patch(
            "main.process_tts",
            new_callable=AsyncMock,
            return_value=b"\xfc\xff\xfe",
        ),
    ):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert audio == b"\xfc\xff\xfe"

"""Tests for openclaw-proxy WebSocket middleware."""

import json
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from starlette.testclient import TestClient

from main import app, process_stt, call_openclaw, process_tts

# ── Helpers ──────────────────────────────────────────────────────────

MOCK_OPENCLAW_OUTPUT = json.dumps({
    "status": "ok",
    "result": {
        "payloads": [{"text": "你好！我在这儿。", "mediaUrl": None}],
    },
}).encode()

def _make_mock_process(
    stdout: bytes = MOCK_OPENCLAW_OUTPUT,
    returncode: int = 0,
) -> AsyncMock:
    """Build a mock asyncio.Process with preset stdout/stderr."""
    proc = AsyncMock()
    proc.communicate.return_value = (stdout, b"")
    proc.returncode = returncode
    return proc

# ── Unit tests: pipeline functions ───────────────────────────────────

@pytest.mark.asyncio
async def test_process_stt_calls_whisper() -> None:
    """STT should write audio to temp file, call Whisper, and return stripped text."""
    mock_model = MagicMock()
    mock_model.transcribe.return_value = {"text": " 你好世界 "}
    with patch("main._get_whisper_model", return_value=mock_model):
        result = await process_stt(b"\x00" * 80)
    assert result == "你好世界"
    mock_model.transcribe.assert_called_once()
    args, kwargs = mock_model.transcribe.call_args
    assert args[0].endswith(".opus")
    assert kwargs == {"language": "zh"}

@pytest.mark.asyncio
async def test_call_openclaw_parses_cli_output() -> None:
    """call_openclaw should parse the CLI JSON and return reply text."""
    mock_proc = _make_mock_process()
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        result = await call_openclaw("你好", "test-session")
    assert result == "你好！我在这儿。"

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_cli_failure() -> None:
    """call_openclaw should raise RuntimeError when CLI exits non-zero."""
    mock_proc = _make_mock_process(stdout=b"", returncode=1)
    mock_proc.communicate.return_value = (b"", b"some error")
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="OpenClaw CLI error"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_call_openclaw_raises_on_empty_payloads() -> None:
    """call_openclaw should raise RuntimeError on empty payloads."""
    empty = json.dumps({"status": "ok", "result": {"payloads": []}}).encode()
    mock_proc = _make_mock_process(stdout=empty)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with pytest.raises(RuntimeError, match="empty payloads"):
            await call_openclaw("你好", "test-session")

@pytest.mark.asyncio
async def test_process_tts_returns_opus_bytes() -> None:
    """TTS should collect MP3 from edge-tts, convert to Opus via ffmpeg."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-part1"}
        yield {"type": "audio", "data": b"mp3-part2"}
        yield {"type": "WordBoundary", "data": None}  # non-audio chunk

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"fake-opus", b"")
    mock_proc.returncode = 0

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        result = await process_tts("测试文本")

    assert result == b"fake-opus"
    mock_proc.communicate.assert_called_once_with(input=b"mp3-part1mp3-part2")

@pytest.mark.asyncio
async def test_process_tts_raises_on_ffmpeg_failure() -> None:
    """TTS should raise RuntimeError when ffmpeg exits non-zero."""

    async def mock_stream() -> None:  # type: ignore[override]
        yield {"type": "audio", "data": b"mp3-data"}

    mock_comm = MagicMock()
    mock_comm.stream = mock_stream

    mock_proc = AsyncMock()
    mock_proc.communicate.return_value = (b"", b"ffmpeg error msg")
    mock_proc.returncode = 1

    with (
        patch("main.edge_tts.Communicate", return_value=mock_comm),
        patch("main.asyncio.create_subprocess_exec", return_value=mock_proc),
    ):
        with pytest.raises(RuntimeError, match="ffmpeg TTS conversion error"):
            await process_tts("测试文本")

# ── Integration tests: HTTP & WebSocket ──────────────────────────────

def test_health_endpoint() -> None:
    """GET /health should return 200 with ok status."""
    client = TestClient(app)
    resp = client.get("/health")
    assert resp.status_code == 200
    assert resp.json() == {"status": "ok"}

def test_websocket_connect_and_disconnect() -> None:
    """Client should be able to connect to /chat and cleanly disconnect."""
    client = TestClient(app)
    with client.websocket_connect("/chat"):
        pass

def test_websocket_text_frame_wake_word() -> None:
    """Sending a wake_word JSON should get back a 'listening' state."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["type"] == "state"
        assert resp["status"] == "listening"

def test_websocket_text_frame_malformed_json() -> None:
    """Malformed JSON should not crash the connection."""
    client = TestClient(app)
    with client.websocket_connect("/chat") as ws:
        ws.send_text("not-valid-json{{{")
        ws.send_text(json.dumps({
            "type": "hardware_event",
            "action": "wake_word",
        }))
        resp = json.loads(ws.receive_text())
        assert resp["status"] == "listening"

def test_websocket_binary_frame_full_pipeline() -> None:
    """Binary audio should trigger STT → LLM → TTS pipeline.

    Expected response sequence:
      1. {"type": "state", "status": "thinking"}
      2. {"type": "text_reply", "text": "..."}
      3. binary bytes (TTS audio)
    """
    mock_proc = _make_mock_process()
    client = TestClient(app)
    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):
        with client.websocket_connect("/chat") as ws:
            ws.send_bytes(b"\x00" * 80)

            # 1) thinking state
            msg1 = json.loads(ws.receive_text())
            assert msg1 == {"type": "state", "status": "thinking"}

            # 2) text reply
            msg2 = json.loads(ws.receive_text())
            assert msg2["type"] == "text_reply"
            assert msg2["text"] == "你好！我在这儿。"

            # 3) TTS audio bytes
            audio = ws.receive_bytes()
            assert isinstance(audio, bytes)
            assert len(audio) > 0

-    with patch("main.asyncio.create_subprocess_exec", return_value=mock_proc):

+            return_value=b"\xfc\xff\xfe",

-            assert isinstance(audio, bytes)

+            assert audio == b"\xfc\xff\xfe"


## Task 2: Implement real STT + TTS in main.py


Updated task #2 status


Add imports at the top:


import asyncio
import json
import logging
import uuid
from enum import Enum
from shutil import which

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import edge_tts
import whisper
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError


The file /root/projects/openclaw-proxy/main.py has been updated successfully.

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import edge_tts
import whisper
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""

import asyncio
import json
import logging
import uuid
from enum import Enum
from shutil import which

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

# ── Logging ──────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("openclaw-proxy")

# ── Configuration ────────────────────────────────────────────────────

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds

# ── Pydantic models ─────────────────────────────────────────────────

class HardwareAction(str, Enum):
    """Actions reported by XiaoZhi ESP32 hardware."""

    KEY_DOWN = "key_down"
    KEY_UP = "key_up"
    WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
    """Incoming JSON frame from hardware."""

    type: str
    action: HardwareAction

class StateNotification(BaseModel):
    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""

    type: str = "state"
    status: str

class TextReply(BaseModel):
    """Outgoing text reply carrying the LLM response."""

    type: str = "text_reply"
    text: str

# ── Mock pipeline ───────────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text (mock).

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.

    TODO (real implementation):
        1. Decode Opus → WAV with ffmpeg-python:
             import ffmpeg
             wav, _ = (ffmpeg
                 .input("pipe:", format="opus")
                 .output("pipe:", format="wav", ar=16000, ac=1)
                 .run(input=audio_bytes, capture_stdout=True, capture_stderr=True))
        2. Or use pydub:
             from pydub import AudioSegment
             seg = AudioSegment.from_file(io.BytesIO(audio_bytes), codec="opus")
             wav = seg.set_frame_rate(16000).set_channels(1).raw_data
        3. Feed WAV bytes to STT (Whisper / FunASR / SenseVoice).
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    return "测试语音输入"

async def call_openclaw(text: str, session_id: str) -> str:
    """Send text to OpenClaw agent via CLI and return its reply.

    Args:
        text: User utterance transcribed by STT.
        session_id: Per-connection session ID for multi-turn conversation.

    Returns:
        Agent reply string.

    Raises:
        RuntimeError: If the CLI call fails or returns unexpected output.
    """
    logger.info("LLM: sending to OpenClaw → %s", text)

    proc = await asyncio.create_subprocess_exec(
        OPENCLAW_CLI, "agent",
        "--agent", OPENCLAW_AGENT_ID,
        "--session-id", session_id,
        "-m", text,
        "--json",
        "--timeout", str(OPENCLAW_TIMEOUT),
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")

    data = json.loads(stdout.decode())
    payloads = data.get("result", {}).get("payloads", [])
    if not payloads:
        raise RuntimeError("OpenClaw returned empty payloads")

    reply = payloads[0].get("text", "")
    logger.info("LLM: reply ← %s", reply)
    return reply

async def process_tts(text: str) -> bytes:
    """Convert text to Opus audio (mock).

    Args:
        text: Text to synthesize.

    Returns:
        Opus-encoded audio bytes.

    TODO (real implementation):
        1. Call TTS engine (edge-tts / CosyVoice / GPT-SoVITS).
        2. Encode output to Opus:
             import ffmpeg
             opus, _ = (ffmpeg
                 .input("pipe:", format="wav")
                 .output("pipe:", format="opus", ar=16000, ac=1)
                 .run(input=wav_bytes, capture_stdout=True))
        3. Stream Opus frames back to ESP32 in ≤960-byte chunks.
    """
    logger.info("TTS: synthesising → %s", text)
    mock_opus = b"\xfc\xff\xfe" * 53  # 159 bytes, resembles Opus silence
    logger.info("TTS: generated %d bytes of audio", len(mock_opus))
    return mock_opus

# ── Frame router ────────────────────────────────────────────────────

async def handle_text_frame(ws: WebSocket, raw: str) -> None:
    """Parse and route an incoming JSON text frame.

    Args:
        ws: Active WebSocket connection.
        raw: Raw text payload received from client.
    """
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        logger.warning("Malformed JSON (ignored): %.120s", raw)
        return

    try:
        event = HardwareEvent(**payload)
    except ValidationError as exc:
        logger.warning("Invalid hardware event: %s", exc.errors())
        return

    logger.info("Hardware event: %s", event.action.value)

    if event.action == HardwareAction.WAKE_WORD:
        ack = StateNotification(status="listening")
        await ws.send_text(ack.model_dump_json())

async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
    """Run the full STT → LLM → TTS pipeline on an audio binary frame.

    Args:
        ws: Active WebSocket connection.
        data: Raw Opus audio bytes.
        session_id: Per-connection session ID.

    Response sequence pushed to client:
        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
        2. {"type": "text_reply", "text": "..."}     — agent answer
        3. <binary>                                   — Opus audio of the answer
    """
    # Stage 1 — STT
    text = await process_stt(data)

    # Notify hardware: entering thinking phase
    await ws.send_text(StateNotification(status="thinking").model_dump_json())

    # Stage 2 — LLM (with periodic thinking heartbeats)
    async def heartbeat() -> None:
        """Send thinking heartbeat every 3 seconds while LLM is working."""
        while True:
            await asyncio.sleep(3)
            try:
                await ws.send_text(
                    StateNotification(status="thinking").model_dump_json()
                )
            except Exception:
                break

    heartbeat_task = asyncio.create_task(heartbeat())
    try:
        reply = await call_openclaw(text, session_id)
    finally:
        heartbeat_task.cancel()

    await ws.send_text(TextReply(text=reply).model_dump_json())

    # Stage 3 — TTS
    audio = await process_tts(reply)
    logger.info("Pushing %d bytes of audio to client", len(audio))
    await ws.send_bytes(audio)

# ── FastAPI application ─────────────────────────────────────────────

app = FastAPI(title="openclaw-proxy", version="0.1.0")

@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
    await ws.accept()
    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    logger.info("Client connected: %s (session=%s)", peer, session_id)

    try:
        while True:
            message = await ws.receive()

            if message["type"] == "websocket.disconnect":
                break

            text_data = message.get("text")
            byte_data = message.get("bytes")

            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect:
        pass
    finally:
        logger.info("Client disconnected: %s (session=%s)", peer, session_id)

@app.get("/health")
async def health() -> dict[str, str]:
    """Liveness probe."""
    return {"status": "ok"}

# ── Entry point ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)


Add configuration constants:


OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds
WHISPER_MODEL_SIZE = "base"
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"


OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds
WHISPER_MODEL_SIZE = "base"
EDGE_TTS_VOICE = "zh-CN-XiaoxiaoNeural"
FFMPEG_BIN = which("ffmpeg") or "ffmpeg"

"""openclaw-proxy — async WebSocket gateway between XiaoZhi ESP32 and OpenClaw agent."""

import asyncio
import json
import logging
import tempfile
import uuid
from enum import Enum
from shutil import which

import edge_tts
import whisper
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ValidationError

# ── Logging ──────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("openclaw-proxy")

# ── Configuration ────────────────────────────────────────────────────

OPENCLAW_CLI = which("openclaw") or "openclaw"
OPENCLAW_AGENT_ID = "main"
OPENCLAW_TIMEOUT = 120  # seconds

# ── Pydantic models ─────────────────────────────────────────────────

class HardwareAction(str, Enum):
    """Actions reported by XiaoZhi ESP32 hardware."""

    KEY_DOWN = "key_down"
    KEY_UP = "key_up"
    WAKE_WORD = "wake_word"

class HardwareEvent(BaseModel):
    """Incoming JSON frame from hardware."""

    type: str
    action: HardwareAction

class StateNotification(BaseModel):
    """Outgoing state frame pushed to hardware (e.g. breathing LED trigger)."""

    type: str = "state"
    status: str

class TextReply(BaseModel):
    """Outgoing text reply carrying the LLM response."""

    type: str = "text_reply"
    text: str

# ── Mock pipeline ───────────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text (mock).

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.

    TODO (real implementation):
        1. Decode Opus → WAV with ffmpeg-python:
             import ffmpeg
             wav, _ = (ffmpeg
                 .input("pipe:", format="opus")
                 .output("pipe:", format="wav", ar=16000, ac=1)
                 .run(input=audio_bytes, capture_stdout=True, capture_stderr=True))
        2. Or use pydub:
             from pydub import AudioSegment
             seg = AudioSegment.from_file(io.BytesIO(audio_bytes), codec="opus")
             wav = seg.set_frame_rate(16000).set_channels(1).raw_data
        3. Feed WAV bytes to STT (Whisper / FunASR / SenseVoice).
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    return "测试语音输入"

async def call_openclaw(text: str, session_id: str) -> str:
    """Send text to OpenClaw agent via CLI and return its reply.

    Args:
        text: User utterance transcribed by STT.
        session_id: Per-connection session ID for multi-turn conversation.

    Returns:
        Agent reply string.

    Raises:
        RuntimeError: If the CLI call fails or returns unexpected output.
    """
    logger.info("LLM: sending to OpenClaw → %s", text)

    proc = await asyncio.create_subprocess_exec(
        OPENCLAW_CLI, "agent",
        "--agent", OPENCLAW_AGENT_ID,
        "--session-id", session_id,
        "-m", text,
        "--json",
        "--timeout", str(OPENCLAW_TIMEOUT),
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()

    if proc.returncode != 0:
        err_msg = stderr.decode(errors="replace").strip()
        logger.error("OpenClaw CLI failed (rc=%d): %s", proc.returncode, err_msg)
        raise RuntimeError(f"OpenClaw CLI error: {err_msg}")

    data = json.loads(stdout.decode())
    payloads = data.get("result", {}).get("payloads", [])
    if not payloads:
        raise RuntimeError("OpenClaw returned empty payloads")

    reply = payloads[0].get("text", "")
    logger.info("LLM: reply ← %s", reply)
    return reply

async def process_tts(text: str) -> bytes:
    """Convert text to Opus audio (mock).

    Args:
        text: Text to synthesize.

    Returns:
        Opus-encoded audio bytes.

    TODO (real implementation):
        1. Call TTS engine (edge-tts / CosyVoice / GPT-SoVITS).
        2. Encode output to Opus:
             import ffmpeg
             opus, _ = (ffmpeg
                 .input("pipe:", format="wav")
                 .output("pipe:", format="opus", ar=16000, ac=1)
                 .run(input=wav_bytes, capture_stdout=True))
        3. Stream Opus frames back to ESP32 in ≤960-byte chunks.
    """
    logger.info("TTS: synthesising → %s", text)
    mock_opus = b"\xfc\xff\xfe" * 53  # 159 bytes, resembles Opus silence
    logger.info("TTS: generated %d bytes of audio", len(mock_opus))
    return mock_opus

# ── Frame router ────────────────────────────────────────────────────

async def handle_text_frame(ws: WebSocket, raw: str) -> None:
    """Parse and route an incoming JSON text frame.

    Args:
        ws: Active WebSocket connection.
        raw: Raw text payload received from client.
    """
    try:
        payload = json.loads(raw)
    except json.JSONDecodeError:
        logger.warning("Malformed JSON (ignored): %.120s", raw)
        return

    try:
        event = HardwareEvent(**payload)
    except ValidationError as exc:
        logger.warning("Invalid hardware event: %s", exc.errors())
        return

    logger.info("Hardware event: %s", event.action.value)

    if event.action == HardwareAction.WAKE_WORD:
        ack = StateNotification(status="listening")
        await ws.send_text(ack.model_dump_json())

async def handle_binary_frame(ws: WebSocket, data: bytes, session_id: str) -> None:
    """Run the full STT → LLM → TTS pipeline on an audio binary frame.

    Args:
        ws: Active WebSocket connection.
        data: Raw Opus audio bytes.
        session_id: Per-connection session ID.

    Response sequence pushed to client:
        1. {"type": "state", "status": "thinking"}  — triggers breathing LED
        2. {"type": "text_reply", "text": "..."}     — agent answer
        3. <binary>                                   — Opus audio of the answer
    """
    # Stage 1 — STT
    text = await process_stt(data)

    # Notify hardware: entering thinking phase
    await ws.send_text(StateNotification(status="thinking").model_dump_json())

    # Stage 2 — LLM (with periodic thinking heartbeats)
    async def heartbeat() -> None:
        """Send thinking heartbeat every 3 seconds while LLM is working."""
        while True:
            await asyncio.sleep(3)
            try:
                await ws.send_text(
                    StateNotification(status="thinking").model_dump_json()
                )
            except Exception:
                break

    heartbeat_task = asyncio.create_task(heartbeat())
    try:
        reply = await call_openclaw(text, session_id)
    finally:
        heartbeat_task.cancel()

    await ws.send_text(TextReply(text=reply).model_dump_json())

    # Stage 3 — TTS
    audio = await process_tts(reply)
    logger.info("Pushing %d bytes of audio to client", len(audio))
    await ws.send_bytes(audio)

# ── FastAPI application ─────────────────────────────────────────────

app = FastAPI(title="openclaw-proxy", version="0.1.0")

@app.websocket("/chat")
async def websocket_chat(ws: WebSocket) -> None:
    """Main WebSocket endpoint for XiaoZhi ESP32 hardware."""
    await ws.accept()
    peer = f"{ws.client.host}:{ws.client.port}" if ws.client else "unknown"
    session_id = f"xiaozhi-{uuid.uuid4().hex[:8]}"
    logger.info("Client connected: %s (session=%s)", peer, session_id)

    try:
        while True:
            message = await ws.receive()

            if message["type"] == "websocket.disconnect":
                break

            text_data = message.get("text")
            byte_data = message.get("bytes")

            if text_data is not None:
                await handle_text_frame(ws, text_data)
            elif byte_data is not None:
                await handle_binary_frame(ws, byte_data, session_id)
    except WebSocketDisconnect:
        pass
    finally:
        logger.info("Client disconnected: %s (session=%s)", peer, session_id)

@app.get("/health")
async def health() -> dict[str, str]:
    """Liveness probe."""
    return {"status": "ok"}

# ── Entry point ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn

    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

+FFMPEG_BIN = which("ffmpeg") or "ffmpeg"


Now replace the mock pipeline section with lazy model loading + real implementations:


# ── Mock pipeline ───────────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text (mock).

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.

    TODO (real implementation):
        1. Decode Opus → WAV with ffmpeg-python:
             import ffmpeg
             wav, _ = (ffmpeg
                 .input("pipe:", format="opus")
                 .output("pipe:", format="wav", ar=16000, ac=1)
                 .run(input=audio_bytes, capture_stdout=True, capture_stderr=True))
        2. Or use pydub:
             from pydub import AudioSegment
             seg = AudioSegment.from_file(io.BytesIO(audio_bytes), codec="opus")
             wav = seg.set_frame_rate(16000).set_channels(1).raw_data
        3. Feed WAV bytes to STT (Whisper / FunASR / SenseVoice).
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    return "测试语音输入"

# ── Whisper model (lazy singleton) ─────────────────────────────────

_whisper_model: whisper.Whisper | None = None

def _get_whisper_model() -> whisper.Whisper:
    """Return the Whisper model, loading it on first call."""
    global _whisper_model
    if _whisper_model is None:
        logger.info("Loading Whisper '%s' model…", WHISPER_MODEL_SIZE)
        _whisper_model = whisper.load_model(WHISPER_MODEL_SIZE)
    return _whisper_model

# ── STT / TTS pipeline ────────────────────────────────────────────

async def process_stt(audio_bytes: bytes) -> str:
    """Convert Opus audio to text via Whisper.

    Args:
        audio_bytes: Raw Opus-encoded audio from ESP32.

    Returns:
        Transcribed text string.
    """
    logger.info("STT: received %d bytes of Opus audio", len(audio_bytes))
    model = _get_whisper_model()
    with tempfile.NamedTemporaryFile(suffix=".opus", delete=True) as f:
        f.write(audio_bytes)
        f.flush()
        result = await asyncio.to_thread(
            model.transcribe, f.name, language="zh"
        )
    text = result["text"].strip()
    logger.info("STT: transcribed → %s", text)
    return text