# Touch Navigation and Bounded Agent History 0.6.3 Base: `main` / `d93ddcf89c31dd685ee27c045169d79a7d5088da` (Watch Glance 0.6.2). Branch: `pi/double-click-shortcuts-0.6.3`. Version: `0.6.3-dev`. This round adds CST820B touch, a host-testable touch gesture reducer, touch as a virtual A/B source for every existing modal owner, top-level swipe navigation, Agent manual paging, and a bounded read-only native Agent History. All physical A/B operation is preserved; Touch can be turned Off in Settings and the buttons remain fully functional. ## Touch driver The CST820B protocol is **verified**, not guessed. It follows the M5GFX StopWatch autodetect path in `firmware/managed_components/M5GFX/src/M5GFX.cpp` (which maps `0x15 // Touch CST820 (for StopWatch)` and constructs `lgfx::Touch_CST816S`) and that driver's `Touch_CST816S::getTouchRaw` / `_check_init`: ```text I2C address 0x15 touch registers read 0x02, 6 bytes count = data[0] & 0x0F x = data[2] | ((data[1] & 0x0F) << 8) y = data[4] | ((data[3] & 0x0F) << 8) chip id 0xA7 (3 bytes) init tail write 0x00=0x00, 0xFA=0x20, 0xED=20 raw range 0..233 on both axes (M5GFX config: x_max=233, y_max=233) INT GPIO13 reset TP_RST via M5IOE1 PYG4 (M5IOE1_PIN_4) ``` `boards/m5stack_stopwatch/cst820_touch.cpp` is a thin reader. It **reuses the board's existing `s_i2c_bus`** and never calls `M5.begin()`, `Wire.begin()` or `i2c_new_master_bus()`. The StopWatch M5GFX config uses its own bus/port; this firmware deliberately does not. `begin()` validates every init register write (`0x00`, `0xFA`, `0xED`). Any failure removes the device and returns the error, so a half-initialized controller is reported as nonfatal FAIL rather than ready. GPIO13 is configured as a negative-edge interrupt whose ISR only sets a flag. **I2C is never touched in an ISR.** `board_update` reads the controller only when the INT fired or a contact is already active; while a finger is down it samples about every 25 ms, and when idle it never polls I2C. Touch initialization failure is **nonfatal**: the log records `[TOUCH] init FAIL ... (nonfatal, buttons unaffected)` and every button path keeps working. Diagnostics are bounded, never per-move-sample: ```text [TOUCH] init OK CST820B addr=0x15 int=GPIO13 raw=0..233 display=468x466 [TOUCH] init FAIL addr=0x15 code=... reset=... (nonfatal, buttons unaffected) [TOUCH] coord top|bottom|left|right|center x= y= (display 468x466) ``` Each of the five zones is logged once, so a physical test can confirm orientation from serial output alone. ## Gesture reducer `wearable::TouchGestureReducer` (`boards/common/include/touch_gesture.h`) is pure and host-testable. It owns no I2C and no clock. ```text Tap movement <= 18 px, released Swipe distance >= 55 px on the dominant axis Swipe axis dominant >= 1.5 x secondary Long held >= 600 ms with movement <= 20 px Press min 30 ms (shorter contacts are noise and ignored) Side split x < 234 -> left half, else right half ``` Gestures: `Tap, SwipeLeft, SwipeRight, SwipeUp, SwipeDown, Long, LongRelease`. One contact resolves to exactly one primary gesture: a Long can never be followed by Tap/Swipe, and a Swipe can never be followed by Tap/Long. Movement beyond the tap slop cancels a future Tap but does not stop a swipe from resolving. `wearable::TouchTransform` (`boards/common/include/touch_transform.h`) is the single place that maps a raw sample to display space, including `swap_xy`, `mirror_x`, `mirror_y`. Coordinate flips are never scattered into UI code, so a physical test only edits this helper if an axis is reversed. ## Touch → virtual A/B Touch never duplicates Approval, Voice, Settings or Navigator logic. One resolved gesture becomes one virtual button gesture fed through the same `handle_gesture` reducer, so modal ownership, approval generation, UI epoch and stale protection are identical to the physical buttons. A contact captures the ownership epoch (`ui_epoch + approval generation`) at its first valid contact, exactly like a physical press records `ButtonGesture::epoch`. Every Tap / Swipe / Long from that contact belongs to that captured epoch. If Approval, Native UserInput, Voice, Settings or any other owner changes the epoch mid-contact, the unresolved gesture is discarded and never applied to the new owner. A Touch Down on the Shell followed by Approval or Native UserInput and then a Touch Up therefore neither approves/rejects nor answers nor navigates. A legitimately triggered `Long` still bumps the epoch when it opens Voice, and its `LONG_RELEASE` is delivered un-gated so it can end that recording, matching the physical hold. The Navigator is the same kind of owner. Its `CLICK` / `LONG` actions are only delivered when the gesture still belongs to the captured epoch; a stale gesture is consumed (so it cannot reach the new owner's reducer) but never acted on. A `LONG_RELEASE` is always delivered, because it is the completion half of an interaction that already legitimately started. Default map: ```text Tap right half -> virtual A CLICK Tap left half -> virtual B CLICK Long right half -> virtual A LONG then LONG_RELEASE Long left half -> virtual B LONG then LONG_RELEASE ``` So Shell right = Next / left = Previous, Voice right = Send / left = Cancel, Approval right = Approve or first native choice / left = Reject or second, Settings right = Next / left = Select or Apply, Navigator right = Next / left = Select. Touch does **not** implement Double Tap. Double A and Double B remain physical button shortcuts only. ### Top-level swipe When `InputModal == NONE`, no approval and no overlay: ```text SwipeLeft -> Next app (equivalent to A) SwipeRight -> Previous app (equivalent to B) ``` ### Agent vertical paging The default Agent body is the existing live/auto tail. The first vertical swipe enters Manual Browse; returning to the newest local page resumes Live Tail. A target change resets the manual page. New Agent output never forcibly jumps a user who is reading an older page; returning to the newest page shows the current authoritative state immediately. Page direction is a single invariant shared by the renderer, `AgentBrowse` and the Older/Newer actions: ```text page 0 = newest page (= the live tail) Older => page index increases page N = oldest page Newer => page index decreases ``` Every page is measured **backwards from the end of the body**, so page 0 is exactly the live tail and adjacent pages never overlap or leave a gap: ```text end = N - page * rows start = max(0, end - rows) ``` The oldest page may therefore be short; pages are never re-aligned to the body head to make page 0 full. `View.page_count` / `View.page_index` and the drawn page both come from the renderer's single `body_range()` calculation, so the live view and manual browse use one implementation (same font, width, wrapping and row count). ```text SwipeDown -> Older SwipeUp -> Newer (at the newest, resumes Live Tail) SwipeDown past the oldest local page -> History newest page ``` ## Bounded native History Read-only. There is no second conversation database on the device or the Gateway: pages are normalized from native persistence on demand. New capability and authenticated WSS messages: ```text capability history device -> {"type":"history_get","request_id":...,"expected_target_generation":N, "target":"agent:...","agent":"pi|codex","session_id":"...","cursor":...} gateway -> {"type":"history_page","request_id":...,"agent":...,"generation":N, "messages":[{"role":"user|assistant","text":"..."}],"next_cursor":...,"oldest":bool} gateway -> {"type":"history_error","request_id":...,"message":"..."} ``` Bounds, enforced by the Gateway: ```text <= 4 messages per page <= 4096 UTF-8 bytes for the whole frame <= 1200 UTF-8 bytes per message, truncated on a code point boundary newest first; opaque cursor for the next OLDER page ``` Only `user` and `assistant` are ever exposed. Reasoning, commentary, tool output, command output, file changes, credentials and internal metadata are excluded, and credentials are redacted before truncation. Device cache is fixed at **3 pages** (`AgentHistoryCache::kCachePages`); it holds the current page plus a small neighbourhood and evicts from the newest end once the window is full. Content and the displayed page are one state: a page fetched by Older becomes the current page, and Older/Newer traverse the cached pages before asking the server again. The whole session history is never stored on the ESP32. Request identity is bound to the device connection, target, target generation, agent and session_id. A target switch, generation change, disconnect, approval, Native UserInput, Voice or Settings takeover cancels or ignores an in-flight or late page; a stale response can never overwrite a new session. The Gateway re-validates the target after the native read completes. ### Codex `read_history` reuses the native `thread/turns/list` API with `sortDirection=desc`, `itemsView=full` (the same source as Last Reply) and never resumes a dormant thread. Each completed turn normalizes to at most one `userMessage` and one `agentMessage`: the user prompt is read from the `content[]` array (the `text` variant; `image` / `audio` / `skill` / `mention` parts are references, not prose), and the assistant prefers `phase=final_answer`, otherwise the last non-commentary `agentMessage`. Reasoning and tool items are skipped. ### Pi The SDK host gains a read-only `history` method that opens `SessionManager.open(info.path).buildSessionContext()` and returns persisted messages. It never resumes the session, never creates a runtime and never mutates anything. Only `role=user` / `role=assistant` text content is normalized; `toolResult`, `toolCall`, `reasoning` and `system` are skipped. The SDK returns the tail oldest -> newest, so the adapter reverses it: every native adapter hands the unified history layer **newest-first** data. ### History UI ```text CODEX · HISTORY PI · HISTORY USER ASSISTANT continue the network already done... fix... 2 / 4 ``` Controls: ```text A / tap right / SwipeUp -> Newer B / tap left / SwipeDown -> Older Hold B / Long left -> Close History ``` Browsing is strictly linear across text screens, so a long message is traversed fully before the next older message appears, and the reverse is exactly reversible: ```text message N screen 1 -> message N screen 2 -> message N screen 3 -> message N-1 screen 1 ``` The footer shows the message position, and the text-screen position only when the current message needs more than one screen: ```text 2 / 4 (single-screen message) 2 / 4 · 2 / 3 (message 2 of 4, screen 2 of 3) ``` Buttons and touch use the same model: there is no second History UI. Fetching a new page resets both the message index and the text screen. ### Button-only entry Settings → Agent → **History** appears only when the selected Agent exposes the `history` capability. It is fully usable with A Newer / B Older / Hold B Close, so no feature requires touch. Touch users can also reach History by swiping down past the oldest local Agent page. ## Touch preference Settings → Device → **Touch On / Off**, default **On**. The NVS codec moves from v4 to **v5** with the new byte `touch_enabled = 1`: ```text v1 (5 bytes), v2 (7), v3 (9), v4 (10), v5 (11) v1-v4 migrate with Touch = On dual-slot, generation, CRC and future-version read-only are preserved ``` Touch Off ignores every touch gesture; the physical buttons stay complete, so Settings → Device → Touch can always turn it back on. ## Accidental touch protection ```text jitter below the tap slop -> never a swipe contact shorter than 30 ms -> ignored (noise) long with movement beyond 20 px -> not a Long, so never Voice/Settings a swipe -> never a Tap more than one reported point -> retried, then treated as no touch ``` The panel is treated as single-touch; no multi-touch complexity is added. ## Tests * `firmware/tests/touch_gesture_test.cpp` — tap slop boundaries, four swipe directions, swipe axis ratio, Long/LongRelease, movement cancels Long, noise ignored, one primary gesture per contact, left/right split, and the transform (map/mirror/clamp). * `firmware/tests/agent_history_test.cpp` — cache bound (3 pages), older/newer boundaries, a fetched older page becoming the current page, eviction keeping the index in range and traversal linear, manual browse + live tail restore + reset, History screen linear paging, multi-screen message traversal (fully older then exactly reversible), message-boundary crossing onto the previous message's last screen, and UTF-8-safe truncation. * `firmware/tests/wearable_renderer_test.cpp` — Agent Older/Newer page direction: a 48-line body is walked page by page and the visible range is strictly monotonic (Older => older text, Newer => newer text), the page count equals the shared renderer pagination, the footer counts 1/N (newest) to N/N (oldest), and the live auto-advance rotates from newest toward older. * `firmware/tests/shell_input_test.cpp` (via `run_shell_input_test.py`) — touch virtual A/B (both halves), top-level swipe navigation, Agent vertical browse and History entry, history page accept/stale rejection, A/B History controls, Hold B close, Touch Off leaving the buttons intact, the Settings Agent History / Device Touch entries, touch contact epoch ownership (Approval takeover and Native UserInput takeover mid-contact never act, a stale Long never opens Voice, a legitimate Touch Long's LONG_RELEASE still ends Voice), Navigator stale-epoch ownership against a real Navigator reducer (a stale gesture never commits a Native UserInput response, a legal Long still finishes), History cross-page fetch/cache traversal (a server page becomes the displayed page and Older/Newer then use the cache without refetching), and multi-screen History browsing with button/touch parity and a reset on a new page. * `server/tests/test_history.py` — cursor round-trip and rejection, Codex normalization from the real `userMessage.content[]` schema (user text kept, non-text parts excluded, legacy flat `text` tolerated), Pi normalization (only user/assistant, final preference, commentary/tool/reasoning exclusion), Pi adapter newest-first ordering, adapter -> normalized history -> pagination integration for both Codex and Pi, page bounds and frame size, cursor pagination, credential redaction, navigator target/generation/identity binding, target-change-during-read, capability absence, and `history_error` emission. * `server/tests/test_speech.py` — the earlier `speech-settings` / `speech-playback` split remains covered. Double-click, Voice, Settings, Approval and Navigator regressions continue to pass in the same host harness, so single / double / long / touch do not contaminate each other. ## Gates ```text git diff --check PASS ./scripts/test-wearable-shell.sh PASS ./scripts/test-network-resilience.sh PASS (88 Python tests) ./scripts/test-server.sh PASS (239 Python tests, plus Node/firmware/smoke) ./scripts/build.sh m5stack-stopwatch PASS ``` ## Not in this round Speech playback repair, alarm, power-save redesign, WSS cadence redesign, multi-touch, complex animations, full conversation database, and tool/reasoning history. ## Open physical uncertainties Host tests cannot prove real coordinates. `[TOUCH] coord ...` records the first top/bottom/left/right/center contact, and `TouchTransform` centralizes any flip. Swipe thresholds (55 px / 18 px / 20 px) and the 600 ms long-touch feel are first physical-test values. Edge behavior near the bezel is unverified.