MultiRoombaRover/plans/overseer v2 detailed architecture.md

# Overseer v2 Detailed Architecture

## Purpose
This document is the durable architecture spec for Overseer v2 so design intent is not lost across chats, sessions, or contributors.

Overseer v2 is a **new, separate service** focused on controlling the room/experience, not just commenting on rover activity.

## Non-Goals
- Do not replace or delete `llmCommentaryService`.
- Do not run two LLM overseer systems simultaneously in normal operation.
- Do not rely on giant unbounded context windows.

## Legacy Service Position
- Keep `llmCommentaryService` as a preserved legacy service.
- Keep it disable-able via config.
- New v2 service should be the primary runtime when enabled.
- Legacy service remains available for fallback/testing.

## Core Product Direction
Overseer v2 is "control-first":
- It can act on room systems and chat.
- Chat is one capability, not the whole job.
- It should feel ominous/playful and intentional, not spammy.

High-level behavior goals:
- Not overbearing.
- Does things when it can make moments interesting.
- Always responds when directly addressed.
- Does not always obey.
- Does not ramble.

## Primary Inputs Overseer Must Understand
Overseer must see all three lanes every decision cycle:

1. People activity
- Who is active now.
- Who is driving what.
- Activity bursts vs quiet periods.
- Direct addresses toward Overseer.

2. Conversation context
- **Actual recent chat conversation** (human + bot messages), not only tags.
- Last N human messages (minimum 5).
- Last 1-2 bot messages for anti-repeat context.

3. World/control state
- Rover states/events.
- Lift state/cooldowns.
- Neato state/cooldowns.
- Home Assistant entity states/availability.
- Safety/lock policy state.

## Runtime Model Strategy (Hardware-Aware)
Given modest hardware and ~30s generation on `mistral-small:24b` today:

- Use one LLM system only.
- Use deterministic program logic gate before LLM calls.
- Keep LLM calls sparse and meaningful.
- Keep context bounded and compact.

### Why Not Tiny-LLM Gate
Default gate should be deterministic logic, not another LLM:
- less latency
- less complexity
- fewer failure modes
- lower cumulative compute

A tiny LLM gate can be reconsidered later only for borderline cases.

## Continuous Loop Design (Never Stops)
System runs continuously but in two stages:

1. Fast gate loop (1-3s)
- No LLM call.
- Evaluates trigger conditions.
- If no trigger: continue.

2. LLM decision loop (on trigger or heartbeat)
- Triggered by gate or max-interval heartbeat.
- Builds bounded conversational context + state update.
- Model decides and optionally calls tools.

### Trigger Examples
- Direct address to Overseer in chat.
- Chat burst / topic change / challenge language.
- High-signal world event (dock/undock/hazard/control transition).
- Heartbeat timeout reached (for ambient presence).

## Decision Modes
The orchestrator should normalize model behavior to one of:
- `SKIP`
- `CHAT`
- `ACTION`
- `ACTION+CHAT`

Meaning:
- `SKIP`: no output, no tool call.
- `CHAT`: post chat only.
- `ACTION`: run tool(s) only.
- `ACTION+CHAT`: run tool(s) and post chat.

## Tool Output vs Chat Output
These are separate channels by design:
- Tool calls are structured actions.
- Chat text is explicit messaging output.
- A tool call does not automatically produce chat.

This separation is required to avoid chat spam and keep control behavior deliberate.

## Conversational Context Format (Important)
Overseer should use:
- Stable `system` prompt with identity/rules/policy.
- Real rolling transcript (users + overseer).
- A compact `STATE_UPDATE` message each decision cycle.

### Bounded Window Rules
To protect latency:
- Keep rolling window bounded (for example 20-40 recent turns, or last 5-10 min).
- Always include last 5 human messages minimum.
- Include last 1-2 bot messages.
- Drop/summarize older content.

## Tool Availability Contract
Critical refinement: do not advertise unusable tools as available.

Each cycle split into:
1. `available_tools`
- callable now

2. `blocked_tools`
- not callable now with reason:
  - cooldown
  - busy
  - unavailable/offline
  - policy lock
  - rate limit

Example:
- If Neato is cooling down, remove `neato_*` from `available_tools` and list under `blocked_tools` with remaining seconds.

This prevents wasted model turns and impossible tool attempts.

## Initial Tool Surface
Planned callable tools (subject to runtime availability):
- `chat_say(text)`
- `lift_up()` / `lift_down()`
- `neato_start()` / `neato_send_home()` / `neato_locate()` / `neato_clear_errors()`
- `ha_set_entity(entity_id, state)` for configured controllable entities
- memory tools:
  - `memory_read()`
  - `memory_write(slot, text)` where slot ∈ {1,2,3}

Optional later:
- controlled button-box count adjustment tool (strictly bounded and auditable)

## Persistent Memory (Tiny, Explicit)
Use tiny explicit memory store:
- exactly 3 slots
- string lines only
- explicit read/write via tools
- no autonomous unbounded memory growth

Model decides which slot to replace when writing.



## Mode and Policy Locks (Tool Disabling Rules)
Overseer tool availability must respect global site mode and room-light policy locks.

Hard requirements:
- When site mode is `admin` or `lockdown`, disable Overseer action tools for:
  - Neato controls (`neato_*`)
  - Lift controls (`lift_*`)
  - Room controls (`ha_set_entity` and related room-light toggles)
- When room lights are locked on by policy, disable room control tools even if site mode is otherwise open.

How this must appear to the model:
- Disabled tools must be removed from `available_tools`.
- Disabled tools must be present in `blocked_tools` with clear reason codes.

Recommended reason codes:
- `policy_lock:site_mode_admin`
- `policy_lock:site_mode_lockdown`
- `policy_lock:lights_locked_on`

Execution-time enforcement:
- Executor must re-check these locks immediately before executing any tool call.
- If a lock changed after context build, block execution and record a blocked action event.

## Safety and Control Guardrails
Must-have guardrails before enabling actions:

1. Hard allowlist
- Only approved tool names callable.

2. Tool-specific cooldowns
- e.g., lift/neato/HA each with independent cooldown rules.

3. Global action budget
- max actions per minute (or window).

4. Policy lock checks
- block actions during admin lock/safety states.

5. Execution-time validation
- arguments validated before invoking any control service.

6. Logging + audit trail
- every attempted/blocked/executed tool call recorded.

## Admin Debug UI Requirement
This is a core requirement.

Need a dedicated v2 admin panel that shows:
- enabled/running status
- loop phase (`idle`, `gate_check`, `awaiting_model`, `tool_exec`, `posted`, `failed`)
- last trigger reason
- last state-update payload snapshot
- model input summary
- model raw output + normalized decision mode
- tool calls attempted/executed/blocked with reasons
- cooldowns and remaining times
- recent action history
- last error/failure

### UI Switching Behavior
- If v2 enabled: show v2 panel.
- If legacy enabled: show legacy panel.
- If neither enabled: show disabled state.

Define a separate v2 status schema (do not overload legacy status object).

## Chat System Plumbing Changes
Keep the existing chat/Discord integration path and extend it minimally.

Decisions:
- Keep current internal bus-based chat flow as the canonical path (`chat:message` / `chat:typing`).
- Do not introduce a second parallel bridge path for v2.
- Add `bot: true|false` on chat message payloads.
- Overseer v2 must publish chat output through the same existing chat pipeline so Discord relay continues to work automatically.
- Keep legacy compatibility while migrating.

Goal:
- Any service can emit chat messages through one standardized path, with bot identity represented by `bot: true`.

## Example Message Topology Per LLM Decision
1. `system`: stable Overseer identity/rules/style/tool policy
2. `user`: compact `STATE_UPDATE`
3. transcript turns: recent chat + overseer messages
4. tool availability block (`available_tools`, `blocked_tools`)
5. model output: decision/tool calls/chat
6. executor: validates, executes tools, posts chat if applicable
7. append tool results and continue loop

## Implementation Phases

### Phase 0: Scaffolding
- Create new service module (suggested name: `overseerControlService`).
- Add config section and runtime enable switch.
- Keep legacy untouched.

### Phase 1: Observe-Only
- Build gate loop and context builder.
- Run model decisions with no real actions (dry-run tools).
- Log what would have happened.

### Phase 2: Chat + Safe Tools
- Enable chat output and lowest-risk tools.
- Verify anti-spam and direct-address behavior.

### Phase 3: Full Control Surface
- Enable lift/neato/HA with strict cooldowns and budgets.
- Add memory tool operations.

### Phase 4: Tuning
- Tune gate thresholds, cadence, cooldowns.
- Tune prompt style and anti-repeat behavior.
- Evaluate model alternatives only after gating/cadence tuning.

## Metrics To Track
- LLM calls per minute
- average LLM latency
- skipped vs acted decision ratio
- stale response rate (acted too late)
- blocked tool call rate and reasons
- action frequency per tool
- chat message frequency per minute
- repeated-line rate

## Rollback Plan
- Single config flip disables v2.
- Legacy overseer can be re-enabled without code rollback.
- Keep migration steps isolated and reversible.

## Practical Defaults (Starting Point)
- fast gate loop: 2s
- heartbeat model run: 30s
- min human chat context: 5 latest messages
- include latest bot messages: 2
- max tool calls per decision: 1 (start conservative)
- global action budget: low (start strict)

## Open Questions (Track During Build)
- Exact per-tool cooldown values for lift/neato/HA.
- Whether button-box count adjustment is worth enabling at all.
- Which model gives best tool reliability per watt on your machine.
- Ideal transcript window (turn-count vs time-window).
- Whether to include lightweight topic tags in addition to raw messages.

## One-Line Summary
Overseer v2 should be a separate, always-running, control-first orchestrator that uses deterministic gating + bounded real conversation context + dynamically available tools, with strict guardrails and a first-class debug UI.

## Naming Configuration Requirement
Overseer name must be configurable from server config.

Requirements:
- Add a config field for Overseer display/invocation name (for example under the v2 service config).
- The configured name must propagate everywhere it matters:
  - in-chat bot nickname
  - system prompt identity text
  - direct-address recognition rules
- Prompt file should remain editable by non-code changes, so use a placeholder token in prompt text (for example `<NAME>`) and replace it at runtime.
- The prompt should still support fallback behavior if name config is missing (default name).

Implementation intent notes (future):
- Keep a single source of truth for name resolution (config + default).
- Avoid scattering hardcoded names in service code.
- If legacy and v2 coexist in codebase, ensure each service can use configured naming without breaking compatibility.

## Canonical "What the Bot Sees" Example
Use this as the concrete reference for message formatting per decision cycle.

```txt
[system]
You are <NAME>, a control-first room AI.
- You may CHAT and/or call tools.
- Respect safety/cooldowns/allowlists.
- Prefer restraint; avoid spam.
- Respond when directly addressed.
- If nothing meaningful changed, SKIP.

[user]
STATE_UPDATE
time: 2026-05-02T22:14:10-04:00
trigger: chat_burst
cooldowns: lift=ready, neato=12s, ha=ready
rovers:
- rover1: driving, driver=alex, docked=false
- rover2: driving, driver=sam, docked=false
- rover3: docked=true, charging=true
lift: down, busy=false
neato: connected=true, state=idle, charging=true
lights:
- shelf=on, bench=off, corner=on, color=purple, aux1=off, aux2=on

[user] alex: overseer you watching this drift?
[user] sam: dont kill my lights again
[assistant] <NAME>: Concrete lane stays lit. Earn the darkness.
[user] alex: i dare you to make this harder
[user] sam: bot pick a side
[assistant] <NAME>: Lift is staying put. Chaos has standards.
[user] alex: sam is about to bonk the dock

[user]
available_tools:
- chat_say(text)
- lift_up()
- lift_down()
- ha_set_entity(entity_id, state)
- memory_read()
- memory_write(slot, text)

[user]
blocked_tools:
- neato_start() reason=cooldown remaining=12s
- neato_send_home() reason=cooldown remaining=12s
- neato_locate() reason=cooldown remaining=12s
- neato_clear_errors() reason=cooldown remaining=12s
```

Notes:
- The conversation transcript is real recent chat (humans + bot), bounded by window rules.
- `STATE_UPDATE` is compact current truth, regenerated each decision cycle.
- Tools must be split into `available_tools` and `blocked_tools`; unavailable tools are not advertised as callable.
- `<NAME>` is runtime-substituted from config.