do it properly with the mediamtx giant publisher file that no one maintains a package of for some god forsaken reason agh

Refactor docking undocking serverside

.dockerignore

@@ -5,6 +5,7 @@
 .git
 .gitignore
 **/node_modules
+webui/.cache/mediamtx/
 # Neither the legacy state directory nor the Compose-mounted replacement may
 # enter an image build. This prevents credentials and backups from becoming
 # image layers after an operator has started using either deployment layout.

.gitignore

@@ -5,6 +5,7 @@ create_2_Open_Interface_Spec.txt
 
 logs
 node_modules/
+webui/.cache/mediamtx/
 __pycache__/
 *.py[cod]
 .pio

compose.yaml

@@ -23,9 +23,13 @@ services:
       # Required for Bluetooth hardware such as the Balance Board.
       - /run/dbus/system_bus_socket:/run/dbus/system_bus_socket:ro
 
-    # Required for Kinect USB access.
+    # Kinect USB access.
     devices:
       - /dev/bus/usb:/dev/bus/usb
+      # For a server-local microphone, uncomment this device and group_add below.
+      # - /dev/snd:/dev/snd
+    # group_add:
+    #   - "${AUDIO_GID:?Set AUDIO_GID to the host sound device group ID}"
 
     cap_add:
       - NET_ADMIN

docs/server-admin-container-migration.md

@@ -26,6 +26,8 @@ Phase 1 must be complete and verified before Phase 2 begins. Containerization mu
 
 ## Decision log
 
+- 2026-10-02: The admin container controls accept a tag within the repository supplied by Compose through `MULTIROVER_TARGET_IMAGE`. A successful healthy replacement saves the tag in the lifecycle status volume; failed replacements retain the previous selection. Checks and later updates follow that saved tag. Changing repositories in Compose resets the selection. Manual Compose deployments still use the Compose image and can replace a UI-selected image; the UI displays both references. The lifecycle helper itself is updated through Compose. Image rollback does not revert application data migrations. Deploy this feature to both the application and lifecycle services before using tag selection.
+
 - 2026-09-14: Publish `ghcr.io/legop3/multiroombarover:latest` only from the repository's main branch. Every other repository branch publishes one moving development image named for that branch, with invalid tag separators normalized; these are development selectors rather than numbered releases. Pull requests only verify that the image builds. There are no release numbers, semantic-version tags, stable/edge channels, or operator-facing version selection. Docker image digests remain an internal mechanism for detecting an available update and retaining the previously running image for rollback.
 - 2026-09-15: Support only `linux/amd64` for the central server image. Rover computers remain independently ARM-capable, but publishing an untested ARM server image would multiply native-worker, media-binary, TTS-library, and hardware validation without serving the current deployment. ARM server support can be added later when a real ARM server exists to verify it.
 - 2026-09-15: Mount the application data directory from the Docker-managed `multirover-data` named volume instead of a host bind path. Docker initializes the empty volume with the image's non-root ownership, eliminating host UID matching, directory creation, ownership commands, and root application startup. The admin backup/restore system is the supported portable interface to the complete data tree.

docs/spectator-bot.md

@@ -64,6 +64,7 @@ socket.on('connect', async () => {
 // previously stored session with this object instead of merging snapshots.
 socket.on('session:sync', (session) => {
   console.log('Session:', session);
+  console.log('Server mode:', session.mode);
 });
 
 // chat:init contains the recent chat history available when the bot connects.
@@ -105,6 +106,21 @@ Run it with:
 node bot.js
 ```
 
+## Reading server mode
+
+Read `session.mode` from the `session:sync` snapshot to get the current server
+mode. Its possible values are `open`, `turns`, `admin`, and `lockdown`.
+
+The server sends a fresh session snapshot whenever the mode changes, so the
+example's handler receives both the initial mode and subsequent updates. No
+polling or separate mode listener is needed. If your bot stores session data for
+its chat-handling logic, replace the stored session on every `session:sync` and
+read the mode from that latest snapshot.
+
+`session.mode` is the server-wide mode; `session.role` is this connection's role
+(such as `spectator`), and `session.operatingMode` is a separate connection field.
+Use `session.mode` when answering chat questions about the server's current mode.
+
 ## Events used
 
 - `nickname:set` sets the bot's visible nickname.

perf/nomad-benchmark.py

@@ -0,0 +1,208 @@
+#!/usr/bin/env python3
+"""CPU benchmark of published NoMaD exploration, without ROS or rover connections.
+
+Run: python3 perf/nomad-benchmark.py --setup
+Subsequent runs: python3 perf/nomad-benchmark.py
+Quick run: python3 perf/nomad-benchmark.py --threads 4 --samples 1 --seconds 2 --iterations 2
+
+--setup installs CPU dependencies in a private cached venv and downloads pinned
+upstream source plus the official checkpoint. Nothing touches server packages.
+Synthetic images measure compute, NOT navigation quality. Excludes capture,
+video decode, network, physical control, training, and concurrent rover load.
+"""
+import argparse
+import hashlib
+import importlib.metadata
+import json
+import math
+import os
+from pathlib import Path
+import platform
+import resource
+import statistics
+import subprocess
+import sys
+import time
+import urllib.request
+import venv
+
+VINT = 'dca79815b704e5aa9c6bdc3082351f9e3b2848c2'
+DIFFUSION = '5ba07ac6661db573af695b419a7947ecb704690f'
+CHECKPOINT_ID = '1YJhkkMJAYOiKNyCaelbS_alpUpAJsOUb'
+SOURCES = {
+    ('robodhruv/visualnav-transformer', VINT): [
+        'train/config/nomad.yaml', 'train/vint_train/data/data_config.yaml',
+        'train/vint_train/models/nomad/nomad.py',
+        'train/vint_train/models/nomad/nomad_vint.py',
+        'train/vint_train/models/vint/self_attention.py',
+    ],
+    ('real-stanford/diffusion_policy', DIFFUSION): [
+        'diffusion_policy/model/diffusion/conditional_unet1d.py',
+        'diffusion_policy/model/diffusion/conv1d_components.py',
+        'diffusion_policy/model/diffusion/positional_embedding.py',
+    ],
+}
+
+
+def positive(value):
+    result = int(value)
+    if result < 1:
+        raise argparse.ArgumentTypeError('Must be positive')
+    return result
+
+
+def arguments():
+    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    p.add_argument('--setup', action='store_true')
+    p.add_argument('--cache', type=Path, default=Path.home() / '.cache/rover-nomad-benchmark')
+    p.add_argument('--threads', type=positive, nargs='+', default=[4, 8, 16])
+    p.add_argument('--samples', type=positive, nargs='+', default=[1, 8], help='Candidate trajectories per decision, not rover count')
+    p.add_argument('--seconds', type=positive, default=10)
+    p.add_argument('--iterations', type=positive, default=10)
+    p.add_argument('--worker', action='store_true', help=argparse.SUPPRESS)
+    p.add_argument('--result', type=Path, help=argparse.SUPPRESS)
+    return p.parse_args()
+
+
+def download(url, path):
+    if path.exists():
+        return
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_suffix(path.suffix + '.part')
+    print('Downloading', path.name, flush=True)
+    with urllib.request.urlopen(url, timeout=120) as response, temporary.open('wb') as output:
+        import shutil
+        shutil.copyfileobj(response, output)
+    temporary.replace(path)
+
+
+def prepare(args):
+    python = args.cache / 'venv/bin/python'
+    if not python.exists():
+        venv.EnvBuilder(with_pip=True).create(args.cache / 'venv')
+    subprocess.run([str(python), '-m', 'pip', 'install', 'torch', 'torchvision',
+                    '--index-url', 'https://download.pytorch.org/whl/cpu'], check=True)
+    subprocess.run([str(python), '-m', 'pip', 'install', 'numpy', 'diffusers',
+                    'efficientnet-pytorch', 'einops', 'PyYAML', 'gdown'], check=True)
+    for (repo, revision), paths in SOURCES.items():
+        for path in paths:
+            download(f'https://raw.githubusercontent.com/{repo}/{revision}/{path}', args.cache / 'source' / revision / path)
+    checkpoint = args.cache / 'nomad.pth'
+    if not checkpoint.exists():
+        temporary = checkpoint.with_suffix('.part')
+        subprocess.run([str(python), '-m', 'gdown', CHECKPOINT_ID, '-O', str(temporary)], check=True)
+        temporary.replace(checkpoint)
+
+
+def worker(args):
+    import numpy as np
+    import torch
+    import yaml
+    from diffusers.schedulers.scheduling_ddpm import DDPMScheduler
+    sys.path[:0] = [str(args.cache / 'source' / VINT / 'train'), str(args.cache / 'source' / DIFFUSION)]
+    from vint_train.models.nomad.nomad import NoMaD, DenseNetwork
+    from vint_train.models.nomad.nomad_vint import NoMaD_ViNT, replace_bn_with_gn
+    from diffusion_policy.model.diffusion.conditional_unet1d import ConditionalUnet1D
+
+    threads, samples = args.threads[0], args.samples[0]
+    torch.set_num_threads(threads)
+    torch.set_num_interop_threads(1)
+    torch.manual_seed(0)
+    c = yaml.safe_load((args.cache / 'source' / VINT / 'train/config/nomad.yaml').read_text())
+    vision = replace_bn_with_gn(NoMaD_ViNT(
+        context_size=c['context_size'], obs_encoding_size=c['encoding_size'],
+        mha_num_attention_heads=c['mha_num_attention_heads'],
+        mha_num_attention_layers=c['mha_num_attention_layers'], mha_ff_dim_factor=c['mha_ff_dim_factor']))
+    model = NoMaD(vision, ConditionalUnet1D(input_dim=2, global_cond_dim=c['encoding_size'],
+        down_dims=c['down_dims'], cond_predict_scale=c['cond_predict_scale']), DenseNetwork(c['encoding_size'])).eval()
+    # Strict loading prevents accidentally benchmarking a partly random model.
+    model.load_state_dict(torch.load(args.cache / 'nomad.pth', map_location='cpu', weights_only=True), strict=True)
+    scheduler = DDPMScheduler(num_train_timesteps=c['num_diffusion_iters'],
+        beta_schedule='squaredcos_cap_v2', clip_sample=True, prediction_type='epsilon')
+    width, height = c['image_size']
+    frames = torch.rand(c['context_size'] + 1, 3, height, width)
+    mean = torch.tensor([.485, .456, .406]).view(1, 3, 1, 1)
+    std = torch.tensor([.229, .224, .225]).view(1, 3, 1, 1)
+    mask = torch.ones(1, dtype=torch.long)
+    stats = yaml.safe_load((args.cache / 'source' / VINT / 'train/vint_train/data/data_config.yaml').read_text())['action_stats']
+    low, high = np.array(stats['min']), np.array(stats['max'])
+
+    def decision():
+        start = time.perf_counter()
+        obs = ((frames - mean) / std).reshape(1, -1, height, width)
+        goal = torch.randn(1, 3, height, width)
+        cond = model('vision_encoder', obs_img=obs, goal_img=goal, input_goal_mask=mask).repeat(samples, 1)
+        encoded = time.perf_counter()
+        action = torch.randn(samples, c['len_traj_pred'], 2)
+        scheduler.set_timesteps(c['num_diffusion_iters'])
+        for step in scheduler.timesteps:
+            noise = model('noise_pred_net', sample=action, timestep=step, global_cond=cond)
+            action = scheduler.step(noise, step, action).prev_sample
+        # Match upstream get_action: unnormalize deltas then accumulate waypoints.
+        waypoints = np.cumsum((action.numpy() + 1) / 2 * (high - low) + low, axis=1)
+        if not np.isfinite(waypoints).all():
+            raise RuntimeError('Non-finite model output')
+        end = time.perf_counter()
+        return (end - start) * 1000, (encoded - start) * 1000, (end - encoded) * 1000
+
+    rows = []
+    with torch.inference_mode():
+        for _ in range(3):
+            decision()
+        start, cpu = time.perf_counter(), time.process_time()
+        while len(rows) < args.iterations or time.perf_counter() - start < args.seconds:
+            rows.append(decision())
+        elapsed, cpu = time.perf_counter() - start, time.process_time() - cpu
+    total = sorted(r[0] for r in rows)
+    result = dict(threads=threads, candidateTrajectories=samples, iterations=len(rows),
+        p50Ms=statistics.median(total), p95Ms=total[math.ceil(.95 * len(total)) - 1], maxMs=max(total),
+        decisionsPerSecond=len(rows)/elapsed, cpuCores=cpu/elapsed,
+        peakMiB=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss/1024,
+        visionMeanMs=statistics.mean(r[1] for r in rows), diffusionAndDecodeMeanMs=statistics.mean(r[2] for r in rows),
+        parameters=sum(p.numel() for p in model.parameters()), diffusionSteps=c['num_diffusion_iters'],
+        imageSize=c['image_size'], frames=c['context_size']+1,
+        versions={name: importlib.metadata.version(name) for name in ['torch','torchvision','diffusers','numpy','efficientnet-pytorch','einops']})
+    args.result.write_text(json.dumps(result))
+
+
+def main():
+    args = arguments()
+    args.cache = args.cache.expanduser().resolve()
+    if args.worker:
+        worker(args)
+        return
+    if args.setup:
+        prepare(args)
+    python = args.cache / 'venv/bin/python'
+    if not python.exists() or not (args.cache / 'nomad.pth').exists():
+        raise RuntimeError('First run: python3 perf/nomad-benchmark.py --setup')
+    print('CPU FP32 NoMaD exploration; synthetic images; no driving-quality measurement.', flush=True)
+    print('Candidates are trajectories for ONE rover. Includes vision + all 10 diffusion steps.', flush=True)
+    print('threads candidates  p50 ms  p95 ms  max ms decisions/s CPU cores peak MiB', flush=True)
+    results = []
+    import tempfile
+    with tempfile.TemporaryDirectory(prefix='nomad-bench-') as directory:
+        for threads in args.threads:
+            for samples in args.samples:
+                path = Path(directory) / 'result.json'
+                subprocess.run([str(python), str(Path(__file__).resolve()), '--worker', '--cache', str(args.cache),
+                    '--threads', str(threads), '--samples', str(samples), '--seconds', str(args.seconds),
+                    '--iterations', str(args.iterations), '--result', str(path)], check=True,
+                    timeout=max(600, args.seconds*20), env={**os.environ, 'PYTHONDONTWRITEBYTECODE':'1'})
+                r = json.loads(path.read_text()); results.append(r)
+                print(f"{threads:7} {samples:10} {r['p50Ms']:7.1f} {r['p95Ms']:7.1f} {r['maxMs']:7.1f} {r['decisionsPerSecond']:11.2f} {r['cpuCores']:9.2f} {r['peakMiB']:8.0f}", flush=True)
+    with (args.cache / 'nomad.pth').open('rb') as f:
+        digest = hashlib.file_digest(f, 'sha256').hexdigest()
+    report = dict(benchmark='nomad-cpu-v1', platform=platform.platform(), logicalCpus=os.cpu_count(),
+        loadAverage=os.getloadavg(), sourceRevisions=dict(vint=VINT,diffusion=DIFFUSION), checkpointSha256=digest,
+        exclusions=['capture','video decode','network','physical control','training','concurrent model workloads'], results=results)
+    print('\nPaste this report back:\n' + json.dumps(report, separators=(',', ':')))
+    print('At 10 decisions/s the total budget is 100ms; at 5/s it is 200ms. Leave room for the excluded pipeline.')
+
+
+if __name__ == '__main__':
+    try:
+        main()
+    except (Exception, KeyboardInterrupt) as error:
+        print(f'Benchmark failed: {error}', file=sys.stderr)
+        sys.exit(1)

pi/roverd/auto_charge.go

@@ -7,7 +7,7 @@ import (
 )
 
 const (
-	autoChargeTimeout  = 1 * time.Second
+	autoChargeTimeout  = 10 * time.Second
 	autoChargeCooldown = 0 * time.Minute
 	sourceHomeBase     = 1 << 1
 )

plans/rover learning service.md

@@ -2,27 +2,34 @@
 
 ## Resume here
 
-Latest change: format 3 command-event policy replaces format 2 continuous wheel
-speeds. All supported controls, including drive and recorded zero-speed stops,
-use scored command templates and the same threshold/checked command pipe. The
-network produces scores and numeric values; it never generates JSON text.
+Latest repair: full learning-path audit. Fixed missing command labels between
+video samples with a shared causal 100ms grid; added real-data score/baseline/
+training-probe diagnostics and normalized saved control profiles for retained
+recordings. Continues policy 5 / lineage v6; this audit does not reset progress.
+See the final audit results section for measured synthetic evidence and limits.
 
-Implemented: remove wheel-speed telemetry and previous drive-command history
-from inputs (raw recordings unchanged); preserve exact numeric zeros; reuse
-recordings with a fresh resume-v4.pt / schedule-v4.json lineage. Startup deletes recognized older-format snapshots/checkpoints and old training
-lineages; recordings and current-format models are preserved. One ongoing learner, gated
-Latest/Previous, and admin-created permanent snapshots remain.
+Latest scope (authorized IPI): only drive, camera tilt, headlight and the four
+operating-mode/dock commands (128/131/132/143). Exclude horn, song, laser and all
+brush/vacuum motor commands from both learning and execution, including raw OI
+bypasses. Raw recordings stay intact. Policy format 5, training lineage v6;
+startup removes earlier models and lineages through v5. Earlier scope below is
+historical. No configuration change required.
 
-Main card shows decoded command parameters, score, applied threshold marker, and
-submission results, including below-threshold proposals. Diagnostics reportVersion
-3 includes command predictions/submissions and event evaluation. Manual inputs
-still never stop or pause autonomy. Disabled remains dormant/off by default.
+Latest change (2026-09-29 IPI): format 4 seeds known user-control command shapes
+before training. Generic peripherals are excluded; sensorStream is ignored without
+invalidating footage. Drive includes zero-speed drive, with no separate stop type.
+Drive, camera, brushes/vacuum, headlight, laser, horn, songs and operating modes
+remain supported. Raw recordings and the existing checked command pipe stay intact.
 
-Focused Node syntax, Python production worker entry points, and card ESLint
-passed. New policy: 803,440 parameters, 30 sensor fields, stateSize 700. No live
-recordings/camera/rover pipeline or authenticated browser were available locally.
-Actual training, stop behavior, concurrent performance and visual appearance
-remain unverified. No tests were added and no processes were left running.
+Held-out data cannot expand the schema. Unknown control variants exclude nearby
+windows only, with command/count/reason diagnostics and a coverage gate that blocks
+promotion. Supported portions still receive evaluation. Current training lineage
+is resume-v5.pt / schedule-v5.json. Startup removes obsolete format 1-3 models and
+old lineages through v4; recordings and current-format models are preserved.
+
+Verification for this increment is recorded at the end. Earlier version-specific
+notes are historical. Manual inputs never pause/cancel autonomy; disabled service
+remains dormant. Actual autonomous performance has not been established.
 
 This document preserves the conversation's decisions across compactions and
 usage cutoffs. Keep the checklist and handoff section current during authorized
@@ -930,3 +937,141 @@ Cleanup runs even when training is disabled, but not when the whole service is o
 Production deletion occurs on the next enabled-service startup after deployment.
 Verification: focused Node syntax and diff whitespace checks passed. No production
 storage was accessed; actual deletion awaits enabled-service startup on the server.
+
+### User-control scope and evaluation coverage (2026-09-29)
+
+Authorized by latest IPI after user clarified that controls include camera, brushes,
+lights, laser, horn and songs but NOT generic peripherals. No new command pipe or
+separate stop command. sensorStream is neither input/target nor a boundary. Actual
+reset/reboot/update remain temporal boundaries because they interrupt observations.
+
+Format 4 seeds drive and motor zero/nonzero field combinations, servo, light/laser
+actions, the two four-frequency horn waveforms and horn release, OI start/safe/full/
+dock bytes and default song. Seed values describe shapes, not artificial training
+examples. Horn frequencies are numeric parameters; integral JSON numbers share
+shapes with equivalent floating representations for drive/motors/servo/horn. Songs
+remain exact demonstrated note arrays; other song/raw variants can extend the
+training schema within its existing 64-slot bound. No promise to pre-enumerate all
+possible tunes. Seeded slots receive negative examples even in sessions where that
+control was not used, avoiding untrained random seeded outputs.
+
+Evaluation freezes the training schema. Unsupported commands are reported with
+payload, reason, occurrence count (up to 32 distinct descriptions), total count and
+affected-window count. The affected observation history/target interval is excluded
+without labeling the unknown command as inaction or skipping the entire recording.
+Any unsupported held-out command blocks promotion; metrics describe only supported
+windows. If none survive, dataset diagnostics still report the coverage problem.
+Existing permission checks, threshold mechanics, recording and manual input behavior
+are unchanged. Generic peripherals are also removed from execution allowlist and
+rover compatibility profile. Previous policy models are automatically cleaned up;
+no configuration edit is required.
+
+Verification completed: train.py --check-dependencies loaded the production
+worker and constructed format 4 with 803,440 parameters and predefined command
+shapes. infer.py --help, Node syntax checks for the four changed service modules,
+focused VipRoverLearningCard ESLint and diff whitespace checks passed.
+No live server data/hardware/browser available locally. Actual training, driving,
+and browser appearance remain unverified. No new tests or background processes.
+Deploying starts a new learner using existing recordings; enabled-service startup
+removes older policy models and optimizer/scheduler lineages through v4. Current
+lineage is v5. No settings changes or prescribed human driving are required.
+
+### Simplified controls, policy 5
+
+User authorized removal of horn/song/laser/brush motors. Retained 13 seeded shapes:
+four drive zero/nonzero combinations, two camera angle shapes, three headlight
+actions, four raw operating-mode/dock constants. Generic peripherals remain excluded.
+Removed excluded command history/targets via policy allowlist and execution via
+Node allowlist; raw payloads must exactly match one of the four mode commands at
+both boundaries. Controller shutdown now sends only zero-speed drive, leaving
+manual brush/horn state alone. Raw recording and sensor telemetry unchanged.
+Old policies are deleted on enabled startup; resume-v6.pt/schedule-v6.json begin a
+fresh learner using retained recordings. Existing evaluation behavior unchanged.
+Verification passed: production train.py --check-dependencies (policy 5, 13 seeded
+shapes), infer.py --help, changed Node module syntax, focused card ESLint, and diff
+whitespace checks. No live hardware/data available locally; actual learning and
+driving quality remain unverified. No tests added or background processes left.
+
+### Active full learning-path audit (authorized IPI)
+
+User explicitly permits local training/inference with controlled synthetic input;
+there are NO real recordings on this machine. Existing user-control simplification
+edits were present at start and must be preserved. No new schema reset is planned.
+
+Reproduced: production reader with a generated lossless 25 FPS video and 31 valid
+human drive events labeled ZERO commands. Old sampler used 120ms frame intervals
+with only 100ms target coverage. Fixed causal 100ms grid shared by dataset/inference
+uses latest past image; no future pixels. The repaired fixture preserves all 31
+commands. Sensor cutoff is strictly before decision in both paths.
+
+Original objective experiment: 12 easy colored-frame samples, 120 steps, seed 7,
+4 threads: loss 0.8206 -> 0.1996, mean true-command score 0.5106 -> 0.7268, recall
+at 0.7 reached 2/3; at 80 steps recall was still zero. This disproves a wholly
+broken gradient path; it does not diagnose real-data score suppression. Keep
+existing loss/threshold until real-data diagnostics justify changing them.
+
+New learning.py shares the actual objective for ongoing training and disposable
+probe. Per-job before/after evaluation on <=96 unchanged training examples, full
+held-out score distributions, threshold sweep, training-only frequency and no-op
+baselines, decoded pre-server-limit parameter error, gradient norms and sampled
+versus natural frequencies. Bounded <=20s/80-step memorization probe on <=12
+training examples at first job/every 1000 steps; copy only, never published or
+used for generalization gate. Continues current checkpoint/optimizer lineage.
+
+Audit results (synthetic only):
+- Round-tripped eight representative retained command payloads, including signed
+  drive/servo values, zero-speed drive, headlight and all four mode commands.
+- Lossless 25 FPS recording with 31 command events: old sampler labeled 0; new
+  sampler labeled 31/31, including interleaved ignored sensorStream events.
+- Full production train.py decoded five generated recordings, trained/exported
+  checkpoints, evaluated held-out sessions, and resumed optimizer/weights for a
+  second job. 120 -> 240 cumulative steps improved recall at unchanged 0.7 from
+  0 to 0.2553, with precision 1.0 at step 240. Mean true-event score rose 0.5346
+  -> 0.6303. Gate correctly remained failed. A lower 0.5 threshold had precision
+  0.6629/recall 0.8369 and 120 false-positive actions, so threshold was NOT lowered.
+- Disposable diverse probe covered zero-speed drive, moving drive and servo;
+  reached recall 0.75 after 39 steps under its 20s budget. Sampling was corrected
+  to avoid probing only the first event category. Probe uses a copy and higher
+  learning rate for memorization; never adopts weights or changes main optimizer.
+- Exported trained checkpoint reload, forward inference and decoding passed.
+- Retained recordings had extra removed actuator fields, causing false model
+  incompatibility. Normalize saved and current profiles through controlProfile.
+  Direct Node exercise accepts removed laser/peripheral differences and still
+  rejects a changed retained camera configuration.
+
+Diagnostics now report accepted vs represented command counts, request/dispatch
+delay, maximum sensor/image age; sampled vs natural command frequencies; gradient
+norms; same training-subset before/after scores; per-command score distributions;
+threshold sweep and no-op/training-frequency baselines; pre-server-limit decoded
+parameter errors; and bounded diverse memorization probe. Evaluation-step identity
+separates last evaluation from current-job progress. Compact history avoids
+repeating entire command/score tables for every job. Four concise numeric rows
+were added to expanded details; full evidence is in Copy diagnostics.
+
+The original loss and configured 0.7 threshold remain unchanged because experiments
+did not establish that a different objective/threshold fixes real recordings.
+Promotion keeps existing checks and additionally must beat the simple training-
+frequency baseline F1. One-time retry of old reader-specific no-aligned-window
+rejections preserves optimizer state and unrelated errors.
+
+Limits: actual human footage and rover/RTSP transport unavailable. Isolated held-out
+windows cannot measure sequential latch behavior or physical driving; decoded
+parameter errors are explicitly before Node's rover limits/permission/safety
+handling. Main model learned some synthetic cues but has not demonstrated useful
+autonomous driving. Camera command encoding still uses whole-degree predictions.
+The user must not be asked to drive prescribed routes or perform a manual checklist.
+
+Verification: full synthetic worker jobs and checkpoint inference above passed;
+focused Node syntax/card ESLint/diff checks passed before final report additions.
+One experimental seeded run failed because its runpy launcher omitted worker
+import path; corrected launcher succeeded. No production import failure occurred.
+Synthetic inference-worker media exercise passed: eight prediction batches,
+threshold update to 0.3, retained command types and discrete latching exercised.
+This check used a generated checkpoint and a FIFO media adapter preserving
+synthetic PTS (RTSP wall-clock override disabled in the experiment only); it does
+not validate real network capture timing. Trained checkpoint forward/decode was
+verified separately. Temporary media launchers needed fixes for file lifetime
+and command-string escaping; these were experiment issues, not production fixes.
+No repository tests added. Experiments finished and no background worker remains.
+Final focused Node syntax and diff whitespace checks passed. Card lint passed.
+No additional schema/optimizer reset introduced by this audit.

server/src/configuration/definition.js

@@ -13,6 +13,7 @@ const audioForward = require('../services/audioForwardService/configuration');
 const audioLevels = require('../services/audioLevelsService/configuration');
 const homeAssistant = require('../services/homeAssistantService/configuration');
 const homeAssistantActivities = require('../services/homeAssistantActivitiesService/configuration');
+const roomAudio = require('../services/roomAudioService/configuration');
 const roomCameras = require('../services/roomCameraService/configuration');
 const ptzCamera = require('../services/ptzCameraService/configuration');
 const kinect = require('../services/kinectService/configuration');
@@ -46,6 +47,7 @@ const definitions = [
   // Activities own a separate catalog even though the HA connection is shared.
   homeAssistantActivities,
   roomCameras,
+  roomAudio,
   ptzCamera,
   kinect,
   balanceBoard,

server/src/configuration/migrations.js

@@ -1,6 +1,6 @@
 // Configuration Database Migrations
 // Purpose: Applies ordered, transactional schema changes to the configuration and administration database.
-// Scope: Owns database structure only; configuration-document evolution belongs to the ordered definition and validation.
+// Scope: Upgrades database structure and stored configuration documents before current-schema validation.
 const migrations = [
   {
     version: 1,
@@ -84,6 +84,34 @@ const migrations = [
       `).run();
     },
   },
+  {
+    version: 4,
+    run(db) {
+      const rows = db.prepare('SELECT id, config_json FROM configuration_revisions').all();
+      const update = db.prepare('UPDATE configuration_revisions SET config_json = ? WHERE id = ?');
+      // Upgrade history as well as the active revision so a settings rollback
+      // cannot reintroduce fields rejected by the new Automations schema.
+      for (const row of rows) {
+        const config = JSON.parse(row.config_json);
+        let changed = false;
+        for (const item of config.homeAssistantActivities?.items || []) {
+          const hasAction = Object.hasOwn(item, 'idleAction');
+          const hasValue = Object.hasOwn(item, 'idleValue');
+          if (!hasAction && !hasValue) continue;
+          item.automations ??= {};
+          item.automations.idle ??= {};
+          // Preserve any explicitly saved new-format setting when both formats
+          // exist, including an empty text value used to clear an entity.
+          if (hasAction && !Object.hasOwn(item.automations.idle, 'action')) item.automations.idle.action = item.idleAction;
+          if (hasValue && !Object.hasOwn(item.automations.idle, 'value')) item.automations.idle.value = item.idleValue;
+          delete item.idleAction;
+          delete item.idleValue;
+          changed = true;
+        }
+        if (changed) update.run(JSON.stringify(config), row.id);
+      }
+    },
+  },
 ];
 
 function applySchemaMigrations(db) {

server/src/services/authService/index.js

@@ -124,7 +124,7 @@ function grantExternalSpectatorAccessAfterAdminLogin(socket) {
 }
 
 io.on('connection', (socket) => {
-  const requestedRole = socket.handshake?.query?.role;
+  const requestedRole = socket.handshake?.auth?.role || socket.handshake?.query?.role;
   /*
     Role is assigned before the browser's full identity heartbeat has completed.
     For admin-gated external spectators, fail closed here; the spectator page can
@@ -188,4 +188,5 @@ module.exports = {
   isAdmin,
   isLockdownAdmin,
   authenticate,
+  canBecomeSpectator,
 };

server/src/services/backupRestoreService/backup.js

@@ -109,6 +109,7 @@ async function copyDurableTree(sourceDir, destinationDir, relativeDir = '') {
     const relativePath = path.join(relativeDir, entry.name);
     if (!relativeDir && EXCLUDED_TOP_LEVEL_NAMES.has(entry.name)) continue;
     if (!relativeDir && DATABASE_FILES.has(entry.name)) continue;
+    if (relativePath === path.join('rover-learning', 'recordings')) continue;
     if (entry.isSymbolicLink()) throw new Error(`Backup cannot include symbolic link: ${relativePath}`);
     if (entry.isDirectory()) {
       skipped.push(...await copyDurableTree(sourceDir, destinationDir, relativePath));

server/src/services/chatService/textCommands.js

@@ -91,9 +91,9 @@ function createWebReplayTextCommand(socket, sendSystemMessage, replayApi) {
 
       const requester = buildRequesterLabel(socket);
       const jobId = buildReplayJobId('web-chat');
-      const attempt = replayApi.tryTriggerReplay({ by: { source: 'web-chat', requester } });
+      const attempt = replayApi.tryTriggerReplay({ jobId, by: { source: 'web-chat', requester } });
       if (!attempt.ok) {
-        await message.reply({ content: `Replay denied: cooldown active. Try again in ${Math.ceil(attempt.remainingMs / 1000)}s.` });
+        await message.reply({ content: `Replay denied: ${attempt.error}` });
         return;
       }
 

server/src/services/commandService/index.js

@@ -85,7 +85,7 @@ function normalizeOutboundCommandPayload(payload = {}) {
   };
 }
 
-function issueCommand(roverId, payload) {
+function issueCommand(roverId, payload, options = {}) {
   const record = roverManager.rovers.get(roverId);
   if (!record || !record.ws) {
     throw new Error('Rover offline');
@@ -93,6 +93,11 @@ function issueCommand(roverId, payload) {
   if (payload?.type === 'laser' && isLaserCommandBlockedByRoomLightLock(payload)) {
     throw new Error('Laser disabled while room lights are locked on');
   }
+  if (options.dockingOwned && payload.driveDirect && getCommandMotionMagnitude('drive', payload) > 0
+    && Date.now() < (driveCooldowns.get(roverId) || 0)) {
+    throw new Error('Drive blocked: safety cooldown');
+  }
+  payload = require('../dockingService').protectCommand(record, payload, options);
   const id = uuidv4();
   const normalizedPayload = normalizeOutboundCommandPayload(payload);
   const message = { ...normalizedPayload, id };
@@ -124,7 +129,14 @@ overcurrentProtectionService.configureCommandIssuer((roverId, payload) => {
     */
     return false;
   }
-  issueCommand(roverId, payload);
+  const record = roverManager.rovers.get(roverId);
+  const reverseAdjustment = payload.driveDirect
+    && (payload.driveDirect.left < 0 || payload.driveDirect.right < 0)
+    && payload.driveDirect.left <= 0 && payload.driveDirect.right <= 0
+    && require('../dockingService').canAdjustReverse(record);
+  // Only protection's adjustments to the active reverse inherit controller
+  // ownership. Zero remains an ordinary stop and cancels the sequence.
+  issueCommand(roverId, payload, { dockingOwned: Boolean(reverseAdjustment) });
   return true;
 });
 
@@ -315,6 +327,22 @@ function submitCommand(socket, { roverId, type, data } = {}, cb) {
     if (!isSongCommand && !isRebootCommand && !isUpdateCommand && !roverManager.canDrive(roverId, socket)) {
       throw new Error('Not your turn or no control');
     }
+    const record = roverManager.rovers.get(roverId);
+    if (!record?.ws) throw new Error('Rover offline');
+    const docking = require('../dockingService');
+    // The controller owns wheel output during undocking. Discard all ordinary
+    // drive input, including neutral, before it can alter protection's intent.
+    if (docking.shouldIgnoreDriveInput(record, payload)) {
+      reply({ ignored: true, reason: 'undocking' });
+      return;
+    }
+    // Reject before recording held motor intent: otherwise a rejected input
+    // could be replayed by protection as soon as the docking lock is released.
+    if (!docking.intents.has(type)) {
+      const guarded = docking.protectCommand(record, { ...payload, type });
+      type = guarded.type;
+      payload = guarded;
+    }
     const driveDirect = payload?.driveDirect;
     if (type === 'drive' && driveDirect) {
       if (!isAdminSocket) {
@@ -361,7 +389,14 @@ function submitCommand(socket, { roverId, type, data } = {}, cb) {
         bypassed: isAdminSocket,
       });
     }
-    const id = issueCommand(roverId, { type, ...payload });
+    const isDockingIntent = docking.intents.has(type);
+    const id = isDockingIntent
+      ? docking.issueIntent(roverManager.rovers.get(roverId), type, payload, socket)
+      : issueCommand(roverId, { ...payload, type });
+    if (isDockingIntent) {
+      // The server acknowledges accepting the intent; telemetry reports its completion.
+      io.emit('commandAck', { roverId, id, status: 'ok' });
+    }
     commandEvents.emit('observation', {
       ts: Date.now(),
       roverId: String(roverId),

server/src/services/discordBotService/commands/replay.js

@@ -1,6 +1,6 @@
 // Discord Replay Command
 // Purpose: Handles replay capture requests from Discord through the shared replay job/status workflow.
-// Scope: Resolves sources, enforces cooldowns, reports job progress, uploads video, and broadcasts media URLs.
+// Scope: Resolves sources, enforces replay availability, reports job progress, uploads video, and broadcasts media URLs.
 const { AttachmentBuilder } = require('discord.js');
 const io = require('../../../globals/io');
 const { hostReplay } = require('../../replayMediaService');
@@ -60,12 +60,6 @@ function createReplayCommand({
       return;
     }
 
-    const attempt = tryTriggerReplay({ by: message.author?.id || null, source: 'discord' });
-    if (!attempt.ok) {
-      await replyDenied(message, `Replay denied: cooldown active. Try again in ${Math.ceil(attempt.remainingMs / 1000)}s.`);
-      return;
-    }
-
     const requester = message.member?.nickname || message.author?.globalName || message.author?.username || 'Discord';
     const job = createReplayJob({
       id: buildReplayJobId('discord'),
@@ -74,17 +68,23 @@ function createReplayCommand({
       sources: resolved.sources || [],
       includeSidebar: true,
     });
-    jobStatus.emit(job, 'accepted', { message: buildAcceptedMessage(job) });
-
-    const progressMessage = await message.reply({
-      content: sanitizeMentions(buildAcceptedMessage(job)),
-      allowedMentions: DEFAULT_ALLOWED_MENTIONS,
-    });
-    const stopTyping = startDiscordTypingLoop(message.channel, logger, 'discord replay command');
+    const attempt = tryTriggerReplay({ jobId: job.id, by: { source: 'discord', requester } });
+    if (!attempt.ok) {
+      await replyDenied(message, `Replay denied: ${attempt.error}`);
+      return;
+    }
 
+    let progressMessage = null;
+    let stopTyping = () => {};
     let builtReplay = null;
     let deliveredMedia = null;
     try {
+      jobStatus.emit(job, 'accepted', { message: buildAcceptedMessage(job) });
+      progressMessage = await message.reply({
+        content: sanitizeMentions(buildAcceptedMessage(job)),
+        allowedMentions: DEFAULT_ALLOWED_MENTIONS,
+      });
+      stopTyping = startDiscordTypingLoop(message.channel, logger, 'discord replay command');
       jobStatus.emit(job, 'building', { message: buildStatusMessage(job, 'building') });
       if (progressMessage?.edit) {
         await progressMessage.edit({ content: sanitizeMentions(buildStatusMessage(job, 'building')), allowedMentions: DEFAULT_ALLOWED_MENTIONS });

server/src/services/dockingService/index.js

@@ -0,0 +1,302 @@
+// Server-owned docking operations.
+const { v4: uuidv4 } = require('uuid');
+const io = require('../../globals/io');
+const logger = require('../../globals/logger').child('dockingService');
+const states = new WeakMap();
+const intents = new Set(['enterDocking', 'cancelDocking', 'undock']);
+const ASSIST_SPEED = 60;
+const SENSOR_TIMEOUT_MS = 1500;
+const UNDOCK_TIMEOUT_MS = 4000;
+const REVERSE_TIMEOUT_MS = 1000;
+// Preserve the working OI preparation sequence; only backing away is sensor-driven.
+const MODE_COMMAND_DELAY_MS = 100;
+const FULL_MODE_SETTLE_MS = 300;
+
+function stateFor(record) {
+  if (!states.has(record)) {
+    states.set(record, {
+      contact: null, charging: false, latched: false, assist: false,
+      operation: null, error: null, seenAt: 0, timer: null, contactDockPending: false,
+    });
+  }
+  return states.get(record);
+}
+
+function snapshot(record) {
+  const state = stateFor(record);
+  return {
+    phase: state.operation ? 'undocking' : state.contact ? (state.charging ? 'charging' : 'docked') : state.assist ? 'assisting' : 'idle',
+    movementLocked: Boolean(state.operation || state.latched || state.error || state.contact == null),
+    cameraLocked: state.assist,
+    error: state.error,
+  };
+}
+
+function publish(record) {
+  if (!record.lastSensor?.decoded) return;
+  record.lastSensor.decoded = { ...record.lastSensor.decoded, docking: snapshot(record) };
+  // State transitions use the same telemetry event, including failures when the
+  // physical sensor stream has stalled. Never manufacture a new sensor timestamp.
+  io.to(record.room).emit('sensorFrame', {
+    roverId: record.id, frame: record.lastSensor.raw, sensors: record.lastSensor.decoded,
+    overcurrentProtection: record.lastOvercurrentProtection || null,
+  });
+}
+
+function send(record, payload) {
+  return require('../commandService').issueCommand(record.id, payload, { dockingOwned: true });
+}
+function stop(record) {
+  // Forget the previous held intent so protection cannot replay it after unlock.
+  require('../overcurrentProtectionService').protectCommand(record.id, 'drive', { driveDirect: { left: 0, right: 0 } });
+  send(record, { type: 'drive', driveDirect: { left: 0, right: 0 } });
+}
+function stopAllMotion(record) {
+  stop(record);
+  const payload = { motorPwm: { main: 0, side: 0, vacuum: 0 } };
+  require('../overcurrentProtectionService').protectCommand(record.id, 'motors', payload);
+  send(record, { type: 'motors', ...payload });
+}
+function mode(record, opcode) {
+  send(record, { type: 'raw', raw: Buffer.from([opcode]).toString('base64') });
+}
+function camera(record, angle) {
+  if (record.meta?.cameraServo?.enabled) send(record, { type: 'servo', servo: { angle } });
+}
+function song(record, notes, slot = 0, duration = 10) {
+  send(record, { type: 'song', song: { slot, notes: notes.map((note) => ({ note, duration })) } });
+}
+function finishAssist(record, { playExitSong = true } = {}) {
+  const state = stateFor(record);
+  if (!state.assist) return;
+  state.assist = false;
+  camera(record, 0);
+  if (playExitSong) song(record, [83, 76]);
+}
+function clearOperation(state) {
+  clearTimeout(state.timer);
+  state.timer = null;
+  state.operation = null;
+}
+function fail(record, code, message) {
+  const state = stateFor(record);
+  clearOperation(state);
+  state.error = { code, message };
+  state.latched = true;
+  try { stop(record); } catch (error) {
+    logger.warn('Unable to stop docking operation', { roverId: record.id, error: error.message });
+  }
+  publish(record);
+}
+function armDeadline(record) {
+  const state = stateFor(record);
+  clearTimeout(state.timer);
+  if (!state.operation) return;
+  const deadline = Math.min(
+    state.seenAt + SENSOR_TIMEOUT_MS,
+    state.operation.startedAt + UNDOCK_TIMEOUT_MS,
+    state.operation.reversingAt ? state.operation.reversingAt + REVERSE_TIMEOUT_MS : Infinity,
+  );
+  state.timer = setTimeout(() => {
+    const stale = Date.now() - state.seenAt >= SENSOR_TIMEOUT_MS;
+    fail(record, stale ? 'telemetryUnavailable' : 'undockTimeout', stale ? 'Rover telemetry stopped during undocking.' : 'Unable to leave the dock.');
+  }, Math.max(1, deadline - Date.now()));
+  state.timer.unref?.();
+}
+
+function handleIntent(record, type, payload, socket) {
+  const state = stateFor(record);
+  if (type === 'cancelDocking') {
+    if (state.operation) fail(record, 'undockCancelled', 'Undocking stopped.');
+    stop(record);
+    finishAssist(record);
+    publish(record);
+    return;
+  }
+  state.error = null;
+  state.contactDockPending = false;
+  require('../roverManager').stopDockGuard(record.id);
+  stop(record);
+  if (type === 'enterDocking') {
+    state.latched = false;
+    if (!state.assist) {
+      state.assist = true;
+      camera(record, record.meta?.cameraServo?.minAngle ?? -45);
+      song(record, [76, 83]);
+    }
+  } else {
+    finishAssist(record);
+    camera(record, 0);
+    state.operation = {
+      startedAt: Date.now(), socket, backoff: payload.backoff !== false, reversing: false,
+      stage: 'start', nextStepAt: Date.now() + MODE_COMMAND_DELAY_MS,
+    };
+    mode(record, 128);
+    armDeadline(record);
+  }
+  publish(record);
+}
+
+function processTelemetry(record, sensors) {
+  const state = stateFor(record);
+  if (!sensors?.chargingSources || !sensors?.oiMode) return snapshot(record);
+  const previousContact = state.contact;
+  state.seenAt = Date.now();
+  state.contact = Boolean(sensors.chargingSources.homeBase);
+  state.charging = [1, 2, 3, 4].includes(sensors.chargingState?.code);
+  if (previousContact == null) state.latched = state.contact;
+  try {
+    const operation = state.operation;
+    if (operation) {
+      const manager = require('../roverManager');
+      if (!operation.socket?.connected || !manager.canDrive(record.id, operation.socket)) {
+        fail(record, 'controlLost', 'Undocking stopped because control changed.');
+      } else if (Date.now() - operation.startedAt >= UNDOCK_TIMEOUT_MS
+        || (operation.reversingAt && Date.now() - operation.reversingAt >= REVERSE_TIMEOUT_MS)) {
+        fail(record, 'undockTimeout', 'Unable to leave the dock.');
+      } else if (Date.now() < operation.nextStepAt) {
+        // Advance only on fresh telemetry, after the previous OI command settles.
+      } else if (operation.stage === 'start') {
+        mode(record, 143);
+        operation.stage = 'dock';
+        operation.nextStepAt = Date.now() + MODE_COMMAND_DELAY_MS;
+      } else if (operation.stage === 'dock') {
+        mode(record, 132);
+        operation.stage = 'full';
+        operation.nextStepAt = Date.now() + FULL_MODE_SETTLE_MS;
+      } else if (sensors.oiMode.label === 'full') {
+        if (!state.contact || !operation.backoff) {
+          stop(record);
+          clearOperation(state);
+          state.latched = state.contact;
+        } else if (!operation.reversing) {
+          operation.reversing = true;
+          operation.reversingAt = Date.now();
+          const speed = Math.max(1, Math.min(500, Number(record.meta?.maxWheelSpeed) || 500));
+          // Track the controller's own intent in existing protection so stalls
+          // still stop it and subsequent adjustments refer to this reverse only.
+          const drive = require('../overcurrentProtectionService').protectCommand(record.id, 'drive', {
+            driveDirect: { left: -speed, right: -speed },
+          }, { bypassed: require('../roleService').isAdmin(operation.socket) });
+          if (!drive.driveDirect.left && !drive.driveDirect.right) throw new Error('Undocking blocked by wheel protection');
+          send(record, { ...drive, type: 'drive' });
+        }
+      } else if (operation.reversing) {
+        fail(record, 'modeChanged', 'Undocking stopped because the Roomba mode changed.');
+      }
+      armDeadline(record);
+    } else if (state.contact) {
+      state.latched = true;
+      if (previousContact !== true) {
+        stopAllMotion(record);
+        state.contactDockPending = true;
+        if (state.assist) song(record, [84], 1, 6);
+      } else if (state.contactDockPending && !state.charging) {
+        const speeds = sensors.wheelSpeedsMmPerSecond;
+        // A stop request does not prove the wheels stopped on the contacts.
+        // Require a subsequent frame with contact and measured stationary wheels.
+        if (speeds?.left === 0 && speeds?.right === 0) {
+          mode(record, 143);
+          state.contactDockPending = false;
+        }
+      }
+      if (state.charging) {
+        state.contactDockPending = false;
+        // An immediate exit song would replace the contact beep on the Roomba.
+        finishAssist(record, { playExitSong: false });
+      }
+    } else {
+      state.contactDockPending = false;
+      if (previousContact === true && state.assist) song(record, [72], 1, 6);
+    }
+  } catch (error) {
+    fail(record, 'commandFailed', error.message);
+  }
+  return snapshot(record);
+}
+
+function shouldIgnoreDriveInput(record, payload) {
+  if (!stateFor(record).operation) return false;
+  if (payload.driveDirect) return true;
+  const raw = payload.raw ? Buffer.from(payload.raw, 'base64') : null;
+  return Boolean(raw && [137, 145].includes(raw[0]));
+}
+
+function protectCommand(record, payload, { dockingOwned = false } = {}) {
+  if (dockingOwned) return payload;
+  const state = stateFor(record);
+  // Raw OI drive packets must obey the same movement lock. Other raw commands,
+  // including mode changes, remain available and can interrupt the operation.
+  const raw = payload.raw ? Buffer.from(payload.raw, 'base64') : null;
+  if (state.operation && raw && [128, 131, 132, 133, 134, 135, 136, 143].includes(raw[0])) {
+    // Explicit mode commands remain available and take precedence over an
+    // automatic sequence, including while it is still waiting for full mode.
+    fail(record, 'modeChanged', 'Undocking stopped by a Roomba mode command.');
+  }
+  if (raw && [137, 145].includes(raw[0])) {
+    if (raw.length !== 5) throw new Error('Invalid raw drive command');
+    const first = raw.readInt16BE(1);
+    const second = raw.readInt16BE(3);
+    if (raw[0] === 137 && first !== 0 && state.assist && !snapshot(record).movementLocked) {
+      const limited = Buffer.from(raw);
+      limited.writeInt16BE(Math.max(-ASSIST_SPEED, Math.min(ASSIST_SPEED, first)), 1);
+      return { ...payload, raw: limited.toString('base64') };
+    }
+    if (raw[0] === 145) {
+      return protectCommand(record, { type: 'drive', driveDirect: { left: second, right: first } });
+    }
+    if (first === 0 && state.operation) fail(record, 'undockStopped', 'Undocking stopped by a stop command.');
+    if (first !== 0 && snapshot(record).movementLocked) throw new Error('Drive blocked while docked or undocking');
+  }
+  // User drive input is discarded at submitCommand before reaching here.
+  // Internal safety stops still interrupt the controller through this path.
+  if (payload.driveDirect) {
+    const { left, right } = payload.driveDirect;
+    if (!Number(left) && !Number(right) && state.operation) {
+      fail(record, 'undockStopped', 'Undocking stopped by a stop command.');
+    }
+    if ((Number(left) || Number(right)) && snapshot(record).movementLocked) throw new Error('Drive blocked while docked or undocking');
+    if (state.assist) {
+      const scale = Math.min(1, ASSIST_SPEED / Math.max(Math.abs(left), Math.abs(right), 1));
+      return { ...payload, driveDirect: { left: Math.round(left * scale), right: Math.round(right * scale) } };
+    }
+  }
+  if (payload.servo && state.assist) throw new Error('Camera locked during dock assist');
+  return payload;
+}
+
+function disconnect(record) {
+  const state = stateFor(record);
+  if (state.operation) fail(record, 'roverDisconnected', 'Rover disconnected during undocking.');
+  clearOperation(state);
+  state.seenAt = 0;
+  state.contact = null;
+  state.assist = false;
+}
+
+function canAdjustReverse(record) {
+  return Boolean(record && stateFor(record).operation?.reversing);
+}
+
+function issueIntent(record, type, payload, socket) {
+  const state = stateFor(record);
+  // Validate before mutating, so duplicate/rejected requests cannot disturb an
+  // operation that is already in progress.
+  if (type !== 'cancelDocking') {
+    if (state.operation) throw new Error('Undocking is already in progress');
+    if (Date.now() - state.seenAt >= SENSOR_TIMEOUT_MS || state.contact == null) {
+      throw new Error('Waiting for current rover telemetry');
+    }
+    // Already at the requested destination; acknowledge without restarting assist.
+    if (type === 'enterDocking' && state.contact) return uuidv4();
+  }
+  const id = uuidv4();
+  try {
+    handleIntent(record, type, payload, socket);
+  } catch (error) {
+    fail(record, 'commandFailed', error.message);
+    throw error;
+  }
+  return id;
+}
+module.exports = { shouldIgnoreDriveInput, canAdjustReverse, snapshot, intents, issueIntent, processTelemetry, protectCommand, disconnect };

server/src/services/embedHttpService/index.js

@@ -16,7 +16,7 @@ const { renderIndexHtml, renderOgImage, renderWebManifest } = require('../embedS
   a direct browser load. Keeping the setup and admin routes in this explicit
   allowlist prevents them from working only after client-side navigation.
 */
-app.get(['/', '/old', '/spectate', '/mini', '/display', '/scanner', '/database', '/ptz', '/reports', '/setup', '/admin'], async (req, res) => {
+app.get(['/', '/old', '/spectate', '/mini', '/display', '/scanner', '/database', '/ptz', '/reports', '/setup', '/admin', '/mic'], async (req, res) => {
   try {
     const html = await renderIndexHtml(req);
     res.type('html').send(html);

server/src/services/homeAssistantActivitiesService/actions.js

@@ -1,5 +1,4 @@
-// User and idle commands share one dispatch contract, while only the trusted
-// idle entry point can bypass per-item user locks.
+// User and automated commands share validation; only trusted automation bypasses user permissions.
 const { buildEntity, buildCommand } = require('./entityHelpers');
 
 function assertAccess(actor = {}) {
@@ -9,13 +8,14 @@ function assertAccess(actor = {}) {
 }
 
 function createActions({ getConfig, ha, locks }) {
-  async function execute(id, value, actor, idle = false) {
-    if (!idle) assertAccess(actor);
+  async function execute(id, value, actor, automated = false) {
+    if (!automated) assertAccess(actor);
     const config = getConfig();
     if (!config.enabled) throw new Error('Activity Controls are disabled');
     const item = config.items.find((entry) => entry.id === id);
     if (!item) throw new Error('Unknown activity item');
-    if (!idle && locks.isLocked(id) && !['admin', 'lockdown'].includes(actor.role)) throw new Error('This item is locked');
+    if (!automated && locks.isLocked(id) && !['admin', 'lockdown'].includes(actor.role)) throw new Error('This item is locked');
+    if (!automated && item.readOnly && !['admin', 'lockdown'].includes(actor.role)) throw new Error('This item is read-only for users');
     if (!ha.enabled || !ha.isConnected()) throw new Error('Home Assistant is offline');
     const entity = buildEntity(item, ha.getRawEntitySnapshot(id));
     if (!entity.available) throw new Error('This item is unavailable');
@@ -25,30 +25,34 @@ function createActions({ getConfig, ha, locks }) {
     await ha.callHomeAssistantService(entity.domain, command.service, command.data);
   }
 
-  async function runIdleActions() {
+  async function runAutomation(trigger, shouldRun = () => true) {
     const config = getConfig();
-    if (!config.enabled) return { action: 'homeAssistantActivitiesIdle', skipped: true };
+    if (!config.enabled) return { action: trigger, skipped: true };
     const results = [];
     // Catch per item so one offline integration cannot prevent the remaining
-    // configured cleanup actions from running during this idle window.
+    // configured actions from running for this trigger.
     for (const item of config.items) {
-      if (!item.idleAction || item.idleAction === 'unchanged' || item.readOnly) continue;
+      // Stop remaining writes if presence or mode changed while HA was responding.
+      if (!shouldRun()) break;
+      const setting = item.automations?.[trigger];
+      if (!setting?.action || setting.action === 'unchanged') continue;
       try {
         const entity = buildEntity(item, ha.getRawEntitySnapshot(item.id));
         if (entity.type === 'readOnly') continue;
-        if ((item.idleAction === 'press') !== (entity.type === 'button')) throw new Error('Idle action does not match the control type');
+        if ((setting.action === 'press') !== (entity.type === 'button')) throw new Error('Automation action does not match the control type');
         // The admin form omits an empty optional string. Treat that as empty
         // text so clearing a message works; other types still reject it through
         // their normal value validation rather than silently receiving zero.
-        await execute(item.id, item.idleAction === 'press' ? 'press' : (item.idleValue ?? ''), null, true);
+        await execute(item.id, setting.action === 'press' ? 'press' : (setting.value ?? ''), null, true);
         results.push({ id: item.id, ok: true });
       } catch (error) {
         results.push({ id: item.id, ok: false, error: error.message });
       }
     }
-    return { action: 'homeAssistantActivitiesIdle', results };
+    return { action: trigger, results };
   }
-  return { act: (id, value, actor) => execute(id, value, actor), runIdleActions };
+  return { act: (id, value, actor) => execute(id, value, actor), runAutomation,
+    runIdleActions: () => runAutomation('idle') };
 }
 
 module.exports = { createActions, assertAccess };

server/src/services/homeAssistantActivitiesService/activities.test.js

@@ -29,7 +29,7 @@ test('sensor states, live constraints, and unknown button states are preserved',
   assert.equal(buildEntity({ id: 'button.bell' }, raw('unknown')).available, true);
   assert.equal(buildEntity({ id: 'button.bell' }, raw('unavailable')).available, false);
   assert.equal(buildEntity({ id: 'button.bell' }, null).available, false);
-  assert.equal(buildEntity({ id: 'switch.fan', readOnly: true }, raw('on')).type, 'readOnly');
+  assert.equal(buildEntity({ id: 'switch.fan', readOnly: true }, raw('on')).type, 'toggle');
 });
 
 test('domain dispatch handles native entities and helpers without room-light payloads', async () => {
@@ -87,22 +87,22 @@ test('permissions, locks, availability and allowlist are authoritative', async (
 test('idle ignores user locks, preserves no-op items, and isolates failures', async () => {
   const h = harness([
     { id: 'switch.keep' },
-    { id: 'number.bad', idleAction: 'set', idleValue: '200' },
-    { id: 'switch.fan', idleAction: 'set', idleValue: 'off' },
+    { id: 'number.bad', automations: { idle: { action: 'set', value: '200' } } },
+    { id: 'switch.fan', automations: { idle: { action: 'set', value: 'off' } } },
     // An empty optional text box is omitted by the schema-driven admin form.
-    { id: 'input_text.message', idleAction: 'set' },
-    { id: 'input_button.stop', idleAction: 'press' },
-    { id: 'sensor.humidity', idleAction: 'set', idleValue: '0' },
-    { id: 'switch.readonly', readOnly: true, idleAction: 'set', idleValue: 'off' },
+    { id: 'input_text.message', automations: { idle: { action: 'set' } } },
+    { id: 'input_button.stop', automations: { idle: { action: 'press' } } },
+    { id: 'sensor.humidity', automations: { idle: { action: 'set', value: '0' } } },
+    { id: 'switch.readonly', readOnly: true, automations: { idle: { action: 'set', value: 'off' } } },
   ], {
-    'number.bad': raw('0', { min: 0, max: 10 }), 'switch.fan': raw('on'),
+    'switch.readonly': raw('on'), 'number.bad': raw('0', { min: 0, max: 10 }), 'switch.fan': raw('on'),
     'input_text.message': raw('Welcome', { min: 0, max: 30 }), 'input_button.stop': raw('unknown'),
   });
   h.locked.add('switch.fan');
   const result = await h.actions.runIdleActions();
   assert.equal(result.results[0].ok, false);
-  assert.equal(result.results.filter((entry) => entry.ok).length, 3);
-  assert.deepEqual(h.calls.map((call) => call[2]), [{ entity_id: 'switch.fan' }, { entity_id: 'input_text.message', value: '' }, { entity_id: 'input_button.stop' }]);
+  assert.equal(result.results.filter((entry) => entry.ok).length, 4);
+  assert.deepEqual(h.calls.map((call) => call[2]), [{ entity_id: 'switch.fan' }, { entity_id: 'input_text.message', value: '' }, { entity_id: 'input_button.stop' }, { entity_id: 'switch.readonly' }]);
 });
 
 test('an outstanding service response does not block later commands', async () => {

server/src/services/homeAssistantActivitiesService/configuration.js

@@ -2,6 +2,14 @@
 // never acquire unrelated devices simply because they share a connection.
 const { strictObject, string, boolean } = require('../../configuration/schemaHelpers');
 
+// All triggers use the same inputs so the admin form and execution contract agree.
+function automation(title, description) {
+  return strictObject({
+    action: string({ title: 'Action', enum: ['unchanged', 'set', 'press'], default: 'unchanged' }),
+    value: string({ title: 'Value', description: 'For set: on/off, a number, exact selection, or text. Empty text clears it. Ignored for press.', maxLength: 255 }),
+  }, { title, description });
+}
+
 module.exports = {
   key: 'homeAssistantActivities',
   feature: true,
@@ -18,9 +26,12 @@ module.exports = {
         icon: string({ description: 'Font Awesome name, as used by social links. Blank or invalid names use a fallback.', maxLength: 80, examples: ['FaFan'] }),
         // Match social-link colors so admins can customize tiles through the existing config form.
         color: string({ description: 'Optional six-digit hexadecimal tile color, like social links.', examples: ['#3B82F6'], pattern: '^#[0-9a-fA-F]{6}$' }),
-        readOnly: boolean({ title: 'Read only', default: false, description: 'Display the value without allowing user or idle commands.' }),
-        idleAction: string({ title: 'When idle', enum: ['unchanged', 'set', 'press'], default: 'unchanged', description: 'Leave unchanged, set the idle value, or press a button once. Runs independently of user locks.' }),
-        idleValue: string({ title: 'Idle value', description: 'For set: on/off, a number, exact selection, or text (empty text clears it). Checked against live entity limits when idle runs.', examples: ['0'], maxLength: 255 }),
+        readOnly: boolean({ title: 'Read only for users', default: false, description: 'Ordinary users can only view this item. Admins and automations can still change supported controls.' }),
+        automations: strictObject({
+          idle: automation('When idle', 'Runs after the idle timeout with no users or admins online.'),
+          locked: automation('When server locked', 'Runs on entering admin or lockdown mode.'),
+          online: automation('When users online', 'Runs when a user or admin comes online, or the server reopens with one online. Suppressed in admin and lockdown modes. Spectators do not count.'),
+        }, { title: 'Automations', description: 'One-time actions that bypass item locks and read-only permissions.' }),
       }, { description: 'One independently controlled or read-only activity control item.', required: ['id'] }),
     },
   }, { title: 'Activity Controls', description: 'Generic activity controls and idle behavior, separate from room lighting.', required: ['enabled', 'items'] }),

server/src/services/homeAssistantActivitiesService/entityHelpers.js

@@ -9,7 +9,8 @@ const TYPES = {
 function buildEntity(item, raw, locked = false) {
   const attributes = raw?.attributes || {};
   const domain = item.id.split('.')[0];
-  const type = item.readOnly ? 'readOnly' : TYPES[domain] || 'readOnly';
+  // Read-only is a user permission, not an entity capability: admins and automation still need its write type.
+  const type = TYPES[domain] || 'readOnly';
   // A never-pressed button legitimately reports unknown. Its state is a last
   // press timestamp, not availability or a boolean toggle state.
   const available = Boolean(raw && raw.state !== 'unavailable' && (raw.state !== 'unknown' || type === 'button'));
@@ -17,7 +18,7 @@ function buildEntity(item, raw, locked = false) {
   return {
     id: item.id, name: item.name?.trim() || attributes.friendly_name || item.id,
     // Include presentation settings in session state so all clients share the configured color.
-    icon: item.icon || '', color: item.color || '', domain, type, locked, available,
+    icon: item.icon || '', color: item.color || '', domain, type, readOnly: Boolean(item.readOnly), locked, available,
     state: raw?.state ?? 'unknown', unit: attributes.unit_of_measurement || '',
     min: numeric(attributes.min), max: numeric(attributes.max), step: numeric(attributes.step),
     options: Array.isArray(attributes.options) ? attributes.options.filter((option) => typeof option === 'string') : [],

server/src/services/homeAssistantActivitiesService/index.js

@@ -8,7 +8,7 @@ const { buildEntity } = require('./entityHelpers');
 const { createLocks, resolveItem } = require('./locks');
 const { createActions } = require('./actions');
 const { getRole } = require('../roleService');
-const { getMode } = require('../modeManager');
+const { getMode, modeEvents } = require('../modeManager');
 const logger = require('../../globals/logger').child('homeAssistantActivitiesService');
 
 const events = new EventEmitter();
@@ -24,6 +24,35 @@ const actions = createActions({ getConfig: () => config, ha: {
   callHomeAssistantService: ha.callHomeAssistantService,
 }, locks });
 
+let operatorsOnline = false;
+let lastAutomationState = null;
+let automationRevision = 0;
+
+function refreshAutomation() {
+  const mode = getMode();
+  const locked = mode === 'admin' || mode === 'lockdown';
+  // Remember transitions, not entity values: HA updates and extra connections
+  // must not repeatedly press buttons or undo a manual admin adjustment.
+  const state = config.enabled && snapshotReady && ha.isConnected()
+    ? (locked ? mode : operatorsOnline ? 'online' : null) : null;
+  if (state === lastAutomationState) return;
+  lastAutomationState = state;
+  const revision = ++automationRevision;
+  if (!state) return;
+  actions.runAutomation(locked ? 'locked' : 'online', () => revision === automationRevision)
+    .then((result) => {
+      const failures = result.results?.filter((item) => !item.ok) || [];
+      if (failures.length) logger.warn('Activity automation failed', { trigger: result.action, failures });
+    }).catch((error) => logger.warn('Activity automation failed', { error: error.message }));
+}
+
+function setOperatorsOnline(online) {
+  operatorsOnline = online;
+  refreshAutomation();
+}
+
+modeEvents.on('change', refreshAutomation);
+
 function getState() {
   return {
     enabled: config.enabled,
@@ -53,14 +82,17 @@ function setLocked(query, locked) {
 ha.homeAssistantEvents.on('snapshot', () => {
   snapshotReady = true;
   emitUpdate();
+  refreshAutomation();
 });
 ha.homeAssistantEvents.on('status', () => {
   snapshotReady = false;
   emitUpdate();
+  refreshAutomation();
 });
 registerConfigurationHandler('homeAssistantActivities', (next) => {
   config = next;
   emitUpdate();
+  refreshAutomation();
 });
 // Resolve access on the server for every command; clients only supply item/value.
 io.on('connection', (socket) => {
@@ -76,4 +108,7 @@ io.on('connection', (socket) => {
   });
 });
 
-module.exports = { ...actions, getState, setLocked, activityEvents: events };
+module.exports = { ...actions, getState, setLocked, setOperatorsOnline, activityEvents: events,
+  // A returning operator ends idle cleanup even if earlier HA calls are still settling.
+  runIdleActions: () => actions.runAutomation('idle', () => !operatorsOnline),
+};

server/src/services/idleService/index.js

@@ -4,11 +4,15 @@
 const logger = require('../../globals/logger').child('idleService');
 const io = require('../../globals/io');
 const { getRole, roleEvents } = require('../roleService');
+const { getMode, MODES, modeEvents } = require('../modeManager');
+const { isDeterred, verificationEvents } = require('../verificationService');
 const { IDLE_TIMEOUT_MS } = require('./constants');
 const { runtime } = require('./state');
 const { runIdleActions } = require('./actions');
+const activityControls = require('../homeAssistantActivitiesService');
 
 function getActivitySnapshot() {
+  const mode = getMode();
   let onlineUsers = 0;
   let onlineAdmins = 0;
   let onlineSpectators = 0;
@@ -28,12 +32,16 @@ function getActivitySnapshot() {
       return;
     }
 
-    /*
-      Lockdown admins are counted with regular admins because both represent a
-      person with operator-level access who may be supervising the room without
-      actively driving a rover. Plain users also count even before they request
-      control, which is the behavior this service now needs.
-    */
+
+    if (
+      isDeterred(socket)
+      || (mode === MODES.ADMIN && role !== 'admin' && role !== 'lockdown')
+      || (mode === MODES.LOCKDOWN && role !== 'lockdown')
+    ) {
+      onlineIgnored += 1;
+      return;
+    }
+
     if (role === 'admin' || role === 'lockdown') {
       onlineAdmins += 1;
       return;
@@ -89,7 +97,7 @@ function scheduleIdleTimer() {
     runtime.deadlineAt = null;
     const activity = getActivitySnapshot();
     if (activity.totalActive > 0) {
-      logger.info('Idle automation skipped; user or admin online', activity);
+      logger.info('Idle automation skipped; eligible operator online', activity);
       return;
     }
     runtime.lastTriggeredAt = Date.now();
@@ -113,16 +121,12 @@ function scheduleIdleTimer() {
 function refreshIdleState() {
   const activity = getActivitySnapshot();
   logger.info('Idle state refresh', activity);
+  activityControls.setOperatorsOnline(activity.totalActive > 0);
   if (activity.totalActive > 0) {
     clearIdleTimer();
     if (runtime.idleActionsCompleted) {
       logger.info('Idle action one-shot reset; operator is online again', activity);
     }
-    /*
-      A user/admin coming online starts a new activity window. When the room
-      later becomes idle again, the cleanup pipeline should be allowed to run
-      once for that new idle period.
-    */
     runtime.idleActionsCompleted = false;
     return;
   }
@@ -130,17 +134,13 @@ function refreshIdleState() {
 }
 
 io.on('connection', (socket) => {
-  /*
-    A user/admin can be online without ever touching rover controls, so socket
-    presence has to be a first-class idle signal. The disconnect hook is just as
-    important: it is what starts the idle timeout after the last non-spectator
-    leaves, even if no driving event happens around that departure.
-  */
   refreshIdleState();
   socket.on('disconnect', refreshIdleState);
 });
 
 roleEvents.on('change', refreshIdleState);
+modeEvents.on('change', refreshIdleState);
+verificationEvents.on('change', refreshIdleState);
 
 refreshIdleState();
 

server/src/services/mediaMtxService/config.js

@@ -20,6 +20,7 @@ function normalizeAdditionalHosts(rawHosts) {
 
 function buildMediaMtxConfig({ config, serverPort, snapshotWriterPath }) {
   const media = config?.media || {};
+  const webrtcPort = media.webrtcPort ?? 8189;
   const configuredHosts = normalizeAdditionalHosts(media.additionalHosts);
   let publicHostname = '';
   try {
@@ -55,10 +56,10 @@ function buildMediaMtxConfig({ config, serverPort, snapshotWriterPath }) {
 
     webrtc: true,
     // WHEP and WHIP signaling is public only through the Node /video proxy.
-    // ICE transport on 8189 remains directly reachable by browsers.
+    // The configured ICE transport port remains directly reachable by browsers.
     webrtcAddress: '127.0.0.1:8889',
-    webrtcLocalUDPAddress: ':8189',
-    webrtcLocalTCPAddress: ':8189',
+    webrtcLocalUDPAddress: `:${webrtcPort}`,
+    webrtcLocalTCPAddress: `:${webrtcPort}`,
     webrtcAdditionalHosts: additionalHosts,
     webrtcICEServers2: [
       { url: 'stun:stun.l.google.com:19302' },
@@ -81,6 +82,7 @@ function buildMediaMtxConfig({ config, serverPort, snapshotWriterPath }) {
       { action: 'pprof' },
     ],
     paths: {
+      '~^room-audio/': { source: 'publisher' },
       all: {
         source: 'publisher',
         sourceOnDemand: false,

server/src/services/mediaMtxService/configuration.js

@@ -1,18 +1,24 @@
 // Media Transport Configuration
-// Purpose: Defines additional MediaMTX ICE hosts not already derived from the server's public URL.
+// Purpose: Defines the WebRTC media port and additional ICE hosts.
 // Scope: Contains configuration metadata only and never starts MediaMTX.
-const { strictObject, stringArray } = require('../../configuration/schemaHelpers');
+const { strictObject, stringArray, integer } = require('../../configuration/schemaHelpers');
 
 module.exports = {
   key: 'media',
   defaultValue: {
+    webrtcPort: 8189,
     additionalHosts: [],
   },
   schema: strictObject({
+    webrtcPort: integer({
+      title: 'WebRTC port',
+      description: 'UDP and TCP port for live media. Forward the same external port to this server; HTTPS signaling continues through the web UI.',
+      minimum: 1, maximum: 65535,
+    }),
     additionalHosts: stringArray({
       title: 'Additional ICE hosts',
       item: { description: 'Hostname or IP address MediaMTX advertises as a WebRTC ICE candidate.', examples: ['rover.example.com', 'media-server.local'], minLength: 1, maxLength: 255 },
       array: { description: 'Extra public or LAN hostnames and addresses browsers may use in addition to the hostname derived from the top-level public URL.', uniqueItems: true },
     }),
-  }, { title: 'Media', description: 'Adds optional WebRTC network candidates to the public hostname and local interfaces generated automatically for MediaMTX.', required: ['additionalHosts'] }),
+  }, { title: 'Media', description: 'Configures the WebRTC media port and additional network candidates.', required: ['webrtcPort', 'additionalHosts'] }),
 };

server/src/services/replayDeliveryService/index.js

@@ -35,9 +35,9 @@ async function deliverReplay(payload = {}) {
     requestedBy: payload.requestedBy,
   });
 
-  jobStatus.emit(job, 'accepted', { message: buildAcceptedMessage(job) });
   let providerContext = null;
   try {
+    jobStatus.emit(job, 'accepted', { message: buildAcceptedMessage(job) });
     let providerError = null;
     if (preferredDeliveryProvider?.begin) {
       try {

server/src/services/replayDeliveryService/workflow.js

@@ -2,6 +2,7 @@
 // Purpose: Provides transport-neutral replay jobs, source lookup, status events, and user-facing progress text.
 // Scope: Keeps Discord-command and web-triggered replay delivery on the same core status pipeline.
 const Fuse = require('fuse.js');
+const { updateReplayStatus } = require('../replayEngineV2/cooldown');
 
 const DEFAULT_ALLOWED_MENTIONS = { parse: [], repliedUser: false };
 const FUZZY_THRESHOLD = 0.42;
@@ -180,6 +181,22 @@ function createJobStatusEmitter({ io, logger, sanitizeMentions }) {
 
   function emit(job, status, extra = {}) {
     const payload = buildPayload(job, status, extra);
+    updateReplayStatus(payload);
+    if (status === 'accepted') {
+      // Sound once per participating rover; a disconnected rover must not abort video creation.
+      const { issueCommand } = require('../commandService');
+      const roverIds = new Set(job.sources.filter((entry) => entry.type === 'rover').map((entry) => entry.id));
+      for (const roverId of roverIds) {
+        try {
+          issueCommand(roverId, {
+            type: 'song',
+            song: { notes: [{ note: 72, duration: 8 }, { note: 76, duration: 8 }, { note: 79, duration: 16 }] },
+          });
+        } catch (err) {
+          logger?.warn?.('Unable to play replay tune', { roverId, error: err.message });
+        }
+      }
+    }
     io.emit('replay:status', payload);
     if (status === 'ready' && extra.media) io.emit('replay:ready', extra.media);
     if (status === 'failed') io.emit('replay:failed', payload);

server/src/services/replayEngineV2/cooldown.js

@@ -4,40 +4,61 @@
 const EventEmitter = require('events');
 
 const COOLDOWN_MS = 10 * 1000;
+const MAX_CONCURRENT_JOBS = 3;
 const replayEvents = new EventEmitter();
+const activeJobs = new Map();
+let status = null;
 let lastTriggeredAt = null;
 let lastTriggeredBy = null;
 
-function getRemainingMs(now = Date.now()) {
-  if (!lastTriggeredAt) return 0;
-  return Math.max(0, COOLDOWN_MS - (now - lastTriggeredAt));
-}
-
 function getReplayState() {
-  const remainingMs = getRemainingMs();
+  const remainingMs = lastTriggeredAt === null ? 0 : Math.max(0, COOLDOWN_MS - (Date.now() - lastTriggeredAt));
   return {
     cooldownMs: COOLDOWN_MS,
     lastTriggeredAt,
     lastTriggeredBy,
     remainingMs,
-    available: remainingMs === 0,
+    atJobLimit: activeJobs.size >= MAX_CONCURRENT_JOBS,
+    activeSourceKeys: [...new Set(Array.from(activeJobs.values()).flatMap((sources) =>
+      sources.map((source) => `${source.type}:${source.id}`),
+    ))],
+    status,
+    available: remainingMs === 0 && activeJobs.size < MAX_CONCURRENT_JOBS,
   };
 }
 
-function tryTriggerReplay(by = null) {
-  const remainingMs = getRemainingMs();
-  if (remainingMs > 0) {
-    return { ok: false, remainingMs, state: getReplayState() };
-  }
-  lastTriggeredAt = Date.now();
-  lastTriggeredBy = by || null;
-  const state = getReplayState();
-  replayEvents.emit('update', { state, by: lastTriggeredBy });
-  return { ok: true, state };
+function emitUpdate() {
+  replayEvents.emit('update', { state: getReplayState(), by: lastTriggeredBy });
 }
 
-module.exports = {
-  tryTriggerReplay,
-  getReplayState,
-  replayEvents,
-};
+function tryTriggerReplay({ jobId, by = null } = {}) {
+  const state = getReplayState();
+  if (state.atJobLimit) {
+    return { ok: false, error: 'Replay limit reached: 3 jobs are already being made', state };
+  }
+  if (state.remainingMs > 0) {
+    return { ok: false, error: `Replay cooldown active. Try again in ${Math.ceil(state.remainingMs / 1000)}s.`, remainingMs: state.remainingMs, state };
+  }
+  if (!jobId) throw new Error('Replay job id required');
+  activeJobs.set(jobId, []);
+  status = { jobId, status: 'accepted' };
+  lastTriggeredAt = Date.now();
+  lastTriggeredBy = by;
+  emitUpdate();
+  return { ok: true, state: getReplayState() };
+}
+
+function updateReplayStatus(payload) {
+  if (!activeJobs.has(payload.jobId)) return;
+  if (payload.jobId === status?.jobId) {
+    status = { jobId: payload.jobId, status: payload.status, title: payload.title, message: payload.message };
+  }
+  if (payload.status === 'ready' || payload.status === 'failed') {
+    activeJobs.delete(payload.jobId);
+  } else {
+    activeJobs.set(payload.jobId, payload.sources || []);
+  }
+  emitUpdate();
+}
+
+module.exports = { tryTriggerReplay, getReplayState, replayEvents, updateReplayStatus };

server/src/services/replayEngineV2/index.js

@@ -6,6 +6,7 @@ const { promisify } = require('util');
 const fsp = require('fs/promises');
 const logger = require('../../globals/logger').child('replayEngineV2');
 const roverManager = require('../roverManager');
+const { roomAudioEvents } = require('../roomAudioService');
 const { getRoomCameras, roomCameraEvents } = require('../roomCameraService');
 
 const {
@@ -40,8 +41,8 @@ function getActiveSegmentRoot() {
   return runtime.activeSegmentRoot;
 }
 
-const workerManager = createWorkerManager({ getActiveSegmentRoot });
 const segmentStore = createSegmentStore({ getActiveSegmentRoot });
+const workerManager = createWorkerManager({ getActiveSegmentRoot, onWorkerClosed: segmentStore.refreshIndexForWorker });
 const sidebarRenderer = createSidebarRenderer({ execFileAsync, ensureDir });
 const replayBuilder = createReplayBuilder({
   execFileAsync,
@@ -50,6 +51,7 @@ const replayBuilder = createReplayBuilder({
   renderSidebarVideo: sidebarRenderer.renderSidebarVideo,
   getVideoEntriesForSource: segmentStore.getVideoEntriesForSource,
   getAudioEntriesForSource: segmentStore.getAudioEntriesForSource,
+  getRoomAudioEntries: segmentStore.getRoomAudioEntries,
   overlapping: segmentStore.overlapping,
 });
 registerReplaySocketHooks({ tryTriggerReplay, validateSources, getDefaultWebSources });
@@ -88,6 +90,10 @@ async function start() {
   }, CLEANUP_INTERVAL_MS);
 }
 
+roomAudioEvents.on('update', () => {
+  tick().catch((err) => logger.warn('room audio update tick failed', err.message));
+});
+
 roomCameraEvents.on('update', () => {
   tick().catch((err) => logger.warn('room camera update tick failed', err.message));
 });

server/src/services/replayEngineV2/replayBuilder.js

@@ -51,7 +51,7 @@ function buildChatEventsForWindow(startMs, endMs, limit = 22, preWindowCount = 1
   return [...beforeWindow, ...inWindow].sort((a, b) => a.ts - b.ts);
 }
 
-function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForSource, overlapping }) {
+function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForSource, getRoomAudioEntries, overlapping }) {
   function resolveReplayWindow({ sources = [], nowMs, guardMs, durationMs }) {
     const tentativeEnd = nowMs - guardMs;
     const sourceEnds = [];
@@ -186,6 +186,28 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
 
       if (!normalizedVideos.length) throw new Error('No replay segments available for selected sources');
 
+      const roomAudioEntries = await pinSegmentFiles(
+        overlapping(getRoomAudioEntries(), tStart, tEnd), tmpDir, 'room-audio-seg',
+      );
+      if (roomAudioEntries.length) {
+        // Place each segment at its recorded time. A mic joining late or
+        // reconnecting must leave silence in the gap, not shift its speech earlier.
+        const audioInputs = roomAudioEntries.flatMap((entry) => ['-i', entry.filePath]);
+        const audioFilters = roomAudioEntries.map((entry, index) => {
+          const start = Math.max(0, (tStart - entry.startMs) / 1000);
+          const duration = (Math.min(tEnd, entry.endMs) - Math.max(tStart, entry.startMs)) / 1000;
+          const delay = Math.max(0, Math.round(entry.startMs - tStart));
+          return `[${index}:a]atrim=start=${start.toFixed(3)}:duration=${duration.toFixed(3)},asetpts=PTS-STARTPTS,adelay=${delay}:all=1[room${index}]`;
+        });
+        const refs = roomAudioEntries.map((_, index) => `[room${index}]`).join('');
+        audioFilters.push(`${refs}amix=inputs=${roomAudioEntries.length}:normalize=0[roomout]`);
+        const roomAudioPath = path.join(tmpDir, 'room-audio.m4a');
+        await execFileAsync(FFMPEG_BIN, ['-y', '-hide_banner', '-loglevel', 'error', ...audioInputs,
+          '-filter_complex', audioFilters.join(';'), '-map', '[roomout]', '-vn', '-ac', '1', '-ar', '48000',
+          '-c:a', 'aac', '-b:a', '96k', roomAudioPath]);
+        normalizedAudios.push(roomAudioPath);
+      }
+
       const layout = buildGridLayout(normalizedVideos.length);
       const { maxWidth, maxHeight } = await probeMaxFrameSize(normalizedVideos.map((v) => v.path));
       let tileWidth = maxWidth || 640;

server/src/services/replayEngineV2/segmentStore.js

@@ -6,9 +6,10 @@ const fs = require('fs');
 const path = require('path');
 const logger = require('../../globals/logger').child('replayEngineV2');
 const { BUFFER_SECONDS, SEGMENT_SECONDS } = require('./constants');
-const { workers, segmentIndex } = require('./state');
+const { workers, pendingWorkerStarts, segmentIndex } = require('./state');
 const { sourceKey, sourceDirForKey } = require('./sources');
 const ptzCameraService = require('../ptzCameraService');
+const SEGMENT_FILENAME_PATTERN = /^seg-[a-f0-9-]+-\d{6}\.mp4$/;
 
 async function ensureDir(dir) {
   await fsp.mkdir(dir, { recursive: true });
@@ -25,7 +26,7 @@ function createSegmentStore({ getActiveSegmentRoot }) {
       return;
     }
 
-    const segmentFiles = files.filter((entry) => entry.isFile() && /^seg-\d{6}\.mp4$/.test(entry.name)).map((entry) => entry.name).sort();
+    const segmentFiles = files.filter((entry) => entry.isFile() && SEGMENT_FILENAME_PATTERN.test(entry.name)).map((entry) => entry.name).sort();
     if (!segmentFiles.length) {
       segmentIndex.set(key, []);
       return;
@@ -64,6 +65,13 @@ function createSegmentStore({ getActiveSegmentRoot }) {
 
   async function cleanupOldFiles() {
     const cutoff = Date.now() - BUFFER_SECONDS * 1000;
+    // Retention applies to stopped workers too; dynamic microphone sessions
+    // must not leave permanent index entries after their audio expires.
+    for (const [key, entries] of segmentIndex) {
+      const retained = entries.filter((entry) => entry.endMs >= cutoff);
+      if (!retained.length && !workers.has(key) && !pendingWorkerStarts.has(key)) segmentIndex.delete(key);
+      else segmentIndex.set(key, retained);
+    }
     const root = getActiveSegmentRoot();
     try {
       await ensureDir(root);
@@ -81,6 +89,10 @@ function createSegmentStore({ getActiveSegmentRoot }) {
             if (stat.mtimeMs < cutoff) await fsp.unlink(filePath);
           } catch {}
         }
+        if (!workers.has(dirent.name) && !pendingWorkerStarts.has(dirent.name)) {
+          // rmdir only removes empty directories, preserving buffered or newly written files.
+          try { await fsp.rmdir(dirPath); } catch {}
+        }
       }
     } catch (err) {
       logger.warn('cleanup failed', err.message);
@@ -115,6 +127,14 @@ function createSegmentStore({ getActiveSegmentRoot }) {
     return [];
   }
 
+  function getRoomAudioEntries() {
+    // Include retained buffers from microphones that disconnected during the
+    // replay window, rather than limiting history to currently live streams.
+    return [...segmentIndex.entries()]
+      .filter(([key]) => key.startsWith('roomAudio__audio__'))
+      .flatMap(([, entries]) => entries);
+  }
+
   function overlapping(entries, startMs, endMs) {
     return entries.filter((entry) => entry.endMs > startMs && entry.startMs < endMs);
   }
@@ -153,7 +173,7 @@ function createSegmentStore({ getActiveSegmentRoot }) {
       try {
         const files = fs.readdirSync(dir);
         for (const name of files) {
-          if (!/^seg-\d{6}\.mp4$/.test(name)) continue;
+          if (!SEGMENT_FILENAME_PATTERN.test(name)) continue;
           const full = path.join(dir, name);
           let stat;
           try { stat = fs.statSync(full); } catch { continue; }
@@ -172,10 +192,12 @@ function createSegmentStore({ getActiveSegmentRoot }) {
 
   return {
     refreshSegmentIndex,
+    refreshIndexForWorker,
     cleanupOldFiles,
     getVideoEntriesForSource,
     getAudioEntriesForRover,
     getAudioEntriesForSource,
+    getRoomAudioEntries,
     overlapping,
     bootstrapIndexFromDisk,
     getReplayHealthSnapshot,

server/src/services/replayEngineV2/socketHooks.js

@@ -1,6 +1,6 @@
 // Replay Socket Hooks
 // Purpose: Registers web socket replay-trigger handler that publishes replay requests.
-// Scope: Applies replay mode/cooldown/source validation for socket-triggered replay requests.
+// Scope: Applies replay mode/availability/source validation for socket-triggered replay requests.
 const io = require('../../globals/io');
 const logger = require('../../globals/logger').child('replaySocket');
 const { getMode, MODES } = require('../modeManager');
@@ -45,9 +45,9 @@ function registerReplaySocketHooks({ tryTriggerReplay, validateSources, getDefau
       const title = normalizeReplayTitle(payload?.title);
       const includeSidebar = normalizeIncludeSidebar(payload?.includeSidebar);
       const jobId = buildReplayJobId('web');
-      const attempt = tryTriggerReplay({ by: { source: 'web', requester } });
+      const attempt = tryTriggerReplay({ jobId, by: { source: 'web', requester } });
       if (!attempt.ok) {
-        cb({ error: 'Replay cooldown active', remainingMs: attempt.remainingMs, state: attempt.state });
+        cb({ error: attempt.error, remainingMs: attempt.remainingMs, state: attempt.state });
         return;
       }
 

server/src/services/replayEngineV2/sources.js

@@ -2,8 +2,10 @@
 // Purpose: Resolves replay-capable media sources and ffmpeg worker arguments.
 // Scope: Converts live rover/room state into stable source descriptors and stream worker config.
 const path = require('path');
+const { randomUUID } = require('crypto');
 const roverManager = require('../roverManager');
 const { getRoomCameras } = require('../roomCameraService');
+const { getActiveRoomAudioStreams } = require('../roomAudioService');
 const ptzCameraService = require('../ptzCameraService');
 const { FFMPEG_BIN, SEGMENT_SECONDS, TARGET_FPS } = require('./constants');
 
@@ -51,12 +53,17 @@ function listDesiredSources() {
   // both come from the same live MediaMTX path, unlike rovers where audio is a
   // separate published stream.
   sources.push(...ptzCameraService.getReplayWorkerSources());
+  for (const stream of getActiveRoomAudioStreams()) {
+    sources.push({ id: stream.id, sourceType: 'roomAudio', kind: 'audio', label: stream.name,
+      inputUrl: `rtsp://127.0.0.1:8554/room-audio/${encodeURIComponent(stream.id)}` });
+  }
   return sources;
 }
 
 function buildWorkerArgs(activeSegmentRoot, source) {
   const dir = sourceDirForKey(activeSegmentRoot, sourceKey(source));
-  const pattern = path.join(dir, 'seg-%06d.mp4');
+  // Each capture process owns its filenames so restarts preserve buffered segments.
+  const pattern = path.join(dir, `seg-${randomUUID()}-%06d.mp4`);
   // MediaMTX and RTSP cameras use TCP so replay capture has one reliable
   // transport and never falls back to separate RTP/RTCP UDP listeners.
   const inputTransport = /^rtsps?:\/\//i.test(source.inputUrl)

server/src/services/replayEngineV2/workerManager.js

@@ -12,7 +12,7 @@ async function ensureDir(dir) {
   await fsp.mkdir(dir, { recursive: true });
 }
 
-function createWorkerManager({ getActiveSegmentRoot }) {
+function createWorkerManager({ getActiveSegmentRoot, onWorkerClosed }) {
   function startWorker(source) {
     const key = sourceKey(source);
     if (workers.has(key) || pendingWorkerStarts.has(key)) return;
@@ -21,26 +21,36 @@ function createWorkerManager({ getActiveSegmentRoot }) {
     const dir = sourceDirForKey(getActiveSegmentRoot(), key);
     ensureDir(dir)
       .then(() => {
-        if (workers.has(key)) {
+        // Directory creation can finish after a microphone has disconnected.
+        if (workers.has(key) || !listDesiredSources().some((entry) => sourceKey(entry) === key)) {
           pendingWorkerStarts.delete(key);
           return;
         }
         const proc = spawn(FFMPEG_BIN, args, { stdio: ['ignore', 'ignore', 'pipe'] });
-        workers.set(key, { key, source, proc });
+        const worker = { key, source, proc, stopping: false };
+        workers.set(key, worker);
         pendingWorkerStarts.delete(key);
         proc.stderr.on('data', (chunk) => {
           const text = String(chunk || '').trim();
           if (!text) return;
           logger.warn('worker stderr', { key, text: text.slice(0, 500) });
         });
-        proc.on('exit', (code, signal) => {
-          const current = workers.get(key);
-          if (current?.proc === proc) workers.delete(key);
-          logger.warn('worker exited', { key, code, signal });
-          setTimeout(() => {
-            const desired = listDesiredSources().find((entry) => sourceKey(entry) === key);
-            if (desired && !workers.has(key)) startWorker(desired);
-          }, 1500);
+        proc.on('error', (err) => logger.warn('worker process failed', { key, error: err.message }));
+        worker.finished = new Promise((resolve) => {
+          proc.once('close', async (code, signal) => {
+            clearTimeout(worker.killTimer);
+            // FFmpeg writes the final MP4 trailer before close. Index it before
+            // releasing this worker so a disconnect cannot drop its last audio.
+            try { await onWorkerClosed(worker); }
+            catch (err) { logger.warn('final worker indexing failed', { key, error: err.message }); }
+            if (workers.get(key) === worker) workers.delete(key);
+            resolve();
+            logger.warn('worker exited', { key, code, signal });
+            if (!worker.stopping) setTimeout(() => {
+              const desired = listDesiredSources().find((entry) => sourceKey(entry) === key);
+              if (desired && !workers.has(key)) startWorker(desired);
+            }, 1500);
+          });
         });
       })
       .catch((err) => {
@@ -52,8 +62,14 @@ function createWorkerManager({ getActiveSegmentRoot }) {
   function stopWorker(key) {
     const worker = workers.get(key);
     if (!worker) return;
-    try { worker.proc.kill('SIGTERM'); } catch {}
-    workers.delete(key);
+    if (!worker.stopping) {
+      worker.stopping = true;
+      worker.proc.kill('SIGTERM');
+      // Bound shutdown if a capture is blocked waiting on its network input.
+      worker.killTimer = setTimeout(() => worker.proc.kill('SIGKILL'), 3000);
+      worker.killTimer.unref();
+    }
+    return worker.finished;
   }
 
   async function syncWorkers() {
@@ -63,9 +79,11 @@ function createWorkerManager({ getActiveSegmentRoot }) {
       const key = sourceKey(source);
       if (!workers.has(key)) startWorker(source);
     }
+    const stops = [];
     for (const key of Array.from(workers.keys())) {
-      if (!desiredKeys.has(key)) stopWorker(key);
+      if (!desiredKeys.has(key)) stops.push(stopWorker(key));
     }
+    await Promise.all(stops);
   }
 
   return { startWorker, stopWorker, syncWorkers };

server/src/services/roomAudioService/configuration.js

@@ -0,0 +1,14 @@
+const { strictObject, string, boolean } = require('../../configuration/schemaHelpers');
+module.exports = {
+  key: 'roomAudio', feature: true,
+  defaultValue: { enabled: false, sources: [] },
+  schema: strictObject({
+    enabled: boolean({ description: 'Enable room audio and local browser microphone publishing.' }),
+    sources: { type: 'array', items: strictObject({
+      id: string({ minLength: 1, maxLength: 80, pattern: '^[a-zA-Z0-9_-]+$' }),
+      name: string({ minLength: 1, maxLength: 120 }),
+      type: { type: 'string', enum: ['device', 'stream'] },
+      input: string({ minLength: 1, maxLength: 2048, description: 'ALSA device (for example default or plughw:CARD=USB,DEV=0), or a network stream URL.' }),
+    }, { required: ['id', 'name', 'type', 'input'] }) },
+  }, { title: 'Room audio', required: ['enabled', 'sources'] }),
+};

server/src/services/roomAudioService/index.js

@@ -0,0 +1,171 @@
+const { spawn } = require('child_process');
+const EventEmitter = require('events');
+const io = require('../../globals/io');
+const logger = require('../../globals/logger').child('roomAudio');
+const { loadConfig, registerConfigurationHandler } = require('../../configuration');
+const { getSocketIp, isLocalNetwork } = require('../../helpers/ipResolver');
+const { getMode, MODES } = require('../modeManager');
+const { getRole, isAdmin, isLockdownAdmin } = require('../roleService');
+const { canBecomeSpectator } = require('../authService');
+const { isDeterred } = require('../verificationService');
+const videoSessions = require('../videoSessions');
+const roomAudioEvents = new EventEmitter();
+const sources = new Map();
+const mediaSessions = new Map();
+let enabled = false;
+let active = [];
+let polling = false;
+let generation = 0;
+
+function canListen(socket) {
+  if (!enabled || !socket) return false;
+  const mode = getMode();
+  if (mode === MODES.LOCKDOWN) return isLockdownAdmin(socket);
+  if (isAdmin(socket)) return true;
+  if (getRole(socket) === 'spectator') return canBecomeSpectator(socket);
+  // Room audio follows general driving eligibility, without tying listening
+  // to rover assignment, private-rover access, or the current driving turn.
+  return getRole(socket) === 'user' && mode !== MODES.ADMIN && !isDeterred(socket);
+}
+function canPublish(socket, id) {
+  return enabled && isLocalNetwork(getSocketIp(socket)) && sources.get(id)?.socketId === socket.id;
+}
+function getRoomAudioStreams(socket) { return canListen(socket) ? active : []; }
+function getActiveRoomAudioStreams() { return active; }
+function trackMediaSession(id, token) { if (id) mediaSessions.set(id, token); }
+function hasStream(id) { return active.some((source) => source.id === id); }
+function stopSource(id) {
+  const source = sources.get(id);
+  if (!source) return;
+  sources.delete(id);
+  clearTimeout(source.timer);
+  source.child?.kill('SIGTERM');
+  videoSessions.revokeWhere((info) => ['roomAudio', 'roomMic'].includes(info.sourceType) && info.sourceId === id);
+  active = active.filter((entry) => entry.id !== id);
+  roomAudioEvents.emit('update');
+}
+function startWorker(source) {
+  if (sources.get(source.id) !== source) return;
+  const args = ['-nostdin', '-hide_banner', '-loglevel', 'warning'];
+  if (source.type === 'device') args.push('-f', 'alsa');
+  else if (/^rtsps?:/i.test(source.input)) args.push('-rtsp_transport', 'tcp', '-timeout', '10000000');
+  else args.push('-rw_timeout', '10000000');
+  args.push('-i', source.input, '-map', '0:a:0', '-vn', '-c:a', 'libopus', '-ar', '48000', '-ac', '1', '-b:a', '64k', '-f', 'rtsp', '-rtsp_transport', 'tcp', `rtsp://127.0.0.1:8554/room-audio/${source.id}`);
+  const child = spawn('ffmpeg', args, { stdio: ['ignore', 'ignore', 'pipe'] });
+  source.child = child;
+  child.stderr.on('data', (data) => logger.warn('Room audio %s: %s', source.name, String(data).trim()));
+  child.on('error', (error) => logger.warn('Room audio %s: %s', source.name, error.message));
+  child.on('close', () => {
+    if (sources.get(source.id) !== source) return;
+    source.child = null;
+    source.timer = setTimeout(() => startWorker(source), 2000);
+  });
+}
+function applyConfig(config = {}) {
+  const ids = (config.sources || []).map((source) => source.id);
+  if (new Set(ids).size !== ids.length) throw new Error('Room audio source IDs must be unique');
+  generation += 1;
+  enabled = Boolean(config.enabled);
+  const configured = new Map((config.sources || []).map((entry) => {
+    const source = { ...entry, id: `static-${entry.id}` };
+    return [source.id, source];
+  }));
+  for (const [id, source] of sources) {
+    const next = configured.get(id);
+    // Static-source edits should not interrupt browser microphones or restart
+    // unchanged captures. Disabling the feature still stops every source.
+    if (!enabled || (!source.socketId && (!next || next.type !== source.type || next.input !== source.input))) stopSource(id);
+  }
+  if (enabled) for (const [id, source] of configured) {
+    const existing = sources.get(id);
+    if (existing) existing.name = source.name;
+    else {
+      sources.set(id, source);
+      startWorker(source);
+    }
+  }
+  roomAudioEvents.emit('update');
+}
+async function listMediaMtxItems(collection) {
+  const items = [];
+  let page = 0;
+  let pageCount = 1;
+  while (page < pageCount) {
+    const response = await fetch(`http://127.0.0.1:9997/v3/${collection}/list?page=${page}`, { signal: AbortSignal.timeout(1500) });
+    if (!response.ok) throw new Error('MediaMTX unavailable');
+    const data = await response.json();
+    items.push(...(data.items || []));
+    pageCount = data.pageCount || 1;
+    page += 1;
+  }
+  return items;
+}
+
+// Advertise actual ready MediaMTX paths, rather than a browser reservation or
+// an ffmpeg process that may still be trying to connect to its input.
+async function refreshStreams() {
+  if ((!enabled && !mediaSessions.size) || polling) return;
+  polling = true;
+  const currentGeneration = generation;
+  let ready = new Set();
+  try {
+    // HTTP authorization is checked at connection time. Recheck established
+    // browser sessions too so lockdown, disable, and disconnect stop live audio.
+    const previousSessionIds = [...mediaSessions.keys()];
+    const sessions = await listMediaMtxItems('webrtcsessions');
+    const liveSessionIds = new Set(sessions.map((session) => session.id));
+    for (const session of sessions) {
+      if (!session.path?.startsWith('room-audio/')) continue;
+      const info = videoSessions.getSession(mediaSessions.get(session.id));
+      const socket = info && io.sockets.sockets.get(info.socketId);
+      const allowed = socket && (session.state === 'publish'
+        ? info.sourceType === 'roomMic' && canPublish(socket, info.sourceId)
+        : info.sourceType === 'roomAudio' && canListen(socket));
+      if (!allowed) await fetch(`http://127.0.0.1:9997/v3/webrtcsessions/kick/${session.id}`, { method: 'POST', signal: AbortSignal.timeout(1500) });
+    }
+    for (const id of previousSessionIds) if (!liveSessionIds.has(id)) mediaSessions.delete(id);
+  } catch (error) {
+    logger.warn('Room audio permission refresh failed: %s', error.message);
+  }
+  // Permission API failures must not make ready streams disappear. Only path
+  // discovery determines the catalog used by playback and replay capture.
+  try {
+    const paths = await listMediaMtxItems('paths');
+    for (const path of paths) if (path.ready) ready.add(path.name);
+  } catch { ready = new Set(); }
+  finally { polling = false; }
+  if (generation !== currentGeneration) return;
+  const next = [...sources.values()].filter((source) => ready.has(`room-audio/${source.id}`)).map(({ id, name }) => ({ id, name }));
+  if (JSON.stringify(next) !== JSON.stringify(active)) {
+    active = next;
+    roomAudioEvents.emit('update');
+  }
+}
+io.on('connection', (socket) => {
+  socket.on('roomAudio:start', (_payload, cb = () => {}) => {
+    if (!enabled || !isLocalNetwork(getSocketIp(socket))) return cb({ error: 'Room microphone publishing requires a local visitor and enabled room audio.' });
+    // One owned path per socket; retries replace credentials without accumulating streams.
+    const id = `mic-${socket.id}`;
+    videoSessions.revokeWhere((info) => info.sourceType === 'roomMic' && info.sourceId === id);
+    sources.set(id, { id, name: 'Room microphone', socketId: socket.id });
+    const token = videoSessions.createSession(socket, { type: 'roomMic', id });
+    cb({ id, token, url: `/video/room-audio/${id}/whip` });
+  });
+  socket.on('roomAudio:release', ({ token } = {}) => {
+    const info = videoSessions.getSession(token);
+    if (info?.socketId === socket.id && info.sourceType === 'roomAudio') videoSessions.revokeSession(token);
+  });
+  socket.on('roomAudio:stop', (_payload, cb = () => {}) => {
+    stopSource(`mic-${socket.id}`);
+    cb({ success: true });
+  });
+  socket.on('disconnect', () => {
+    stopSource(`mic-${socket.id}`);
+  });
+});
+applyConfig(loadConfig().roomAudio);
+registerConfigurationHandler('roomAudio', applyConfig);
+const pollTimer = setInterval(refreshStreams, 2000);
+pollTimer.unref();
+process.once('exit', () => { for (const source of sources.values()) source.child?.kill('SIGTERM'); });
+module.exports = { getActiveRoomAudioStreams, trackMediaSession, roomAudioEvents, getRoomAudioStreams, canListen, canPublish, hasStream };

server/src/services/roomCameraService/configuration.js

@@ -1,7 +1,7 @@
 // Room-Camera Configuration
 // Purpose: Defines the optional named snapshot and stream camera catalog.
 // Scope: Contains configuration metadata only and never contacts a camera.
-const { strictObject, string, boolean } = require('../../configuration/schemaHelpers');
+const { strictObject, string, boolean, integer } = require('../../configuration/schemaHelpers');
 
 module.exports = {
   key: 'roomCameras',
@@ -10,6 +10,7 @@ module.exports = {
   // initial configuration while the feature switch prevents network requests.
   defaultValue: {
     enabled: false,
+    streamIntervalMs: 2000,
     cameras: [
       {
         id: 'lobby',
@@ -29,6 +30,7 @@ module.exports = {
   },
   schema: strictObject({
     enabled: boolean({ description: 'Immediately publishes the configured room-camera catalog and enables camera snapshots and streams.' }),
+    streamIntervalMs: integer({ title: 'Stream interval (ms)', description: 'Interval for sending room camera images to clients. Lowering this increases bandwidth usage FAST because it sends a full image each time.', minimum: 1 }),
     cameras: {
       type: 'array',
       description: 'Room cameras available to the web UI and replay system.',

server/src/services/roomCameraService/index.js

@@ -8,10 +8,12 @@ const replay = require('../replayEngineV2/roomCameraReplayBuilder');
 const { loadConfig, registerConfigurationHandler } = require('../../configuration');
 
 let enabled = false;
+let streamIntervalMs = 2000;
 
 const snapshotEngine = createSnapshotEngine({ getRoomCameras, roomCameraEvents });
 function applyRoomCameraConfig(roomCameraConfig = {}) {
   enabled = Boolean(roomCameraConfig.enabled);
+  streamIntervalMs = roomCameraConfig.streamIntervalMs ?? 2000;
   // Loading an empty catalog on disable causes the snapshot engine's existing
   // update listener to close every stream and timer without unregistering the
   // stable browser gateway.
@@ -19,6 +21,7 @@ function applyRoomCameraConfig(roomCameraConfig = {}) {
 }
 
 registerRoomCameraSocketGateway({
+  getStreamIntervalMs: () => streamIntervalMs,
   getRoomCamera,
   getRoomCameras,
   getRoomCameraState: snapshotEngine.getRoomCameraState,

server/src/services/roomCameraService/socketGateway.js

@@ -8,7 +8,6 @@ const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
 
 const SUBSCRIBE_LIMIT = 50;
 const SUBSCRIBE_WINDOW_MS = 10000;
-const STREAM_INTERVAL_MS = 2000;
 
 function passesMode(socket) {
   const mode = getMode();
@@ -20,7 +19,7 @@ function passesMode(socket) {
   return true;
 }
 
-function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoomCameraState, roomCameraStreamEvents }) {
+function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoomCameraState, roomCameraStreamEvents, getStreamIntervalMs }) {
   const cameraSubscribers = new Map();
   const socketSubscriptions = new Map();
   const subscribeBuckets = new Map();
@@ -82,7 +81,7 @@ function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoo
       }
       const lastSent = lastMap.get(id) || 0;
       const now = ts || Date.now();
-      if (now - lastSent < STREAM_INTERVAL_MS) return;
+      if (now - lastSent < getStreamIntervalMs()) return;
       lastMap.set(id, now);
       sendFrame(socket, id, { ts }, buffer);
     });

server/src/services/roverLearningService/capabilities.js

@@ -1,16 +1,18 @@
 // Match actuator configuration, not a rover's display name or changing telemetry.
 const { isDeepStrictEqual } = require('node:util');
-const POLICY_VERSION = 3;
+const POLICY_VERSION = 5;
 
 function controlProfile(meta = {}) {
-  return Object.fromEntries(['cameraServo', 'peripherals', 'headlight', 'laser', 'roomba', 'platform']
+  return Object.fromEntries(['cameraServo', 'headlight', 'roomba', 'platform']
     .map((key) => [key, meta?.[key] ?? null]));
 }
 
 function compatible(model, record) {
   if (model?.specification?.version !== POLICY_VERSION || !record) return false;
   const profile = controlProfile(record.meta);
-  return Array.isArray(model.controlProfiles) && model.controlProfiles.some((saved) => isDeepStrictEqual(saved, profile));
+  // Retained recordings may include removed actuator fields. Compare only
+  // controls this policy actually uses, on both sides of the boundary.
+  return Array.isArray(model.controlProfiles) && model.controlProfiles.some((saved) => isDeepStrictEqual(controlProfile(saved), profile));
 }
 
 module.exports = { POLICY_VERSION, controlProfile, compatible };

server/src/services/roverLearningService/diagnostics.js

@@ -7,7 +7,7 @@ const path = require('node:path');
 const { createHash } = require('node:crypto');
 const applicationVersion = require('../../../package.json').version;
 const hash = createHash('sha256');
-for (const file of ['workers/policy.py', 'workers/dataset.py', 'workers/train.py', 'workers/infer.py',
+for (const file of ['workers/policy.py', 'workers/dataset.py', 'workers/train.py', 'workers/infer.py', 'workers/observations.py', 'workers/learning.py',
   'capabilities.js', 'errors.js', 'storage.js', 'runtime.js', 'models.js', 'driving.js', 'recording.js', 'training.js', 'socketGateway.js', 'diagnostics.js']) {
   hash.update(fs.readFileSync(path.join(__dirname, file)));
 }
@@ -41,11 +41,12 @@ function report(state, controller) {
       sessions: state.recording.sessions.map((item) => ({ startedAtUnixMs: item.startedAt,
         status: item.video, droppedEvents: item.droppedEvents, error: redact(item.error), failureDetails: item.failureDetails || null, stderr: redact(item.stderr), errorCategory: errorCategory(item.error) })) },
     training: { status: training.status, steps: training.steps, examples: training.examples,
-      selection: training.selection || null,
+      selection: training.selection || null, evaluatedSteps: training.evaluatedSteps ?? null,
       loss: training.loss, losses: training.losses || null, publication: training.publication || null,
+      trainingAudit: metrics.trainingAudit || null,
       evaluation: training.evaluation || null, dataset: metrics.dataset || training.dataset || null,
       resources: { decodeSeconds: metrics.decodeSeconds ?? null, trainingSeconds: metrics.trainingSeconds ?? null,
-        evaluationSeconds: metrics.evaluationSeconds ?? null, checkpointSeconds: metrics.checkpointSeconds ?? null, cpuSeconds: metrics.cpuSeconds ?? null,
+        evaluationSeconds: metrics.evaluationSeconds ?? null, probeSeconds: metrics.probeSeconds ?? null, beforeAuditSeconds: metrics.beforeAuditSeconds ?? null, checkpointSeconds: metrics.checkpointSeconds ?? null, cpuSeconds: metrics.cpuSeconds ?? null,
         peakRssMiB: metrics.peakRssMiB ?? null },
       history: training.history || [], rejectionReasons: (training.rejectionReasons || []).map(redact), rejectionCategories: (training.rejectionReasons || []).map(errorCategory),
       error: redact(training.error), errorCategory: errorCategory(training.error), nextAttemptAtUnixMs: training.nextAttemptAt },

server/src/services/roverLearningService/driving.js

@@ -9,7 +9,8 @@ const { recordCommandRequest } = require('../commandService/recording');
 const { acquireModel } = require('./models');
 const { compatible } = require('./capabilities');
 
-const OPERATING = new Set(['drive', 'motors', 'servo', 'headlight', 'laser', 'horn', 'peripheral', 'song', 'raw']);
+const OPERATING = new Set(['drive', 'servo', 'headlight', 'raw']);
+const MODE_COMMANDS = new Set([128, 131, 132, 143].map((opcode) => Buffer.from([opcode]).toString('base64')));
 const clamp = (value, min, max) => Math.max(min, Math.min(max, Number(value) || 0));
 
 function canControl(socket, roverId) {
@@ -34,14 +35,12 @@ function createDriving({ root, config, onChange, activity }) {
     if (session.status === 'stopping') return;
     if (!canControl(session.socket, session.roverId)) { stop(session.roverId, 'Control permission lost'); return; }
     if (!OPERATING.has(command?.type)) throw new Error('Unsupported model command');
+    if (command.type === 'raw' && !MODE_COMMANDS.has(command.raw)) throw new Error('Unsupported model operating-mode command');
     const record = roverManager.rovers.get(session.roverId);
     if (command.type === 'drive') {
       const limit = Math.min(500, Number(record.meta?.maxWheelSpeed) || 500);
       command.driveDirect = { left: Math.round(clamp(command.driveDirect?.left, -limit, limit)), right: Math.round(clamp(command.driveDirect?.right, -limit, limit)) };
     }
-    if (command.type === 'motors') {
-      command.motorPwm = { main: clamp(command.motorPwm?.main, -127, 127), side: clamp(command.motorPwm?.side, -127, 127), vacuum: clamp(command.motorPwm?.vacuum, 0, 127) };
-    }
     if (command.type === 'servo') {
       const servo = record.meta?.cameraServo;
       if (!servo?.enabled) { session.submissions.push({ command, result: 'Servo unavailable' }); return; }
@@ -64,8 +63,6 @@ function createDriving({ root, config, onChange, activity }) {
     // A stop must still reach the rover after turn permission is revoked.
     for (const command of [
       { type: 'drive', driveDirect: { left: 0, right: 0 } },
-      { type: 'motors', motorPwm: { main: 0, side: 0, vacuum: 0 } },
-      { type: 'horn', horn: { action: 'stop' } },
     ]) { try { issueCommand(roverId, command); } catch { /* Rover may already be offline. */ } }
     session.worker.kill('SIGTERM');
     const timer = setTimeout(() => session.worker.kill('SIGKILL'), 2000);

server/src/services/roverLearningService/models.js

@@ -142,8 +142,8 @@ function removeObsoleteModels(root) {
       else { await fs.rm(target, { recursive: true, force: true }); retired.delete(target); }
     }
     // These filenames belong to superseded optimizer/scheduler lineages, not
-    // policy versions. The current command policy uses lineage v4.
-    for (const suffix of ['', '-v1', '-v2', '-v3']) {
+    // policy versions. The current command policy uses lineage v6.
+    for (const suffix of ['', '-v1', '-v2', '-v3', '-v4', '-v5']) {
       for (const filename of [`resume${suffix}.pt`, `resume${suffix}.tmp`,
         `schedule${suffix}.json`, `schedule${suffix}.json.tmp`]) {
         await fs.rm(path.join(root, 'training', filename), { force: true });

server/src/services/roverLearningService/training.js

@@ -33,10 +33,17 @@ function createTrainer({ root, config, actionThreshold, logger }) {
         }
       }
       try {
-        const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v4.json'), 'utf8'));
+        const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v6.json'), 'utf8'));
         ledger = saved.sessions;
         state = { ...saved.progress, status: 'waiting' };
         if (!ledger || typeof ledger !== 'object') throw new Error('Invalid training schedule');
+        // Retry this reader-specific rejection once after the fixed-grid repair.
+        // Keep optimizer progress and unrelated recording failures intact.
+        if (saved.samplingVersion !== 2) {
+          for (const [id, entry] of Object.entries(ledger)) {
+            if (entry.rejected === 'No complete, aligned observation/action windows') delete ledger[id];
+          }
+        }
       } catch (error) { if (error.code !== 'ENOENT') throw error; }
     } catch (error) {
       startupError = true;
@@ -46,8 +53,8 @@ function createTrainer({ root, config, actionThreshold, logger }) {
   }
 
   async function saveLedger() {
-    const filename = path.join(trainingRoot, 'schedule-v4.json');
-    await fs.writeFile(`${filename}.tmp`, JSON.stringify({ sessions: ledger, progress: state }));
+    const filename = path.join(trainingRoot, 'schedule-v6.json');
+    await fs.writeFile(`${filename}.tmp`, JSON.stringify({ samplingVersion: 2, sessions: ledger, progress: state }));
     await fs.rename(`${filename}.tmp`, filename);
   }
 
@@ -159,6 +166,7 @@ function createTrainer({ root, config, actionThreshold, logger }) {
             state = { ...state, status: stopped ? 'stopped' : 'waiting', steps: result.steps ?? state.steps,
               completedAt: Date.now(), elapsedSeconds: result.elapsedSeconds ?? null, error: null,
               examples: result.examples ?? state.examples, loss: result.loss ?? null,
+              evaluatedSteps: result.trained ? result.steps : state.evaluatedSteps,
               diagnostics: result.diagnostics ?? state.diagnostics, publication: result.publication ?? state.publication, evaluation: result.evaluation ?? state.evaluation,
               reason: result.reason ?? 'Waiting for the next scheduled training job' };
             await saveLedger();
@@ -169,7 +177,9 @@ function createTrainer({ root, config, actionThreshold, logger }) {
           logger.error('Training result persistence failed', { error: error.message, failure: errorDetails(error, 'training result persistence') });
         } finally {
           history.push({ at: Date.now(), status: state.status, steps: state.steps, loss: state.loss,
-            losses: state.losses || null, evaluation: state.evaluation || null,
+            losses: state.losses || null, evaluation: state.evaluation ? Object.fromEntries(
+              ['status', 'windows', 'precision', 'recall', 'f1', 'predictedActions', 'positiveActions', 'numericMse', 'missedStops', 'gateReasons']
+                .map((key) => [key, state.evaluation[key]])) : null,
             elapsedSeconds: state.elapsedSeconds || null });
           if (history.length > 20) history.shift();
           leases.clear();

server/src/services/roverLearningService/workers/dataset.py

@@ -8,6 +8,7 @@ from pathlib import Path
 
 import av
 import numpy as np
+from observations import decision_frames, history_frames
 
 from policy import SLOTS, VALUES, HEIGHT, WIDTH, SENSORS, command_family, is_stop, encode_command, sensor_vector
 
@@ -47,7 +48,7 @@ def load_events(directory):
     return metadata, sorted(events, key=lambda e: e['ts'])
 
 
-def prepare(directory, schema, stats):
+def prepare(directory, schema, stats, extend):
     metadata, events = load_events(directory)
     requests = {}
     accepted = set()
@@ -72,14 +73,28 @@ def prepare(directory, schema, stats):
     if not sensors:
         raise ValueError('No sensor observations')
     candidate = copy.deepcopy(schema)
-    history, targets = [], []
+    history, targets, unsupported_times = [], [], []
+    unsupported = {}
+    dispatch_delays = []
     stats.update({'sensorFrames': len(sensors), 'dispatches': len(dispatches),
                   'failedCommands': len(failed), 'sensorPresentFraction': np.mean([row[1][len(SENSORS):] for row in sensors], axis=0).tolist()})
     for data in dispatches:
         command = data.get('command') or {}
-        if command.get('type') in ('reset', 'reboot', 'update', 'sensorStream'):
+        if command.get('type') in ('reset', 'reboot', 'update'):
             boundaries.append(data['ts'])
-        encoded = encode_command(command, candidate, extend=True)
+        try:
+            encoded = encode_command(command, candidate, extend=extend)
+        except ValueError as error:
+            # Unrepresentable controls invalidate only nearby observations/targets,
+            # not the rest of a recording. Never turn them into negative labels.
+            request = requests.get(data.get('requestId'))
+            unsupported_times.extend([data['ts'], request['ts'] if request else data['ts']])
+            key = json.dumps(command, sort_keys=True)
+            if key not in unsupported and len(unsupported) < 32:
+                unsupported[key] = {'command': command, 'reason': str(error), 'count': 0}
+            if key in unsupported:
+                unsupported[key]['count'] += 1
+            continue
         if encoded is None:
             continue
         stamp = data['ts']
@@ -90,35 +105,43 @@ def prepare(directory, schema, stats):
             target_time = request['ts']
             if not 0 <= stamp - target_time <= 1000:
                 raise ValueError('Invalid request/dispatch timing')
+            dispatch_delays.append(stamp - target_time)
             targets.append((target_time, *encoded))
         else:
             # Server safety overrides belong in history, never demonstration
             # targets. Exclude windows around them rather than teaching inaction.
             boundaries.append(stamp)
-    if not targets:
+    stats['requestToDispatchMs'] = {'mean': sum(dispatch_delays) / max(1, len(dispatch_delays)),
+                                    'max': max(dispatch_delays, default=0)}
+    stats['acceptedCommandEvents'] = len(targets)
+    stats['representedCommandEvents'] = 0
+    stats['commandEventCounts'] = [0] * SLOTS
+    stats['unsupportedCommands'] = list(unsupported.values())
+    stats['unsupportedCommandEvents'] = len(unsupported_times) // 2
+    if not targets and not unsupported_times:
         raise ValueError('No accepted client operating commands')
     schema[:] = candidate
-    return metadata, sensors, sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries)
+    return metadata, sensors, sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries), sorted(unsupported_times)
 
 
-def windows(directory, schema, cancelled=lambda: False, stats=None):
+def windows(directory, schema, cancelled=lambda: False, stats=None, extend=True):
     directory = Path(directory)
     stats = stats if stats is not None else {}
-    metadata, sensors, history, targets, boundaries = prepare(directory, schema, stats)
+    metadata, sensors, history, targets, boundaries, unsupported_times = prepare(directory, schema, stats, extend)
     stats.update({'usableWindows': 0, 'commandWindows': 0, 'stopWindows': 0, 'noCommandWindows': 0,
-                  'staleSensorWindows': 0, 'boundaryWindows': 0,
-                  'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0})
+                  'staleSensorWindows': 0, 'boundaryWindows': 0, 'maxSensorAgeMs': 0, 'maxFrameAgeMs': 0,
+                  'unsupportedWindows': 0, 'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0})
     sensor_times = [row[0] for row in sensors]
     target_times = [row[0] for row in targets]
     available = np.zeros(SLOTS, dtype=np.float32)
-    for _, slot, _ in history:
-        available[slot] = 1
+    available[:len(schema)] = 1
     command_values = np.zeros((SLOTS, VALUES), dtype=np.float32)
     command_times = np.full(SLOTS, -np.inf)
     cursor = 0
     recent = collections.deque(maxlen=32)
-    last_sample, last_pts = -math.inf, -math.inf
+    last_pts = -math.inf
     def frames():
+        nonlocal last_pts
         files = metadata['video'].get('files', ['video.mkv'])
         for name in files:
             if Path(name).name != name:
@@ -126,27 +149,24 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
             with av.open(str(directory / name)) as container:
                 stream = container.streams.video[0]
                 stream.codec_context.thread_count = 1
-                yield from container.decode(stream)
+                for frame in container.decode(stream):
+                    if frame.pts is None:
+                        raise ValueError('Video frame has no timestamp')
+                    stamp = float(frame.pts * frame.time_base) * 1000
+                    stats['videoFrames'] += 1
+                    if math.isfinite(last_pts):
+                        stats['maxVideoGapMs'] = max(stats['maxVideoGapMs'], stamp - last_pts)
+                    last_pts = stamp
+                    if not metadata['startedAt'] - 2000 <= stamp <= metadata['endedAt'] + 2000:
+                        raise ValueError('Video PTS is not aligned to session Unix time')
+                    yield frame
 
-    for frame in frames():
+    represented = set()
+    for stamp, frame, captured_at in decision_frames(frames()):
         if cancelled():
             return
-        if frame.pts is None:
-            raise ValueError('Video frame has no timestamp')
-        stamp = float(frame.pts * frame.time_base) * 1000
-        if not math.isfinite(stamp) or stamp < last_pts:
-            raise ValueError('Invalid/nonmonotonic video timestamps')
-        stats['videoFrames'] += 1
-        if math.isfinite(last_pts):
-            stats['maxVideoGapMs'] = max(stats['maxVideoGapMs'], stamp - last_pts)
-        last_pts = stamp
-        if not metadata['startedAt'] - 2000 <= stamp <= metadata['endedAt'] + 2000:
-            raise ValueError('Video PTS is not aligned to session Unix time')
         if stamp < metadata.get('trainingStartedAt', metadata['startedAt']):
             continue
-        if stamp - last_sample < 99:
-            continue
-        last_sample = stamp
         while cursor < len(history) and history[cursor][0] < stamp:
             when, slot, values = history[cursor]
             # Do not let drive predictions simply copy earlier human/model drive commands.
@@ -154,24 +174,24 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
                 command_values[slot] = values
                 command_times[slot] = when
             cursor += 1
-        sensor_index = bisect.bisect_right(sensor_times, stamp) - 1
+        sensor_index = bisect.bisect_left(sensor_times, stamp) - 1
         if sensor_index < 0 or stamp - sensor_times[sensor_index] > 1000:
             stats['staleSensorWindows'] += 1
             recent.clear()
             continue
+        stats['maxSensorAgeMs'] = max(stats['maxSensorAgeMs'], stamp - sensor_times[sensor_index])
+        stats['maxFrameAgeMs'] = max(stats['maxFrameAgeMs'], stamp - captured_at)
         seen = np.isfinite(command_times).astype(np.float32)
         ages = np.minimum((stamp - command_times) / 10000, 1).astype(np.float32)
         state = np.concatenate((sensors[sensor_index][1], command_values.flatten(), ages, seen))
         pixels = frame.to_ndarray(width=WIDTH, height=HEIGHT, format='rgb24').transpose(2, 0, 1).copy()
         recent.append((stamp, pixels, state))
-        chosen = []
-        for offset in (900, 600, 300, 0):
-            options = [row for row in recent if row[0] <= stamp - offset + 0.01]
-            if not options or stamp - offset - options[-1][0] > 200:
-                break
-            chosen.append(options[-1])
+        chosen = history_frames(recent, stamp)
         if len(chosen) != 4 or stamp + 100 > metadata['endedAt']:
             continue
+        if bisect.bisect_right(unsupported_times, stamp + 100) > bisect.bisect_left(unsupported_times, chosen[0][0]):
+            stats['unsupportedWindows'] += 1
+            continue
         if bisect.bisect_right(boundaries, stamp + 100) > bisect.bisect_left(boundaries, chosen[0][0]):
             stats['boundaryWindows'] += 1
             continue
@@ -184,7 +204,7 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
         for _, slot, vector in targets[start:end]:
             template = schema[slot]['template']
             family = command_family(template)
-            if ((actions[slot] and (template['type'] in ('headlight', 'laser', 'horn', 'song', 'raw') or not np.array_equal(values[slot], vector)))
+            if ((actions[slot] and (template['type'] in ('headlight', 'raw') or not np.array_equal(values[slot], vector)))
                     or (not actions[slot] and family in families)):
                 ambiguous = True
                 break
@@ -193,6 +213,10 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
         if ambiguous:
             stats['ambiguousWindows'] += 1
             continue
+        represented.update(range(start, end))
+        stats['representedCommandEvents'] = len(represented)
+        for slot in np.flatnonzero(actions):
+            stats['commandEventCounts'][slot] += 1
         stats['usableWindows'] += 1
         stats['commandWindows' if actions.any() else 'noCommandWindows'] += 1
         stats['stopWindows'] += int(any(actions[slot] and is_stop(descriptor) for slot, descriptor in enumerate(schema)))

server/src/services/roverLearningService/workers/infer.py

@@ -13,6 +13,7 @@ import time
 import av
 import numpy as np
 import torch
+from observations import decision_frames, history_frames
 
 from policy import Policy, VERSION, SLOTS, VALUES, HEIGHT, WIDTH, SENSORS, select_slots, decode_command, encode_command, sensor_vector
 
@@ -54,23 +55,16 @@ def main():
     sensors, sensor_time = {}, 0
     history = collections.deque(maxlen=32)
     latched = set()
-    sampled = 0
     with av.open(args.url, options={'rtsp_transport': 'tcp', 'timeout': '3000000',
                                    'use_wallclock_as_timestamps': '1'}) as video:
         stream = video.streams.video[0]
         stream.codec_context.thread_count = 1
-        for frame in video.decode(stream):
+        for stamp, frame, captured_at in decision_frames(video.decode(stream)):
             if closed.is_set():
                 return
-            if frame.pts is None:
-                continue
-            stamp = float(frame.pts * frame.time_base) * 1000
             now = time.time() * 1000
-            if not math.isfinite(stamp) or abs(now - stamp) > 2000:
+            if abs(now - captured_at) > 2000:
                 raise ValueError('Video time is stale or not aligned to server time')
-            if stamp - sampled < 99:
-                continue
-            sampled = stamp
             with mutex:
                 while observations and observations[0]['ts'] < stamp:
                     event = observations.popleft()
@@ -94,12 +88,7 @@ def main():
                                     np.isfinite(last_commands).astype(np.float32)))
             pixels = frame.to_ndarray(width=WIDTH, height=HEIGHT, format='rgb24').transpose(2, 0, 1).copy()
             history.append((stamp, pixels, state))
-            selected = []
-            for offset in (900, 600, 300, 0):
-                candidates = [entry for entry in history if entry[0] <= stamp - offset + .01]
-                if not candidates or stamp - offset - candidates[-1][0] > 200:
-                    break
-                selected.append(candidates[-1])
+            selected = history_frames(history, stamp)
             if len(selected) != 4:
                 continue
             before = time.perf_counter()
@@ -116,7 +105,7 @@ def main():
             proposals, commands = [], []
             for slot, descriptor in enumerate(schema):
                 command = decode_command(descriptor, predictions[slot])
-                discrete = command['type'] in ('headlight', 'laser', 'horn', 'song', 'raw')
+                discrete = command['type'] in ('headlight', 'raw')
                 reason = ('Below threshold' if slot not in active else
                           'Competing action' if slot not in winners else
                           'Held discrete action' if discrete and slot in latched else 'Proposed')
@@ -125,7 +114,7 @@ def main():
                     commands.append(command)
                 if slot in winners and discrete:
                     latched.add(slot)
-            print(json.dumps({'kind': 'prediction', 'frameAt': stamp, 'sensorAt': sensor_time,
+            print(json.dumps({'kind': 'prediction', 'frameAt': captured_at, 'decisionAt': stamp, 'sensorAt': sensor_time,
                               'latencyMs': (time.perf_counter() - before) * 1000,
                               'sensorPresent': sensor_vector(sensors)[len(SENSORS):].tolist(),
                               'commands': commands, 'proposals': proposals, 'threshold': args.threshold}, allow_nan=False), flush=True)

server/src/services/roverLearningService/workers/learning.py

@@ -0,0 +1,181 @@
+"""Training objective and bounded, observational learning diagnostics."""
+import copy
+import time
+
+import numpy as np
+import torch
+from torch.nn import functional as F
+
+from policy import SLOTS, VALUES, select_slots, decode_command, is_stop
+
+
+def loss_setup(samples, schema):
+    numeric_mask = torch.zeros(SLOTS, VALUES)
+    for slot, descriptor in enumerate(schema):
+        numeric_mask[slot, :len(descriptor['fields'])] = 1
+    positives = np.sum([sample[2] for sample in samples], axis=0)
+    opportunities = np.sum([sample[4] for sample in samples], axis=0)
+    weight = torch.from_numpy(np.clip((opportunities - positives) / np.maximum(positives, 1), 1, 20)).float()
+    return numeric_mask, weight
+
+
+def tensors(samples):
+    images, states, targets, values, available = [torch.from_numpy(np.stack(items)) for items in zip(*samples)]
+    return images.float().div_(255), states, targets, values, available
+
+
+def losses(logits, predictions, targets, values, available, numeric_mask, positive_weight):
+    events = F.binary_cross_entropy_with_logits(logits, targets, pos_weight=positive_weight, reduction='none')
+    events = (events * available).sum() / available.sum().clamp_min(1)
+    mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1)
+    parameters = ((predictions - values).square() * mask).sum() / mask.sum().clamp_min(1)
+    return events, parameters
+
+
+def distribution(values):
+    values = np.asarray(values, dtype=np.float64)
+    if not values.size:
+        return {'count': 0, 'mean': None, 'p50': None, 'p95': None, 'max': None}
+    return {'count': int(values.size), 'mean': float(values.mean()),
+            'p50': float(np.quantile(values, .5)), 'p95': float(np.quantile(values, .95)), 'max': float(values.max())}
+
+
+def decisions(scores, schema, threshold):
+    selected = np.zeros_like(scores, dtype=bool)
+    for row, vector in enumerate(scores):
+        for slot in select_slots(vector, schema, threshold):
+            selected[row, slot] = True
+    return selected
+
+
+def counts(predicted, actual):
+    matched = int((predicted & actual).sum())
+    proposed, wanted = int(predicted.sum()), int(actual.sum())
+    empty = ~actual.any(axis=1)
+    return {'predictedActions': proposed, 'positiveActions': wanted, 'matchedActions': matched,
+            'precision': matched / max(1, proposed), 'recall': matched / max(1, wanted),
+            'f1': 2 * matched / max(1, proposed + wanted),
+            'falsePositiveActions': proposed - matched,
+            'noCommandWindows': int(empty.sum()), 'unwantedCommandWindows': int((empty & predicted.any(axis=1)).sum())}
+
+
+def summarize(scores, actual, schema, threshold, frequencies):
+    active_scores = scores[:, :len(schema)]
+    labels = actual[:, :len(schema)]
+    thresholds = sorted(set([.1, .3, .5, .7, .9, threshold]))
+    sweep = [{'threshold': value, **counts(decisions(scores, schema, value), actual)} for value in thresholds]
+    frequent = np.broadcast_to(frequencies, scores.shape)
+    baseline = [{'threshold': value, **counts(decisions(frequent, schema, value), actual)} for value in thresholds]
+    return {'scoreOnHumanCommands': distribution(active_scores[labels]),
+            'scoreOnAbsentCommands': distribution(active_scores[~labels]),
+            'maxScoreOnNoCommandWindows': distribution(active_scores[~actual.any(axis=1)].max(axis=1)) if (~actual.any(axis=1)).any() else distribution([]),
+            'thresholds': sweep, 'baselines': {'sendNothing': counts(np.zeros_like(actual), actual),
+                'trainingFrequency': baseline, 'frequencySource': 'Training windows only; no held-out fitting'},
+            'perCommandScores': [{'slot': slot, 'onHumanCommand': distribution(scores[actual[:, slot], slot]),
+                                   'otherwise': distribution(scores[~actual[:, slot], slot])} for slot in range(len(schema))]}
+
+
+def evaluate(model, samples, schema, threshold, frequencies, batch_size, cancelled):
+    """Same score selection as runtime; isolated windows cannot measure latching."""
+    model.eval()
+    score_rows, target_rows = [], []
+    value_error = value_count = 0
+    decoded = {}
+    with torch.inference_mode():
+        for offset in range(0, len(samples), batch_size):
+            if cancelled():
+                break
+            images, states, targets, values, _ = tensors(samples[offset:offset + batch_size])
+            logits, numeric = model(images, states)
+            if not torch.isfinite(logits).all() or not torch.isfinite(numeric).all():
+                raise ValueError('Nonfinite evaluation output')
+            score_rows.append(logits.sigmoid().numpy())
+            target_rows.append(targets.numpy().astype(bool))
+            for row, slot in targets.nonzero().tolist():
+                descriptor = schema[slot]
+                n = len(descriptor['fields'])
+                value_error += (numeric[row, slot, :n] - values[row, slot, :n]).square().sum().item()
+                value_count += n
+                prediction = decode_command(descriptor, numeric[row, slot])
+                expected = decode_command(descriptor, values[row, slot])
+                for field in descriptor['fields']:
+                    a, b = prediction, expected
+                    for key in field['path']:
+                        a, b = a[key], b[key]
+                    key = descriptor['template']['type'] + '.' + '.'.join(map(str, field['path']))
+                    total, count = decoded.get(key, (0, 0))
+                    decoded[key] = (total + abs(a - b), count + 1)
+    if not score_rows:
+        return {'windows': 0, 'complete': False}
+    scores, actual = np.concatenate(score_rows), np.concatenate(target_rows)
+    predicted = decisions(scores, schema, threshold)
+    stop_slots = [i for i, d in enumerate(schema) if is_stop(d)]
+    move_slots = [i for i, d in enumerate(schema) if d['template']['type'] == 'drive' and not is_stop(d)]
+    stop_count = int(actual[:, stop_slots].sum())
+    stop_match = int((actual[:, stop_slots] & predicted[:, stop_slots]).sum())
+    move_count = int(actual[:, move_slots].sum())
+    move_match = int((actual[:, move_slots] & predicted[:, move_slots]).sum())
+    return {'windows': len(scores), 'complete': len(scores) == len(samples),
+            'stopEvents': stop_count, 'missedStops': stop_count - stop_match,
+            'stopRecall': stop_match / max(1, stop_count),
+            'falseStops': int((predicted[:, stop_slots] & ~actual[:, stop_slots]).sum()),
+            'movementEvents': move_count, 'movementRecall': move_match / max(1, move_count),
+            'commands': [{'slot': i, 'command': d['template'], 'actual': int(actual[:, i].sum()),
+                          'predicted': int(predicted[:, i].sum()),
+                          'matched': int((actual[:, i] & predicted[:, i]).sum())} for i, d in enumerate(schema)],
+            'numericMse': value_error / max(1, value_count),
+            'decodedParameterMaeBeforeServerLimits': {key: total / count for key, (total, count) in decoded.items()},
+            **counts(decisions(scores, schema, threshold), actual),
+            **summarize(scores, actual, schema, threshold, frequencies)}
+
+
+def probe(model, samples, schema, threshold, frequencies, cancelled, max_seconds=20):
+    """Try fitting a small real training subset on a disposable copy, never publish it."""
+    started = time.monotonic()
+    # Include both event and quiet windows; source is training only. No random
+    # state/optimizer changes escape into the continuing learner.
+    quiet = [row for row in samples if not row[2].any()][:4]
+    # A reservoir is grouped by category: taking its first events could probe
+    # only zero-speed drive and falsely suggest all controls were learnable.
+    by_slot = [[i for i, row in enumerate(samples) if row[2][slot]] for slot in range(len(schema))]
+    chosen = []
+    for rank in range(8):
+        for indices in by_slot:
+            if rank < len(indices) and indices[rank] not in chosen and len(chosen) < 8:
+                chosen.append(indices[rank])
+    subset = [samples[i] for i in chosen] + quiet
+    if not chosen:
+        return {'status': 'No command examples', 'steps': 0}
+    with torch.random.fork_rng(devices=[]):
+        candidate = copy.deepcopy(model)
+        optimizer = torch.optim.AdamW(candidate.parameters(), lr=.001)
+        numeric_mask, weights = loss_setup(subset, schema)
+        images, states, targets, values, available = tensors(subset)
+        initial_loss = final_loss = gradient_norm = None
+        completed = 0
+        expired = lambda: cancelled() or time.monotonic() - started >= max_seconds
+        for _ in range(80):
+            if expired():
+                break
+            candidate.train()
+            optimizer.zero_grad(set_to_none=True)
+            logits, predictions = candidate(images, states)
+            event_loss, value_loss = losses(logits, predictions, targets, values, available, numeric_mask, weights)
+            loss = event_loss + value_loss
+            if not torch.isfinite(loss):
+                raise ValueError('Nonfinite learning probe loss')
+            if initial_loss is None:
+                initial_loss = loss.item()
+            loss.backward()
+            norm = torch.nn.utils.clip_grad_norm_(candidate.parameters(), 1)
+            if not torch.isfinite(norm):
+                raise ValueError('Nonfinite learning probe gradient')
+            gradient_norm = float(norm)
+            optimizer.step()
+            final_loss = loss.item()
+            completed += 1
+        result = evaluate(candidate, subset, schema, threshold, frequencies, 16, cancelled)
+    return {'status': 'completed' if completed == 80 else 'budget limited', 'steps': completed,
+            'windows': len(subset), 'initialLoss': initial_loss, 'lastStepLoss': final_loss,
+            'lastGradientNormBeforeClipping': gradient_norm, 'elapsedSeconds': time.monotonic() - started,
+            'evaluation': result, 'meaning': 'Memorization on a disposable copy of training data, not generalization'}

server/src/services/roverLearningService/workers/observations.py

@@ -0,0 +1,45 @@
+"""Shared causal 10 Hz sampling for recorded and live video."""
+import math
+
+
+def decision_frames(frames):
+    """Yield decision time, latest past image, and its actual capture timestamp.
+
+    Sampling every third/fourth decoded frame leaves holes at rates such as 25
+    FPS. A fixed grid covers each 100ms command interval exactly once. A frame
+    after the decision can release the pending tick, but is never its input.
+    """
+    previous = None
+    previous_time = None
+    tick = None
+    for frame in frames:
+        if frame.pts is None:
+            raise ValueError('Video frame has no timestamp')
+        stamp = float(frame.pts * frame.time_base) * 1000
+        if not math.isfinite(stamp) or (previous_time is not None and stamp < previous_time):
+            raise ValueError('Invalid/nonmonotonic video timestamps')
+        if tick is None:
+            tick = stamp
+        while tick < stamp - .01:
+            # Do not synthesize observations over missing/stalled video. History
+            # selection will reject a temporal gap until fresh history exists.
+            if previous is not None and tick - previous_time <= 200:
+                yield tick, previous, previous_time
+            elif stamp - tick > 200:
+                tick += max(1, math.floor((stamp - tick) / 100)) * 100
+                continue
+            tick += 100
+        if abs(stamp - tick) <= .01:
+            yield tick, frame, stamp
+            tick += 100
+        previous, previous_time = frame, stamp
+
+
+def history_frames(history, stamp):
+    selected = []
+    for offset in (900, 600, 300, 0):
+        candidates = [row for row in history if row[0] <= stamp - offset + .01]
+        if not candidates or stamp - offset - candidates[-1][0] > 200:
+            return []
+        selected.append(candidates[-1])
+    return selected

server/src/services/roverLearningService/workers/policy.py

@@ -4,6 +4,8 @@ Slots describe command shapes, not arbitrary executable instructions. Numeric
 leaves are regressed; string/bool/list values remain exact template constants.
 No command is sent by this module. Execution must use server authorization.
 """
+import base64
+import itertools
 import copy
 import json
 import math
@@ -12,12 +14,13 @@ import numpy as np
 import torch
 from torch import nn
 
-VERSION = 3
+VERSION = 5
 SLOTS = 64
 VALUES = 8
 FRAMES = 4
 HEIGHT, WIDTH = 120, 160
-OPERATING_TYPES = {"drive", "motors", "servo", "headlight", "laser", "horn", "peripheral", "song", "raw"}
+OPERATING_TYPES = {"drive", "servo", "headlight", "raw"}
+MODE_COMMANDS = tuple(base64.b64encode(bytes([opcode])).decode() for opcode in (128, 131, 132, 143))
 SENSORS = [
     ("bumpsAndWheelDrops.bumpLeft", 1), ("bumpsAndWheelDrops.bumpRight", 1),
     ("bumpsAndWheelDrops.wheelDropLeft", 1), ("bumpsAndWheelDrops.wheelDropRight", 1),
@@ -54,6 +57,10 @@ def sensor_vector(sensors):
 def describe_command(command):
     if command.get('type') not in OPERATING_TYPES:
         return None
+    # Raw is restricted to the existing mode/dock buttons, not arbitrary OI
+    # packets that could reintroduce excluded actuators or songs.
+    if command['type'] == 'raw' and command.get('raw') not in MODE_COMMANDS:
+        return None
     if len(json.dumps(command)) > 4096:
         raise ValueError('Operating command exceeds 4096-byte schema limit')
     values, fields = [], []
@@ -61,20 +68,17 @@ def describe_command(command):
     def visit(value, path=()):
         if isinstance(value, dict):
             return {key: visit(item, (*path, key)) for key, item in sorted(value.items())}
-        # Exact zeros preserve demonstrated releases/stops. Identity fields must
-        # also stay exact: regression must never invent a peripheral address.
-        if value == 0 or (path and path[-1] in ('id', 'control')):
+        # Exact actuator zeros preserve demonstrated releases.
+        if value == 0:
             return copy.deepcopy(value)
         if isinstance(value, (int, float)) and not isinstance(value, bool):
             if not math.isfinite(value) or len(values) >= VALUES:
                 raise ValueError('Nonfinite command or too many numeric control fields')
-            scale = {'drive': 500, 'motors': 255, 'servo': 180, 'peripheral': 255,
-                     'horn': 1000, 'song': 127}.get(command['type'], 1)
-            fields.append({'path': list(path), 'scale': scale, 'integer': isinstance(value, int)})
+            scale = {'drive': 500, 'servo': 180}.get(command['type'], 1)
+            fields.append({'path': list(path), 'scale': scale, 'integer': command['type'] in ('drive', 'servo') or isinstance(value, int)})
             values.append(normalize(value, scale))
             return {'$number': len(values) - 1}
-        # Arrays (e.g. songs) are discrete exact demonstrations, not variable
-        # length regression outputs. Never reinterpret raw OI bytes as numbers.
+        # Mode bytes and named actions remain exact constants.
         return copy.deepcopy(value)
 
     template = visit(command)
@@ -92,7 +96,7 @@ def encode_command(command, schema, extend=False):
         if not extend:
             raise ValueError('Command shape missing from model schema')
         if len(schema) >= SLOTS:
-            raise ValueError('64 command shapes exhausted; recording skipped, not silently truncated')
+            raise ValueError('64 command shapes exhausted; affected windows excluded')
         slot = len(schema)
         schema.append(descriptor)
     vector = np.zeros(VALUES, dtype=np.float32)
@@ -100,10 +104,25 @@ def encode_command(command, schema, extend=False):
     return slot, vector
 
 
+def initial_schema():
+    # Match the retained commandPipeline.js controls. Values describe fields,
+    # not artificial demonstrations; zero-speed drive remains a drive command.
+    commands = []
+    for left, right in itertools.product((0, 1), repeat=2):
+        commands.append({'type': 'drive', 'driveDirect': {'left': left, 'right': right}})
+    for angle in (0, 1):
+        commands.append({'type': 'servo', 'servo': {'angle': angle}})
+    for action in ('on', 'off', 'toggle'):
+        commands.append({'type': 'headlight', 'headlight': {'action': action}})
+    for raw in MODE_COMMANDS:
+        commands.append({'type': 'raw', 'raw': raw})
+    schema = []
+    for command in commands:
+        encode_command(command, schema, extend=True)
+    return schema
+
+
 def command_family(command):
-    if command['type'] == 'peripheral':
-        peripheral = command.get('peripheral') or {}
-        return ('peripheral', str(peripheral.get('id')), str(peripheral.get('control')))
     return (command['type'],)
 
 
@@ -162,7 +181,7 @@ class Policy(nn.Module):
 
 
 def specification(schema):
-    return {'version': VERSION, 'architecture': 'spatial-cnn-gru-events-v3',
+    return {'version': VERSION, 'architecture': 'spatial-cnn-gru-events-v5',
             'frames': FRAMES, 'height': HEIGHT, 'width': WIDTH,
             'historyStrideMs': 300, 'decisionIntervalMs': 100,
             'stateSize': STATE_SIZE, 'sensorFields': SENSORS,

server/src/services/roverLearningService/workers/train.py

@@ -39,13 +39,13 @@ def main():
     import numpy as np
     import av
     import torch
-    from torch.nn import functional as functional
+    from learning import loss_setup, tensors, losses as policy_losses, evaluate, probe
     from dataset import windows
-    from policy import Policy, VERSION, SLOTS, VALUES, specification, is_stop, select_slots
+    from policy import Policy, VERSION, SLOTS, VALUES, specification, is_stop, select_slots, initial_schema
 
     if args.check_dependencies:
         emit('dependencies', torch=str(torch.__version__), numpy=str(np.__version__), av=str(av.__version__),
-             parameters=sum(p.numel() for p in Policy().parameters()), specification=specification([]))
+             parameters=sum(p.numel() for p in Policy().parameters()), specification=specification(initial_schema()))
         return
     if not args.job:
         parser.error('--job is required unless --check-dependencies is used')
@@ -72,12 +72,13 @@ def main():
     optimizer = torch.optim.AdamW(model.parameters(), lr=0.0001)
     # The command-event policy has different inputs, targets and weights. Keep its
     # optimizer lineage separate; reuse recordings with their unchanged partition.
-    resume_path = root / 'training' / 'resume-v4.pt'
-    schema, steps, examples, last_published = [], 0, 0, 0
+    resume_path = root / 'training' / 'resume-v6.pt'
+    schema, steps, examples, last_published = initial_schema(), 0, 0, 0
     trained_rovers = set()
     control_profiles = []
     distinct_windows = {}
     published_windows = 0
+    last_probe_steps, learning_probe = 0, None
     if resume_path.exists():
         saved = torch.load(resume_path, map_location='cpu', weights_only=True)
         if saved['version'] != VERSION:
@@ -91,6 +92,8 @@ def main():
         control_profiles = saved.get('controlProfiles', [])
         distinct_windows = saved.get('distinctWindows', {})
         published_windows = saved.get('publishedWindows', 0)
+        last_probe_steps = saved.get('lastProbeSteps', 0)
+        learning_probe = saved.get('learningProbe')
     cpu_started = time.process_time()
     decode_started = time.monotonic()
     dataset_stats = []
@@ -113,7 +116,7 @@ def main():
         held_out = is_held_out(session_id)
         try:
             directory = root / 'recordings' / session_id
-            for sample in windows(directory, candidate_schema, cancelled, stats):
+            for sample in windows(directory, candidate_schema, cancelled, stats, extend=not held_out):
                 seen += 1
                 # Keep evaluation naturally distributed. Training reserves space
                 # for recorded commands and releases so periods without commands
@@ -137,15 +140,15 @@ def main():
             if cancelled():
                 break  # Do not call a partially decoded recording validated.
             if not samples:
+                if held_out and stats.get('unsupportedCommandEvents'):
+                    dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': 0,
+                                          'skipReason': 'No supported aligned windows'})
+                    continue
                 raise ValueError('No complete, aligned observation/action windows')
             # Entire sessions stay in one partition across all jobs. Neighboring
             # video windows must never straddle training and validation.
             held_out = is_held_out(session_id)
             if held_out:
-                if candidate_schema != schema:
-                    dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': len(samples),
-                                          'skipReason': 'Vocabulary absent from training data'})
-                    continue  # Retry after training data introduces these command shapes.
                 validation.extend(samples)
             else:
                 schema = candidate_schema
@@ -171,15 +174,16 @@ def main():
         return
 
     decode_seconds = time.monotonic() - decode_started
+    audit_started = time.monotonic()
+    numeric_mask, positive_weight = loss_setup(pool, schema)
+    training_rows = [row for row in dataset_stats if row.get('partition') == 'training']
+    natural_count = sum(row.get('usableWindows', 0) for row in training_rows)
+    frequencies = np.sum([row.get('commandEventCounts', [0] * SLOTS) for row in training_rows], axis=0) / max(1, natural_count)
+    audit_subset = pool[::max(1, len(pool) // 96)][:96]
+    before_training = evaluate(model, audit_subset, schema, job['actionThreshold'], frequencies, config['batchSize'], cancelled)
+    before_audit_seconds = time.monotonic() - audit_started
     train_started = time.monotonic()
-    numeric_mask = torch.zeros(SLOTS, VALUES)
-    for slot, descriptor in enumerate(schema):
-        numeric_mask[slot, :len(descriptor['fields'])] = 1
-    # Balance sparse events without assuming that every time step should fire
-    # every control. This is training loss, not a driving-quality evaluation.
-    positives = np.sum([sample[2] for sample in pool], axis=0)
-    opportunities = np.sum([sample[4] for sample in pool], axis=0)
-    positive_weight = torch.from_numpy(np.clip((opportunities - positives) / np.maximum(positives, 1), 1, 20)).float()
+    gradient_norms = []
     losses = []
     model.train()
     for _ in range(config['passesPerJob']):
@@ -188,20 +192,18 @@ def main():
             if cancelled() or steps - initial_steps >= config['maxStepsPerJob']:
                 break
             batch = pool[offset:offset + config['batchSize']]
-            images, states, targets, values, available = [torch.from_numpy(np.stack(items)) for items in zip(*batch)]
-            images = images.float().div_(255)
+            images, states, targets, values, available = tensors(batch)
             optimizer.zero_grad(set_to_none=True)
             action_logits, predictions = model(images, states)
-            event_loss = functional.binary_cross_entropy_with_logits(action_logits, targets,
-                                                                       pos_weight=positive_weight, reduction='none')
-            event_loss = (event_loss * available).sum() / available.sum().clamp_min(1)
-            mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1)
-            value_loss = ((predictions - values).square() * mask).sum() / mask.sum().clamp_min(1)
+            event_loss, value_loss = policy_losses(action_logits, predictions, targets, values, available, numeric_mask, positive_weight)
             loss = event_loss + value_loss
             if not torch.isfinite(loss):
                 raise ValueError('Nonfinite training loss; checkpoint not saved')
             loss.backward()
-            torch.nn.utils.clip_grad_norm_(model.parameters(), 1)
+            gradient_norm = torch.nn.utils.clip_grad_norm_(model.parameters(), 1)
+            if not torch.isfinite(gradient_norm):
+                raise ValueError('Nonfinite training gradient; checkpoint not saved')
+            gradient_norms.append(float(gradient_norm))
             optimizer.step()
             steps += 1
             examples += len(batch)
@@ -219,89 +221,65 @@ def main():
     evaluation_started = time.monotonic()
     distinct_windows.update(observed_windows)
     unique_count = sum(distinct_windows.values())
-    evaluation = {'status': 'waiting for held-out sessions', 'windows': len(validation)}
+    coverage = [row for row in dataset_stats if row.get('partition') == 'heldOut']
+    unsupported_events = sum(row.get('unsupportedCommandEvents', 0) for row in coverage)
+    unsupported_windows = sum(row.get('unsupportedWindows', 0) for row in coverage)
+    evaluation = {'status': 'No supported held-out windows' if unsupported_events and not validation else 'waiting for held-out sessions',
+                  'windows': len(validation), 'unsupportedCommandEvents': unsupported_events,
+                  'unsupportedWindows': unsupported_windows}
     if validation and not cancelled():
-        model.eval()
-        true_positive = predicted_positive = actual_positive = 0
-        value_error = value_count = evaluated = 0
-        stop_actual = stop_detected = false_stops = 0
-        drive_actual = drive_detected = 0
-        empty_windows = unwanted_windows = 0
-        stop_slots = [slot for slot, descriptor in enumerate(schema) if is_stop(descriptor)]
-        drive_slots = [slot for slot, descriptor in enumerate(schema)
-                       if descriptor['template']['type'] == 'drive' and not is_stop(descriptor)]
-        per_command = [{'slot': slot, 'command': descriptor['template'], 'actual': 0,
-                        'predicted': 0, 'matched': 0} for slot, descriptor in enumerate(schema)]
-        with torch.inference_mode():
-            for offset in range(0, len(validation), config['batchSize']):
-                if cancelled():
-                    break
-                batch = validation[offset:offset + config['batchSize']]
-                images, states, targets, values, available = [torch.from_numpy(np.stack(items)) for items in zip(*batch)]
-                logits, predictions = model(images.float() / 255, states)
-                predicted = torch.zeros_like(targets, dtype=torch.bool)
-                # Use the same threshold and competing-command selection as live
-                # inference. Reservoir samples cannot assess temporal latching or
-                # closed-loop motion; report this explicitly rather than implying it.
-                for row, scores in enumerate(logits.sigmoid().tolist()):
-                    for slot in select_slots(scores, schema, job['actionThreshold']):
-                        predicted[row, slot] = True
-                actual = targets.bool()
-                true_positive += (predicted & actual).sum().item()
-                predicted_positive += predicted.sum().item()
-                actual_positive += actual.sum().item()
-                mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1)
-                value_error += ((predictions - values).square() * mask).sum().item()
-                value_count += mask.sum().item()
-                empty = ~actual.any(dim=1)
-                empty_windows += empty.sum().item()
-                unwanted_windows += (empty & predicted.any(dim=1)).sum().item()
-                if stop_slots:
-                    stop_actual += actual[:, stop_slots].sum().item()
-                    stop_detected += (actual[:, stop_slots] & predicted[:, stop_slots]).sum().item()
-                    false_stops += (predicted[:, stop_slots] & ~actual[:, stop_slots]).sum().item()
-                if drive_slots:
-                    drive_actual += actual[:, drive_slots].sum().item()
-                    drive_detected += (actual[:, drive_slots] & predicted[:, drive_slots]).sum().item()
-                for item in per_command:
-                    slot = item['slot']
-                    item['actual'] += actual[:, slot].sum().item()
-                    item['predicted'] += predicted[:, slot].sum().item()
-                    item['matched'] += (actual[:, slot] & predicted[:, slot]).sum().item()
-                evaluated += len(batch)
-        precision = true_positive / max(1, predicted_positive)
-        recall = true_positive / max(1, actual_positive)
-        mse = value_error / max(1, value_count)
-        stop_recall = stop_detected / max(1, stop_actual)
+        measured = evaluate(model, validation, schema, job['actionThreshold'], frequencies, config['batchSize'], cancelled)
+        evaluated = measured['windows']
+        precision, recall = measured.get('precision', 0), measured.get('recall', 0)
+        actual_positive = measured.get('positiveActions', 0)
+        mse = measured.get('numericMse', 0)
+        stop_actual = measured.get('stopEvents', 0)
+        stop_recall = measured.get('stopRecall', 0)
+        drive_actual = measured.get('movementEvents', 0)
+        drive_recall = measured.get('movementRecall', 0)
+        frequency_f1 = max((row['f1'] for row in measured.get('baselines', {}).get('trainingFrequency', [])), default=0)
         held_out_sessions = sum(row.get('partition') == 'heldOut' and not row.get('skipReason') for row in dataset_stats)
         gate_reasons = []
         for condition, reason in [
+            (unsupported_events == 0, 'Held-out controls are not fully represented; see unsupportedCommands'),
             (evaluated == len(validation), 'Evaluation interrupted'),
             (held_out_sessions >= 2, 'Need two usable held-out recordings'),
             (evaluated >= 128, 'Need 128 held-out windows'),
             (actual_positive >= 32, 'Need 32 held-out command events'),
             (stop_actual >= 8, 'Need eight held-out stop events'),
             (drive_actual >= 16, 'Need 16 held-out movement commands'),
-            (drive_detected / max(1, drive_actual) >= .5, 'Movement command recall below 0.5'),
+            (drive_recall >= .5, 'Movement command recall below 0.5'),
             (precision >= .5, 'Command precision below 0.5'),
             (recall >= .5, 'Command recall below 0.5'),
             (stop_recall >= .5, 'Stop recall below 0.5'),
-            (value_count == 0 or mse <= .1, 'Command parameter error above 0.1'),
+            (measured.get('f1', 0) > frequency_f1, 'Command F1 does not beat the training-frequency baseline'),
+            (mse <= .1, 'Command parameter error above 0.1'),
         ]:
             if not condition:
                 gate_reasons.append(reason)
-        evaluation = {'status': 'imitation gate not met' if gate_reasons else 'passed imitation gate',
-                      'method': 'Held-out 100ms command events; not a closed-loop rollout',
+        evaluation = {**measured, 'status': 'imitation gate not met' if gate_reasons else 'passed imitation gate',
+                      'unsupportedCommandEvents': unsupported_events, 'unsupportedWindows': unsupported_windows,
+                      'method': 'Isolated held-out 100ms events; live selection, no temporal latch/physical rollout',
                       'threshold': job['actionThreshold'], 'heldOutSessions': held_out_sessions,
-                      'windows': evaluated, 'precision': precision, 'recall': recall,
-                      'numericMse': mse, 'positiveActions': actual_positive,
-                      'predictedActions': predicted_positive, 'falsePositiveActions': predicted_positive - true_positive,
-                      'stopEvents': stop_actual, 'missedStops': stop_actual - stop_detected,
-                      'stopRecall': stop_recall, 'falseStops': false_stops,
-                      'movementEvents': drive_actual, 'movementRecall': drive_detected / max(1, drive_actual),
-                      'noCommandWindows': empty_windows, 'unwantedCommandWindows': unwanted_windows,
-                      'commands': per_command, 'gateReasons': gate_reasons}
-    evaluation_seconds = time.monotonic() - evaluation_started
+                      'gateReasons': gate_reasons}
+    after_training = evaluate(model, audit_subset, schema, job['actionThreshold'], frequencies, config['batchSize'], cancelled)
+    probe_seconds = 0
+    if not cancelled() and (learning_probe is None or steps - last_probe_steps >= 1000):
+        remaining = config['maxJobSeconds'] - (time.monotonic() - started)
+        if remaining > 10:
+            probe_started = time.monotonic()
+            learning_probe = probe(model, audit_subset, schema, job['actionThreshold'], frequencies,
+                                   cancelled, max_seconds=min(20, remaining / 2))
+            learning_probe['atTrainingStep'] = steps
+            last_probe_steps = steps
+            probe_seconds = time.monotonic() - probe_started
+    training_audit = {'before': before_training, 'after': after_training,
+                      'gradientNormBeforeClipping': {'min': min(gradient_norms), 'max': max(gradient_norms)},
+                      'positiveWeights': positive_weight[:len(schema)].tolist(),
+                      'naturalTrainingCommandFrequency': frequencies[:len(schema)].tolist(),
+                      'sampledCommandFrequency': np.mean([row[2] for row in pool], axis=0)[:len(schema)].tolist(),
+                      'learningProbe': learning_probe}
+    evaluation_seconds = time.monotonic() - evaluation_started - probe_seconds
     publication = {'distinctMinutes': unique_count / 600,
                    'newMinutes': (unique_count - published_windows) / 600,
                    'minimumMinutes': config['minimumDrivingMinutes'],
@@ -354,9 +332,9 @@ def main():
         'schema': schema, 'steps': steps, 'examples': examples,
         'lastPublished': last_published, 'trainedRovers': sorted(trained_rovers),
         'controlProfiles': control_profiles, 'distinctWindows': distinct_windows,
-        'publishedWindows': published_windows,
+        'publishedWindows': published_windows, 'lastProbeSteps': last_probe_steps, 'learningProbe': learning_probe,
     }, resume_path)
-    diagnostics = {'dataset': dataset_stats, 'decodeSeconds': decode_seconds,
+    diagnostics = {'dataset': dataset_stats, 'trainingAudit': training_audit, 'probeSeconds': probe_seconds, 'beforeAuditSeconds': before_audit_seconds, 'decodeSeconds': decode_seconds,
                    'trainingSeconds': training_seconds, 'evaluationSeconds': evaluation_seconds, 'checkpointSeconds': time.monotonic() - save_started,
                    'cpuSeconds': time.process_time() - cpu_started,
                    'peakRssMiB': resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024,

server/src/services/roverManager/index.js

@@ -274,6 +274,7 @@ odometerService.odometerEvents.on('update', ({ roverId, odometer }) => {
 setInterval(tickPrivateAutoClose, PRIVATE_AUTO_CLOSE_TICK_MS);
 
 module.exports = {
+  stopDockGuard,
   upsertRover,
   removeRover,
   lockRover,

server/src/services/roverManager/rosterLifecycle.js

@@ -117,6 +117,7 @@ function createRosterLifecycle(deps) {
       offline, while removeRoverDrivers still receives the captured record so
       it can clean the reverse membership index and Socket.IO room membership.
     */
+    require('../dockingService').disconnect(record);
     rovers.delete(id);
     removeRoverDrivers(id, record);
     stopDockGuard(id);

server/src/services/roverManager/sensorPipeline.js

@@ -389,6 +389,10 @@ function createSensorPipeline(deps) {
 
   function evaluateDockGuard(record, sensors) {
     if (!record || !sensors) return;
+    if (sensors.docking?.phase === 'undocking' || sensors.docking?.phase === 'assisting') {
+      stopDockGuard(record.id);
+      return;
+    }
     const state = getDockGuardState(record.id);
     const now = Date.now();
     const docked = Boolean(sensors?.chargingSources?.homeBase);
@@ -564,6 +568,10 @@ function createSensorPipeline(deps) {
       };
       record.lastSensor = { raw: frame, decoded };
     }
+    if (decoded) {
+      decoded = { ...decoded, docking: require('../dockingService').processTelemetry(record, decoded) };
+      record.lastSensor = { raw: frame, decoded };
+    }
     /*
       Rover manager remains responsible only for decoding and routing sensor
       frames. The dedicated service receives the completed sensor object after
@@ -577,6 +585,7 @@ function createSensorPipeline(deps) {
     const overcurrentProtection = decoded && typeof processOvercurrentTelemetry === 'function'
       ? processOvercurrentTelemetry(roverId, decoded)
       : null;
+    record.lastOvercurrentProtection = overcurrentProtection;
     updateMovement(record, decoded);
     const hasDockInfo = decoded?.chargingSources != null;
     if (hasDockInfo) {
@@ -589,6 +598,10 @@ function createSensorPipeline(deps) {
     if (bumps?.bumpLeft || bumps?.bumpRight) record.lastBumpAt = Date.now();
     handlePrivateButtonHold(record, decoded);
     evaluatePrivateSafety(record, decoded);
+    if (decoded) {
+      decoded = { ...decoded, docking: require('../dockingService').snapshot(record) };
+      record.lastSensor = { raw: frame, decoded };
+    }
     io.to(record.room).volatile.emit('sensorFrame', {
       roverId,
       frame,

server/src/services/serverControlService/controller.js

@@ -1,6 +1,6 @@
 // Restricted Container Lifecycle Controller
 // Purpose: Pulls and replaces only the fixed MultiRover application container through Docker's local API.
-// Scope: Runs as the private lifecycle Compose service; it exposes no TCP listener and accepts no caller-selected targets.
+// Scope: Runs as the private lifecycle Compose service; it exposes no TCP listener and accepts tags only within the deployment-configured repository.
 const fs = require('fs');
 const path = require('path');
 const http = require('http');
@@ -9,6 +9,12 @@ const Docker = require('dockerode');
 const docker = new Docker({ socketPath: '/var/run/docker.sock' });
 const TARGET_CONTAINER_NAME = 'multirover';
 const TARGET_IMAGE = process.env.MULTIROVER_TARGET_IMAGE || 'ghcr.io/legop3/multiroombarover:latest';
+// A registry port is not a tag separator. Strip a digest before splitting the tag.
+const configuredName = TARGET_IMAGE.split('@')[0];
+const tagSeparator = configuredName.lastIndexOf(':');
+const hasConfiguredTag = tagSeparator > configuredName.lastIndexOf('/');
+const IMAGE_REPOSITORY = hasConfiguredTag ? configuredName.slice(0, tagSeparator) : configuredName;
+const DEFAULT_TAG = TARGET_IMAGE.includes('@') ? null : (hasConfiguredTag ? configuredName.slice(tagSeparator + 1) : 'latest');
 const SOCKET_PATH = process.env.MULTIROVER_LIFECYCLE_SOCKET || '/run/multirover/lifecycle.sock';
 // Status belongs beside the controller's private socket instead of in the
 // application's data volume. The controller runs as root for Docker access;
@@ -20,6 +26,32 @@ const HEALTH_POLL_MS = 1000;
 
 let operationRunning = false;
 let status = readStatus();
+// A repository change in Compose invalidates the previous repository's selection.
+if (status.selectedRepository !== IMAGE_REPOSITORY) {
+  status.selectedRepository = IMAGE_REPOSITORY;
+  status.selectedTag = DEFAULT_TAG;
+}
+
+function selectedImage() {
+  return status.selectedTag ? `${IMAGE_REPOSITORY}:${status.selectedTag}` : TARGET_IMAGE;
+}
+
+function publicStatus() {
+  return {
+    ...status,
+    configuredImage: TARGET_IMAGE,
+    repository: IMAGE_REPOSITORY,
+    targetImage: selectedImage(),
+  };
+}
+
+function requestedImage(tag) {
+  if (tag === null) return { imageReference: selectedImage(), tag: status.selectedTag };
+  if (!/^[a-zA-Z0-9_][a-zA-Z0-9_.-]{0,127}$/.test(tag)) {
+    throw new Error('Enter a valid container tag (1–128 letters, numbers, underscores, periods, or hyphens).');
+  }
+  return { imageReference: `${IMAGE_REPOSITORY}:${tag}`, tag };
+}
 
 function readStatus() {
   try {
@@ -52,11 +84,12 @@ function readStatus() {
 }
 
 function writeStatus(patch) {
-  status = { ...status, ...patch };
+  const nextStatus = { ...status, ...patch };
   fs.mkdirSync(path.dirname(STATUS_PATH), { recursive: true });
   const temporaryPath = `${STATUS_PATH}.tmp`;
-  fs.writeFileSync(temporaryPath, `${JSON.stringify(status, null, 2)}\n`, { encoding: 'utf8', mode: 0o600 });
+  fs.writeFileSync(temporaryPath, `${JSON.stringify(nextStatus, null, 2)}\n`, { encoding: 'utf8', mode: 0o600 });
   fs.renameSync(temporaryPath, STATUS_PATH);
+  status = nextStatus;
   return status;
 }
 
@@ -68,23 +101,23 @@ async function inspectTargetContainer() {
   return docker.getContainer(TARGET_CONTAINER_NAME).inspect();
 }
 
-async function inspectLocalTargetImage() {
+async function inspectLocalTargetImage(imageReference = selectedImage()) {
   try {
-    return await docker.getImage(TARGET_IMAGE).inspect();
+    return await docker.getImage(imageReference).inspect();
   } catch (error) {
     if (error.statusCode === 404) return null;
     throw error;
   }
 }
 
-async function pullTargetImage() {
+async function pullTargetImage(imageReference = selectedImage()) {
   const stream = await new Promise((resolve, reject) => {
-    docker.pull(TARGET_IMAGE, (error, pullStream) => (error ? reject(error) : resolve(pullStream)));
+    docker.pull(imageReference, (error, pullStream) => (error ? reject(error) : resolve(pullStream)));
   });
   await new Promise((resolve, reject) => {
     docker.modem.followProgress(stream, (error) => (error ? reject(error) : resolve()));
   });
-  const image = await inspectLocalTargetImage();
+  const image = await inspectLocalTargetImage(imageReference);
   if (!image) throw new Error('Docker finished pulling but the target image is unavailable.');
   return image;
 }
@@ -193,13 +226,14 @@ async function refreshImageStatus() {
   ]);
   return writeStatus({
     runningImage: container.Image,
+    runningImageReference: container.Config?.Image || null,
     availableImage: image?.Id || null,
     updateAvailable: image ? container.Image !== image.Id : null,
   });
 }
 
 async function runCheck() {
-  writeStatus({ message: `Pulling ${TARGET_IMAGE}…` });
+  writeStatus({ message: `Pulling ${selectedImage()}…` });
   await pullTargetImage();
   const current = await refreshImageStatus();
   return current.updateAvailable ? 'An application update is available.' : 'The application is current.';
@@ -213,9 +247,9 @@ async function runRestart() {
   return 'The application container restarted successfully.';
 }
 
-async function runUpdate() {
-  writeStatus({ message: `Pulling ${TARGET_IMAGE}…`, rollback: null });
-  const targetImage = await pullTargetImage();
+async function runUpdate({ imageReference, tag }) {
+  writeStatus({ message: `Pulling ${imageReference}…`, rollback: null });
+  const targetImage = await pullTargetImage(imageReference);
   const previousContainer = await inspectTargetContainer();
   const previousImage = previousContainer.Image;
   let replacementStarted = false;
@@ -228,11 +262,15 @@ async function runUpdate() {
       updateAvailable: previousImage !== targetImage.Id,
     });
     replacementStarted = true;
-    await replaceContainer({ template: previousContainer, imageReference: TARGET_IMAGE });
+    await replaceContainer({ template: previousContainer, imageReference: targetImage.Id });
     writeStatus({ message: 'Waiting for the updated application to become healthy…' });
     const healthy = await waitForHealthyContainer();
     writeStatus({
+      // Commit the tag only once the replacement passes its health check.
+      selectedRepository: IMAGE_REPOSITORY,
+      selectedTag: tag,
       runningImage: healthy.Image,
+      runningImageReference: healthy.Config?.Image || null,
       availableImage: targetImage.Id,
       updateAvailable: false,
     });
@@ -284,7 +322,7 @@ function queueOperation(operation, runner) {
       operationRunning = false;
     }
   }, 500);
-  return status;
+  return publicStatus();
 }
 
 function sendJson(response, statusCode, value) {
@@ -294,10 +332,11 @@ function sendJson(response, statusCode, value) {
 
 const server = http.createServer((request, response) => {
   try {
+    const url = new URL(request.url, 'http://localhost');
     if (request.method === 'GET' && request.url === '/status') {
       refreshImageStatus()
         .catch(() => status)
-        .then((current) => sendJson(response, 200, { available: true, ...current }));
+        .then(() => sendJson(response, 200, { available: true, ...publicStatus() }));
       return;
     }
     if (request.method === 'POST' && request.url === '/check') {
@@ -308,8 +347,10 @@ const server = http.createServer((request, response) => {
       sendJson(response, 202, { available: true, ...queueOperation('restart', runRestart) });
       return;
     }
-    if (request.method === 'POST' && request.url === '/update') {
-      sendJson(response, 202, { available: true, ...queueOperation('update', runUpdate) });
+    if (request.method === 'POST' && url.pathname === '/update') {
+      const target = requestedImage(url.searchParams.get('tag'));
+      queueOperation('update', () => runUpdate(target));
+      sendJson(response, 202, { available: true, ...publicStatus() });
       return;
     }
     sendJson(response, 404, { error: 'Lifecycle operation not found.' });

server/src/services/serverControlService/index.js

@@ -88,14 +88,14 @@ io.on('connection', (socket) => {
       .catch((error) => cb({ error: error.message, code: error.code || null }));
   });
 
-  socket.on('server:updateApplication', (_payload = {}, cb = () => {}) => {
+  socket.on('server:updateApplication', (payload = {}, cb = () => {}) => {
     Promise.resolve()
       .then(() => requireRecentPassword(socket))
-      .then(() => lifecycleClient.updateApplication())
+      .then(() => lifecycleClient.updateApplication(payload?.tag))
       .then((lifecycle) => {
         // Wait for controller acceptance so rejected updates do not kick users.
         notifyDriversOfRestart(actorFor(socket));
-        database.recordAuditEvent(actorFor(socket), 'application.update-requested', {});
+        database.recordAuditEvent(actorFor(socket), 'application.update-requested', { tag: payload?.tag ?? lifecycle.selectedTag });
         io.emit('server:restarting', { reason: 'application-update' });
         cb({ success: true, lifecycle });
       })

server/src/services/serverControlService/lifecycleClient.js

@@ -1,6 +1,6 @@
 // Container Lifecycle Client
 // Purpose: Sends fixed lifecycle requests from the application to the private controller socket.
-// Scope: This client cannot select images, containers, or Docker arguments; the controller owns those constants.
+// Scope: This client can request a tag; the controller owns the repository, container, and Docker arguments.
 const http = require('http');
 
 const LIFECYCLE_SOCKET_PATH = process.env.MULTIROVER_LIFECYCLE_SOCKET || '/run/multirover/lifecycle.sock';
@@ -68,5 +68,11 @@ module.exports = {
   checkForUpdate: () => requestLifecycleController({ method: 'POST', path: '/check' }),
   getLifecycleStatus,
   restartApplication: () => requestLifecycleController({ method: 'POST', path: '/restart' }),
-  updateApplication: () => requestLifecycleController({ method: 'POST', path: '/update' }),
+  updateApplication: (tag) => {
+    if (tag !== undefined && typeof tag !== 'string') throw new Error('Container tag must be a string.');
+    return requestLifecycleController({
+      method: 'POST',
+      path: tag === undefined ? '/update' : `/update?tag=${encodeURIComponent(tag)}`,
+    });
+  },
 };

server/src/services/sessionService/index.js

@@ -12,6 +12,7 @@ const { managerEvents } = roverManager;
 const assignmentService = require('../assignmentService');
 const { getActiveDrivers, getTurnQueues, canDrive, turnEvents } = require('../turnService');
 const { buildRoverTurn } = require('../turnService/display');
+const { getRoomAudioStreams, roomAudioEvents } = require('../roomAudioService');
 const { getRoomCameras, roomCameraEvents } = require('../roomCameraService');
 const {
   getPublicState: getPtzCameraState,
@@ -214,6 +215,7 @@ function buildSession(socket) {
       status: assignmentRoverId ? assignment.status : assignment.status === 'waiting' ? 'waiting' : null,
     },
     roomCameras: getRoomCameras(),
+    roomAudioStreams: getRoomAudioStreams(socket),
     ptzCamera: getPtzCameraState(socket),
     homeAssistant: getHomeAssistantState(),
     homeAssistantActivities: getActivityState(),
@@ -407,6 +409,8 @@ turnEvents.on('queue', (event = {}) => {
   syncAll();
 });
 
+roomAudioEvents.on('update', () => syncAll());
+
 roomCameraEvents.on('update', () => {
   logger.info('Room camera change detected; syncing all clients');
   syncAll();
@@ -459,7 +463,7 @@ balanceBoardEvents.on('change', () => {
 });
 
 replayEvents.on('update', () => {
-  logger.info('Replay cooldown updated; syncing all clients');
+  logger.info('Replay state updated; syncing all clients');
   syncAll();
 });
 

server/src/services/videoAuthService/httpRoute.js

@@ -11,6 +11,7 @@ function registerVideoAuthRoute(deps) {
     logAdminEvent,
     extractStreamInfoFromBody,
     canAccessStream,
+    trackRoomAudioSession,
   } = deps;
 
   app.post('/mediamtx/auth', (req, res) => {
@@ -36,7 +37,8 @@ function registerVideoAuthRoute(deps) {
     const isLoopback = ip === '127.0.0.1' || ip === '::1';
     const isRoverForwardAudioRead = action === 'read'
       && isRtspProtocol
-      && streamInfo?.id?.endsWith('-fwd');
+      && streamInfo?.type === 'rover'
+      && streamInfo.id.endsWith('-fwd');
     // Replay and snapshot workers read MediaMTX through loopback RTSP. They do
     // not represent a browser session. Rovers likewise read their dedicated
     // -fwd speaker feed without browser credentials; every other non-loopback
@@ -50,7 +52,7 @@ function registerVideoAuthRoute(deps) {
       callback for RTSP, so explicitly admit that publish protocol while leaving WHEP reads
       under the existing session and role checks below.
     */
-    if (action === 'publish' && isRtspProtocol) {
+    if (action === 'publish' && isRtspProtocol && (streamInfo?.type !== 'roomAudio' || isLoopback)) {
       return res.status(200).end();
     }
 
@@ -62,7 +64,9 @@ function registerVideoAuthRoute(deps) {
     const info = videoSessions.getSession(sessionId);
     const streamTypeMatches =
       info &&
-      (info.sourceType === streamInfo.type || (info.sourceType === 'roverMic' && streamInfo.type === 'rover'));
+      (info.sourceType === streamInfo.type
+        || (info.sourceType === 'roomMic' && streamInfo.type === 'roomAudio')
+        || (info.sourceType === 'roverMic' && streamInfo.type === 'rover'));
     if (!info || !streamTypeMatches || info.sourceId !== streamInfo.id) {
       logger.warn('invalid session %s for stream %s:%s', sessionId, streamInfo.type, streamInfo.id);
       return res.status(401).end();
@@ -78,6 +82,7 @@ function registerVideoAuthRoute(deps) {
       return res.status(401).end();
     }
 
+    if (streamInfo.type === 'roomAudio' && protocol === 'webrtc') trackRoomAudioSession(body.id, sessionId);
     return res.status(200).end();
   });
 }

server/src/services/videoAuthService/index.js

@@ -4,6 +4,7 @@
 const { app } = require('../../globals/http');
 const io = require('../../globals/io');
 const logger = require('../../globals/logger').child('videoAuth');
+const roomAudioService = require('../roomAudioService');
 const videoSessions = require('../videoSessions');
 const { getMode, MODES } = require('../modeManager');
 const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
@@ -28,6 +29,7 @@ const { canAccessStream } = createVideoAuthPolicy({
   turnService,
   roverManager,
   ptzCameraService,
+  roomAudioService,
   getSocketIp,
   isLocalNetwork,
   io,
@@ -42,6 +44,7 @@ registerVideoAuthRoute({
   logAdminEvent,
   extractStreamInfoFromBody,
   canAccessStream,
+  trackRoomAudioSession: roomAudioService.trackMediaSession,
 });
 
 module.exports = {};

server/src/services/videoAuthService/policy.js

@@ -17,6 +17,7 @@ function createVideoAuthPolicy(deps) {
     turnService,
     roverManager,
     ptzCameraService,
+    roomAudioService,
     getSocketIp,
     isLocalNetwork,
     io,
@@ -58,6 +59,11 @@ function createVideoAuthPolicy(deps) {
   }
 
   function canAccessStream({ socket, streamInfo, action, sourceType }) {
+    if (streamInfo.type === 'roomAudio') {
+      return action === 'publish'
+        ? sourceType === 'roomMic' && roomAudioService.canPublish(socket, streamInfo.id)
+        : action === 'read' && sourceType === 'roomAudio' && roomAudioService.canListen(socket) && roomAudioService.hasStream(streamInfo.id);
+    }
     if (!canView(socket)) {
       return false;
     }

server/src/services/videoAuthService/streamParsing.js

@@ -33,6 +33,8 @@ function extractStreamInfo(path) {
     return { type: 'rover', id: rawId, baseId };
   }
 
+  if (remaining.length === 2 && remaining[0] === 'room-audio') return { type: 'roomAudio', id: remaining[1] };
+
   if (remaining.length === 2 && remaining[0] === 'room') {
     return { type: 'room', id: remaining[1] || '' };
   }

server/src/services/videoSocketService/index.js

@@ -18,7 +18,9 @@ const {
 
 function buildWhepUrlForSource(source) {
   const segments = [];
-  if (source.type === 'room') {
+  if (source.type === 'roomAudio') {
+    segments.push('room-audio', encodeURIComponent(source.id));
+  } else if (source.type === 'room') {
     segments.push('room', encodeURIComponent(source.id));
   } else if (source.type === 'ptz') {
     /*
@@ -145,6 +147,9 @@ io.on('connection', (socket) => {
           */
           throw new Error('Live video is limited to this rover queue');
         }
+      } else if (target.type === 'roomAudio') {
+        const roomAudio = require('../roomAudioService');
+        if (!roomAudio.canListen(socket) || !roomAudio.hasStream(target.id)) throw new Error('Room audio unavailable');
       } else if (target.type === 'room') {
         throw new Error('Room cameras now use the snapshot feed');
       } else if (target.type === 'ptz') {

to-do.md

@@ -3,20 +3,9 @@
 research how midis can be played easier with drag and drop, auto selection based on which playback mode would fit, more useful and clear toggles, sample midi files to analyze how they would play on a single tone system, better live playback to see what notes are actually playing from each source and what simply wont play on the speaker
 and make sure each toggle and counter actually has a purpose besides debugging or something more useful to the user, note skipped is just for debugging
 ```
-1. add new discord channel: live status
-   1. one concise message which contains the status of all the rovers
-   2. bot clears the channel, then sends a new message
-   3. has a fancy big embed that shows way more stuff
-   4. also has a small text copy at the top (readable on smartwatch lol)
-      1. ```
-      freaky: docked
-      wall-e: user1 driving
-      bweeble: NEEDS HELP
-      ```
-2. setting to disable replay popups in spectator settings menu
-3. add admin ui for VIP and private requests instead of only through discord
-4. make roverd self update checkout to main branch
-5.  fix this:
+1. add admin ui for VIP and private requests instead of only through discord
+2. make roverd self update checkout to main branch
+3.  fix this:
 `Jun 18 15:14:18 roombaserver.local node[216731]: /home/daniel/MultiRoombaRover/server/src/services/roverManager/socketHandlers.js:92
 Jun 18 15:14:18 roombaserver.local node[216731]:         cb({ error: err.message });
 Jun 18 15:14:18 roombaserver.local node[216731]:         ^

webui/package.json

@@ -4,7 +4,9 @@
   "version": "0.0.0",
   "type": "module",
   "scripts": {
-    "dev": "VITE_ROVERD_URL=https://rover.otter.land vite",
+    "predev": "node scripts/prepareMediaMtx.js",
+    "dev": "VITE_ROVERD_URL=https://rovertest.otter.land vite",
+    "prebuild": "node scripts/prepareMediaMtx.js",
     "build": "vite build",
     "lint": "eslint .",
     "test": "node --test src/controls/inputs/*.test.js src/components/GamepadMappingSettings/*.test.js",

webui/scripts/prepareMediaMtx.js

@@ -0,0 +1,45 @@
+import { createHash } from 'node:crypto';
+import { mkdir, readFile, rename, writeFile } from 'node:fs/promises';
+
+const version = '1.15.3'; // Matches the Dockerfile's default MediaMTX version.
+const baseUrl = `https://raw.githubusercontent.com/bluenviron/mediamtx/v${version}/`;
+const output = new URL('../.cache/mediamtx/publisher.js', import.meta.url);
+const moduleHash = 'e903555223f196aca86d7adce5cf4437824435291bf9b47f284aa767fffabaaa';
+const hash = (content) => createHash('sha256').update(content).digest('hex');
+
+async function download(path, checksum) {
+  const response = await fetch(`${baseUrl}${path}`, { signal: AbortSignal.timeout(30000) });
+  if (!response.ok) throw new Error(`MediaMTX download failed: ${path} (${response.status})`);
+  const content = await response.text();
+  if (hash(content) !== checksum) throw new Error(`MediaMTX checksum mismatch: ${path}`);
+  return content;
+}
+
+let cached;
+try {
+  cached = await readFile(output, 'utf8');
+} catch (error) {
+  if (error.code !== 'ENOENT') throw error;
+}
+
+if (cached && hash(cached) === moduleHash) {
+  console.info(`Using cached MediaMTX publisher v${version}`);
+} else {
+  const [source, license] = await Promise.all([
+    download('internal/servers/webrtc/publisher.js', '6b4448939b45c30507bce59acaf434e3133e6efac3bd77d7e20fb01462958509'),
+    download('LICENSE', 'ecae73b0a23185a35a1222edc0aae5245bb9739420bad0125b0c42e935301a80'),
+  ]);
+  // Keep upstream behavior and its license; only adapt the browser-global export for Vite.
+  const module = `// MediaMTX v${version} publisher client, matching the Dockerfile default.\n`
+    + `// Source: ${baseUrl}internal/servers/webrtc/publisher.js\n`
+    + '// Only modification: export the class as an ES module for Vite.\n'
+    + `/*\n${license}\n*/\n`
+    + source.replace('window.MediaMTXWebRTCPublisher = MediaMTXWebRTCPublisher;', 'export default MediaMTXWebRTCPublisher;');
+  if (hash(module) !== moduleHash) throw new Error('Generated MediaMTX module checksum mismatch');
+  await mkdir(new URL('.', output), { recursive: true });
+  // Publish the verified module atomically so an interrupted download cannot poison the cache.
+  const temporary = new URL(`${output.href}.${process.pid}.tmp`);
+  await writeFile(temporary, module);
+  await rename(temporary, output);
+  console.info(`Downloaded and verified MediaMTX publisher v${version}`);
+}

webui/src/admin/api.js

@@ -28,7 +28,7 @@ export const deleteAdministrator = (socket, id) => emitAdminRequest(socket, 'adm
 export const restartApplication = (socket) => emitAdminRequest(socket, 'server:restartApplication');
 export const getApplicationLifecycleStatus = (socket) => emitAdminRequest(socket, 'server:lifecycleStatus');
 export const checkForApplicationUpdate = (socket) => emitAdminRequest(socket, 'server:checkForUpdate');
-export const updateApplication = (socket) => emitAdminRequest(socket, 'server:updateApplication');
+export const updateApplication = (socket, tag) => emitAdminRequest(socket, 'server:updateApplication', { tag });
 export const getBackupRestoreStatus = (socket) => emitAdminRequest(socket, 'backupRestore:status');
 export const createFullBackup = (socket) => emitAdminRequest(socket, 'backupRestore:createBackup');
 export const createRestoreUpload = (socket) => emitAdminRequest(socket, 'backupRestore:createRestoreUpload');

webui/src/admin/components/AdminOverview.jsx

@@ -21,6 +21,7 @@ export default function AdminOverview({ snapshot, socket, runSensitive, onSnapsh
   const [restartRequested, setRestartRequested] = useState(false);
   const [lifecycle, setLifecycle] = useState(null);
   const [lifecycleError, setLifecycleError] = useState('');
+  const [containerTag, setContainerTag] = useState(null);
 
   useEffect(() => {
     let mounted = true;
@@ -90,10 +91,13 @@ export default function AdminOverview({ snapshot, socket, runSensitive, onSnapsh
   }
 
   async function update() {
-    if (!window.confirm('Pull the configured MultiRover image and replace the application container now? The page will reconnect automatically.')) return;
+    const tag = containerTag === null ? undefined : containerTag.trim();
+    const target = tag === undefined ? lifecycle.targetImage : `${lifecycle.repository}:${tag}`;
+    if (!window.confirm(`Pull ${target} and replace the application container now? The page will reconnect automatically.`)) return;
     try {
-      const response = await runSensitive(() => updateApplication(socket));
+      const response = await runSensitive(() => updateApplication(socket, tag));
       setLifecycle(response.lifecycle);
+      setContainerTag(null);
     } catch (error) {
       setLifecycleError(error.message);
     }
@@ -121,15 +125,36 @@ export default function AdminOverview({ snapshot, socket, runSensitive, onSnapsh
       <CardFrame title="Application container" meta={lifecycleAvailable ? lifecycle?.state : 'controller unavailable'} bodyClassName="space-y-1 p-1 text-sm">
         <div className="surface max-w-4xl space-y-0.5 p-1 text-slate-300">
           <p>{lifecycleAvailable ? updateSummary : 'Container updates are unavailable until the lifecycle service is running.'}</p>
+          {lifecycleAvailable ? (
+            <div className="space-y-0.5 break-all">
+              <p>Repository: {lifecycle.repository}</p>
+              <p>Selected image: {lifecycle.targetImage}</p>
+              <p>Running image: {lifecycle.runningImage}</p>
+              <p>Compose image: {lifecycle.configuredImage}</p>
+              {lifecycle.targetImage !== lifecycle.configuredImage ? <p className="text-amber-300">A manual Compose deployment may restore the Compose image.</p> : null}
+            </div>
+          ) : null}
           {lifecycle?.message ? <p className="text-slate-400">{lifecycle.message}</p> : null}
           {lifecycle?.rollback ? <p>Rollback: {lifecycle.rollback.status}{lifecycle.rollback.error ? ` — ${lifecycle.rollback.error}` : ''}</p> : null}
           {lifecycleError ? <p className="text-red-300">{lifecycleError}</p> : null}
         </div>
+        <label className="flex max-w-md flex-col gap-0.5">
+          <span>Container tag</span>
+          <input
+            className="field-input w-full"
+            value={containerTag ?? lifecycle?.selectedTag ?? ''}
+            placeholder="Enter a tag"
+            maxLength={128}
+            disabled={!lifecycleAvailable || lifecycleBusy}
+            onChange={(event) => setContainerTag(event.target.value)}
+          />
+          <span className="text-xs text-slate-400">Update and restart applies this tag. Update checks use the saved selected image.</span>
+        </label>
         <div className="flex flex-wrap gap-0.5">
           <button type="button" className="button-dark" disabled={!lifecycleAvailable || lifecycleBusy} onClick={checkForUpdate}>
             {lifecycle?.operation === 'check' && lifecycleBusy ? 'Checking for update…' : 'Check for update'}
           </button>
-          <button type="button" className="button-danger" disabled={!lifecycleAvailable || lifecycleBusy} onClick={update}>
+          <button type="button" className="button-danger" disabled={!lifecycleAvailable || lifecycleBusy || (containerTag !== null && !/^[a-zA-Z0-9_][a-zA-Z0-9_.-]{0,127}$/.test(containerTag.trim()))} onClick={update}>
             {lifecycle?.operation === 'update' && lifecycleBusy ? 'Updating application…' : 'Update and restart'}
           </button>
           <button type="button" className="button-danger" disabled={restartRequested || lifecycleBusy} onClick={restart}>

webui/src/components/AppErrorBoundary.jsx

@@ -0,0 +1,51 @@
+import { Component } from 'react';
+
+export default class AppErrorBoundary extends Component {
+  state = { crashed: false, error: null, componentStack: '', copyStatus: '' };
+
+  static getDerivedStateFromError(error) {
+    return { crashed: true, error };
+  }
+
+  componentDidCatch(error, info) {
+    this.setState({ componentStack: info.componentStack || '' });
+  }
+
+  getDetails() {
+    const { error, componentStack } = this.state;
+    return [
+      String(error?.message ?? error ?? 'Unknown error'),
+      'JavaScript stack:',
+      error?.stack || 'Unavailable',
+      'React component stack:',
+      componentStack || 'Unavailable',
+    ].join('\n\n');
+  }
+
+  copyDetails = async () => {
+    try {
+      await navigator.clipboard.writeText(this.getDetails());
+      this.setState({ copyStatus: 'Copied.' });
+    } catch {
+      // Clipboard access may be unavailable on the display device; the text
+      // remains selectable so the error can still be copied manually.
+      this.setState({ copyStatus: 'Copy unavailable. Select the details below to copy manually.' });
+    }
+  };
+
+  render() {
+    if (!this.state.crashed) return this.props.children;
+
+    // Keep recovery independent of providers and shared UI components: one of
+    // those dependencies may be the reason the application failed.
+    return (
+      <main style={{ backgroundColor: '#b2339f', minHeight: '100vh' }}>
+        <h1>Application crashed</h1>
+        <button type="button" onClick={this.copyDetails}>Copy details</button>
+        <button type="button" onClick={() => window.location.reload()}>Reload</button>
+        {this.state.copyStatus ? <p role="status">{this.state.copyStatus}</p> : null}
+        <pre>{this.getDetails()}</pre>
+      </main>
+    );
+  }
+}

webui/src/components/DriveDockAction/driveDockState.js

@@ -11,10 +11,10 @@ export function deriveDriveDockStateFromTelemetry(dockTelemetry) {
   const oiNormalized = oiLabel.toLowerCase();
   const chargingLabel = dockTelemetry?.chargingStateLabel || '';
   const docked = Boolean(dockTelemetry?.homeBase);
-  const charging = docked && chargingLabel.toLowerCase() !== 'not charging' && chargingLabel !== '';
-  const driving = oiNormalized === 'full';
+  const charging = dockTelemetry?.dockingPhase === 'charging';
+  const driving = oiNormalized === 'full' && !dockTelemetry?.movementLocked;
   const dockedNotCharging = docked && !charging;
-  const dockingInProgress = !docked && !charging && oiNormalized === 'passive';
+  const dockingInProgress = !docked && !charging && oiNormalized === 'passive' && dockTelemetry?.dockingPhase !== 'undocking';
   return { driving, docked, charging, dockedNotCharging, dockingInProgress, oiLabel, chargingLabel };
 }
 

webui/src/components/DriveDockAction/index.jsx

@@ -1,13 +1,12 @@
 // Drive Dock Action
 // Purpose: Defines the Drive Dock Action module and the local helpers/components used in this file.
 // Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
-import { useState } from 'react';
+import { useEffect, useState } from 'react';
 import '../MobileControls/mobileControls.css';
 import { useControlActions, useControlSelector } from '../../controls/index.js';
 import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
 import { dockTelemetryEqual, selectDockTelemetry } from '../../context/telemetryViews.js';
 import ControlHint from '../ControlHint/index.jsx';
-import { useManualDockAssist } from '../../features/manualDockAssist/useManualDockAssist.js';
 import { deriveDriveDockStateFromTelemetry } from './driveDockState.js';
 import { triggerTouchHaptic } from '../../lib/touchHaptics.js';
 
@@ -91,14 +90,13 @@ export default function DriveDockAction({
   const isMobile = layout === 'mobile';
   const roverId = useControlSelector((control) => control.state.roverId);
   const actions = useControlActions();
-  const dockAssist = useManualDockAssist();
   const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
   const state = driveDockState ?? deriveDriveDockStateFromTelemetry(dockTelemetry);
   const { driving, docked, charging, dockingInProgress } = state;
-  const [pending, setPending] = useState(null);
+  const pending = dockTelemetry.dockingPhase === 'undocking' ? 'drive' : null;
   const [confirmOpen, setConfirmOpen] = useState(false);
   const [showModal, setShowModal] = useState(false);
-  const manualAssistActive = Boolean(dockAssist.active);
+  const manualAssistActive = Boolean(dockTelemetry.cameraLocked);
 
   const driveDisabled = !roverId || pending !== null;
   const dockDisabled = !roverId || pending !== null;
@@ -124,64 +122,37 @@ export default function DriveDockAction({
   const chargeValue = charging ? 'Charging' : docked ? 'Charging soon...' : '—';
   const chargeTone = charging ? 'good' : docked ? 'warn' : 'bad';
 
-  const handleReturnToDrive = async () => {
-    if (!roverId || pending) return;
-    if (isMobile) triggerTouchHaptic('button');
-    setPending('drive');
-    try {
-      dockAssist.exitAssist();
-      actions.setMode('drive');
-      await actions.runMacro('drive-sequence');
-    } catch (err) {
-      alert(err.message);
-    } finally {
-      setPending(null);
-    }
-  };
-
-  const handleStartDrive = async () => {
+  const handleStartDrive = () => {
     if (!roverId || pending) return;
     if (isMobile) triggerTouchHaptic('button');
     setConfirmOpen(false);
     setShowModal(false);
-    setPending('drive');
-    try {
-      dockAssist.exitAssist();
-      actions.setMode('drive');
-      await actions.runMacro('drive-sequence');
-    } catch (err) {
-      alert(err.message);
-    } finally {
-      setPending(null);
-    }
+    actions.undock();
   };
 
-  const handleConfirmDock = async () => {
+  const handleConfirmDock = () => {
     if (!roverId || pending) return;
     if (isMobile) triggerTouchHaptic('button');
-    setPending('dock');
-    try {
-      dockAssist.enterAssist();
-    } catch (err) {
-      alert(err.message);
-    } finally {
-      setPending(null);
-      setConfirmOpen(false);
-      setShowModal(false);
-    }
+    actions.sendCommand('enterDocking');
+    setConfirmOpen(false);
+    setShowModal(false);
   };
 
   const handleOpenDock = () => {
     if (dockDisabled) return;
     if (isMobile) triggerTouchHaptic('button');
     if (manualAssistActive) {
-      dockAssist.exitAssist();
+      actions.sendCommand('cancelDocking');
       return;
     }
     setShowModal(true);
     setConfirmOpen(true);
   };
 
+  useEffect(() => {
+    if (dockTelemetry.dockingError) alert(dockTelemetry.dockingError);
+  }, [dockTelemetry.dockingError]);
+
   // The drive/dock card appears inside the mobile controls and can be held or
   // tapped repeatedly; attach mobile touch suppression directly to the card so
   // Safari does not select labels or open callouts during those interactions.
@@ -254,7 +225,7 @@ export default function DriveDockAction({
       <button
         type="button"
         disabled={driveDisabled}
-        onClick={handleReturnToDrive}
+        onClick={handleStartDrive}
         onContextMenu={(event) => event.preventDefault()}
         className={`${baseCardClasses} ${filledHeight} ${compactHeight} ${ctaText} ${layoutClass} ${ctaSize} ${amberCta}`}
       >

webui/src/components/DriverVideo/index.jsx

@@ -48,7 +48,7 @@ export default function DriverVideo({ layoutFormat = 'desktop' }) {
             mobileHud={mobileHud}
             labelScale={1}
           />
-          <ManualDockAssistOverlay mobileHud={mobileHud} />
+          <ManualDockAssistOverlay roverId={roverId} mobileHud={mobileHud} />
           <HudChatInput compact={mobileHud} />
           <OvercurrentOverlay compact={mobileHud} />
           <LowBatteryOverlay compact={mobileHud} />

webui/src/components/GamepadMappingSettings/constants.js

@@ -56,7 +56,7 @@ export const ACTIONS = [
   { id: 'allAux', label: 'All cleaning motors', kind: 'button', section: 'Aux buttons' },
   { id: 'mainReverse', label: 'Main reverse toggle', kind: 'button', section: 'Brush toggles' },
   { id: 'sideReverse', label: 'Side reverse toggle', kind: 'button', section: 'Brush toggles' },
-  { id: 'driveMacro', label: 'Drive / undock sequence', kind: 'button', section: 'Mode controls' },
+  { id: 'driveMacro', label: 'Drive / undock', kind: 'button', section: 'Mode controls' },
   { id: 'dockMacro', label: 'Manual docking assist', kind: 'button', section: 'Mode controls' },
   { id: 'headlightToggle', label: 'Headlight toggle', kind: 'button', section: 'Camera' },
   { id: 'laserToggle', label: 'Laser toggle', kind: 'button', section: 'Camera' },

webui/src/components/HomeAssistantActivitiesPanel/index.jsx

@@ -24,8 +24,9 @@ function ActivityTile({ entity, connected, allowed, admin }) {
   // Translucent fills preserve text contrast; an amber border still identifies
   // locked tiles regardless of the admin's chosen color.
   const color = /^#[0-9a-f]{6}$/i.test(entity.color) ? entity.color : '#3b82f6';
-  const Control = controls[entity.type] || ReadOnlyControl;
-  const disabled = !connected || !entity.available || !allowed || (entity.locked && !admin);
+  // Ordinary users see only the value; admins retain the entity's actual input.
+  const Control = entity.readOnly && !admin ? ReadOnlyControl : controls[entity.type] || ReadOnlyControl;
+  const disabled = !connected || !entity.available || !allowed || ((entity.locked || entity.readOnly) && !admin);
   return <div className="flex min-w-0 flex-col gap-0.5 rounded-sm border px-0.5 py-0.5"
     style={{ borderColor: entity.locked ? '#b45309' : `${color}aa`, backgroundColor: `${color}33` }}>
     <div className="-mx-0.5 -mt-0.5 flex min-h-5 min-w-0 items-center gap-0.5 rounded-t px-0.5 py-0.5 text-white" style={{ backgroundColor: `${color}66` }}>

webui/src/components/HudOverlays/ManualDockAssistOverlay/index.jsx

@@ -1,23 +1,24 @@
+import { dockTelemetryEqual, selectDockTelemetry } from '../../../context/telemetryViews.js';
+import { useTelemetrySelector } from '../../../context/TelemetryContext.jsx';
 import React, { useEffect, useRef, useState } from 'react';
-import { useManualDockAssist } from '../../../features/manualDockAssist/useManualDockAssist.js';
 
-function ManualDockAssistOverlay({ mobileHud = false }) {
-  const { visible, statusLabel, statusTone, active, charging } = useManualDockAssist({ manageLifecycle: true });
+function ManualDockAssistOverlay({ roverId, mobileHud = false }) {
+  const telemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
+  const active = telemetry.cameraLocked;
+  const charging = telemetry.dockingPhase === 'charging';
+  const visible = active || telemetry.homeBase;
+  const statusLabel = charging ? 'Docked and charging' : telemetry.homeBase ? 'Docked' : 'Docking assist active';
+  const statusTone = charging ? 'good' : telemetry.homeBase ? 'warn' : 'active';
   const [popupMessage, setPopupMessage] = useState('');
   const timerRef = useRef(null);
-  const prevActiveRef = useRef(active);
+  const [previousActive, setPreviousActive] = useState(active);
 
-  useEffect(() => {
-    const prevActive = prevActiveRef.current;
-    const enabledNow = active && !prevActive;
-    const autoDisabledOnCharge = !active && prevActive && charging;
-    if (enabledNow) {
-      setPopupMessage('Dock assist mode enabled');
-    } else if (autoDisabledOnCharge) {
-      setPopupMessage('Docking successful! Thank you!');
-    }
-    prevActiveRef.current = active;
-  }, [active, charging]);
+  // These messages are presentation only; telemetry owns the operation itself.
+  if (previousActive !== active) {
+    setPreviousActive(active);
+    if (active) setPopupMessage('Dock assist mode enabled');
+    else if (charging) setPopupMessage('Docking successful! Thank you!');
+  }
 
   useEffect(() => {
     if (!popupMessage) return undefined;

webui/src/components/HudOverlays/newgen/CornerPods/BottomRightPod.jsx

@@ -1,3 +1,5 @@
+import { useTelemetrySelector } from '../../../../context/TelemetryContext.jsx';
+import { selectDockTelemetry, dockTelemetryEqual } from '../../../../context/telemetryViews.js';
 // Bottom-right Corner Pod
 // Purpose: Provides a compact circular camera-tilt control using the existing servo command path.
 import { useCallback } from 'react';
@@ -31,7 +33,7 @@ export default function BottomRightPod({ roverId }) {
   const layout = useLayout();
   const [open, setOpen] = usePodVisibility('camera', true);
   const camera = useControlSelector((control) => control.state.camera);
-  const dockAssistActive = useControlSelector((control) => Boolean(control.state.manualDockAssist?.active));
+  const { cameraLocked } = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
   const { setServoAngle } = useControlActions();
   const canControl = useCanControlRover(roverId);
   const config = camera?.config;
@@ -47,7 +49,7 @@ export default function BottomRightPod({ roverId }) {
   const cameraPodOpen = showCameraControls && enabled && open;
 
   const updateFromPointer = useCallback((event) => {
-    if (!canControl || !enabled || dockAssistActive || !(max > min)) return;
+    if (!canControl || !enabled || cameraLocked || !(max > min)) return;
     const bounds = event.currentTarget.getBoundingClientRect();
     const x = ((event.clientX - bounds.left) / bounds.width) * 144;
     const y = ((event.clientY - bounds.top) / bounds.height) * 144;
@@ -64,7 +66,7 @@ export default function BottomRightPod({ roverId }) {
     }
     const nextFraction = arcDegrees / ARC_SWEEP_DEGREES;
     setServoAngle(min + nextFraction * (max - min));
-  }, [canControl, dockAssistActive, enabled, max, min, setServoAngle]);
+  }, [canControl, cameraLocked, enabled, max, min, setServoAngle]);
 
   return (
     <>

webui/src/components/HudOverlays/newgen/DockingHud/index.jsx

@@ -8,7 +8,6 @@ import { useControlActions } from '../../../../controls/index.js';
 import ControlHint from '../../../ControlHint/index.jsx';
 import { useTelemetrySelector } from '../../../../context/TelemetryContext.jsx';
 import { dockTelemetryEqual, selectDockTelemetry } from '../../../../context/telemetryViews.js';
-import { useManualDockAssist } from '../../../../features/manualDockAssist/useManualDockAssist.js';
 import { useSettingsNamespace } from '../../../../settings/index.js';
 import useCanControlRover from '../../../../hooks/useCanControlRover.js';
 import { useLayout } from '../../../../layouts/LayoutContext.jsx';
@@ -253,9 +252,6 @@ export default function DockingHud({ roverId }) {
   const layout = useLayout();
   const actions = useControlActions();
   const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
-  // This replaces ManualDockAssistOverlay as the current HUD's one lifecycle owner. It preserves the
-  // success sounds, camera positioning, speed cap, and automatic exit after charging begins.
-  const dockAssist = useManualDockAssist({ manageLifecycle: true });
   const canControl = useCanControlRover(roverId);
   const batteryState = useSessionSelector((state) => {
     const rover = (state.session?.roster || []).find((entry) => String(entry.id) === String(roverId));
@@ -268,15 +264,14 @@ export default function DockingHud({ roverId }) {
   const [dockExpansionOpen, setDockExpansionOpen] = usePodVisibility('dockAssist', true);
   // The action name is presentation state as well as busy state. Keeping the
   // reason prevents a passive, already-undocked rover from ever saying "Undocking".
-  const [pendingAction, setPendingAction] = useState(null);
-  const [error, setError] = useState('');
   const [showUndockTransition, setShowUndockTransition] = useState(false);
 
   const docked = Boolean(dockTelemetry?.homeBase);
   const oiMode = String(dockTelemetry?.oiModeLabel || '').toLowerCase();
   // The established UI contract treats exactly passive + undocked as the Roomba's
   // autonomous docking attempt. Unknown telemetry must not fabricate that state.
-  const autoDocking = !docked && !dockAssist.active && oiMode === 'passive';
+  const autoDocking = !docked && !dockTelemetry.cameraLocked && oiMode === 'passive';
+  const pendingAction = (dockTelemetry.dockingPhase === 'undocking') ? (docked ? 'undocking' : 'resuming') : null;
   const pending = pendingAction !== null;
   /* Mobile already presents its own touch-oriented driving controls. The docked
      action therefore keeps its plain-language instruction without advertising a
@@ -287,55 +282,22 @@ export default function DockingHud({ roverId }) {
   // The camera arc is the shared circular-pod reference size. Keep the dock expansion flush
   // against the battery shell after enlarging that gauge to the same 8.5-rem footprint.
   const cornerOffsetClass = batteryPodOpen ? 'right-34' : 'right-0';
-  const previousDockedRef = useRef(docked);
+  const [previousDocked, setPreviousDocked] = useState(docked);
   const finishUndockTransition = useCallback(() => setShowUndockTransition(false), []);
 
-  useEffect(() => {
-    const wasDocked = previousDockedRef.current;
-    // Only a real live transition plays the cue. An already-undocked rover must not animate
-    // merely because the user opened or refreshed the page.
-    if (wasDocked && !docked) {
-      setShowUndockTransition(true);
-    } else if (docked) {
-      setShowUndockTransition(false);
-    }
-    previousDockedRef.current = docked;
-  }, [docked]);
+  // Presentation-only transition; mounting with an undocked rover does not animate.
+  if (previousDocked !== docked) {
+    setPreviousDocked(docked);
+    setShowUndockTransition(!docked);
+  }
 
-  const startDriving = async (action) => {
+  const startDriving = () => {
     if (!roverId || pending || !canControl) return;
-    setPendingAction(action);
-    setError('');
-    try {
-      // The established drive sequence is the canonical undock path: it restores the camera,
-      // enters full Open Interface mode, and performs the short physical back-away from the dock.
-      dockAssist.exitAssist();
-      actions.setMode('drive');
-      await actions.runMacro('drive-sequence');
-    } catch (caughtError) {
-      setError(caughtError?.message || 'Unable to start driving. Please try again.');
-    } finally {
-      setPendingAction(null);
-    }
+    actions.undock();
   };
-
-  const startUndocking = () => {
-    startDriving('undocking');
-  };
-  const resumeDriving = () => {
-    startDriving('resuming');
-  };
-
   const startDocking = () => {
     if (!roverId || pending || !canControl) return;
-    setError('');
-    try {
-      // Enter on the first click. The explanation appears as the resulting active state instead
-      // of forcing the user through a modal and a second confirmation action.
-      dockAssist.enterAssist();
-    } catch (caughtError) {
-      setError(caughtError?.message || 'Unable to start dock assist. Please try again.');
-    }
+    actions.sendCommand('enterDocking');
   };
 
   return (
@@ -358,20 +320,20 @@ export default function DockingHud({ roverId }) {
           driveKeyLabel={driveKeyLabel}
           pending={pendingAction === 'undocking'}
           controlsDisabled={!canControl}
-          error={error}
-          onUndock={startUndocking}
+          error={dockTelemetry.dockingError}
+          onUndock={startDriving}
         />
       ) : autoDocking || pendingAction === 'resuming' ? (
         <AutoDockingAction
           driveKeyLabel={driveKeyLabel}
           pending={pendingAction === 'resuming'}
           controlsDisabled={!canControl}
-          error={error}
-          onResumeDriving={resumeDriving}
+          error={dockTelemetry.dockingError}
+          onResumeDriving={startDriving}
         />
       ) : (
         <>
-          {dockAssist.active ? (
+          {dockTelemetry.cameraLocked ? (
             /* Dock assist needs the unobscured camera image. A thin cyan frame communicates the
                 temporary mode without adding another instruction surface over the video. */
             <div className="pointer-events-none absolute inset-0 z-50 ring-8 ring-inset ring-cyan-200/90" aria-hidden="true" />
@@ -379,15 +341,15 @@ export default function DockingHud({ roverId }) {
           {/* Desktop keeps the compact corner entry point. Mobile starts assist from
               its dedicated control column, but once active it still mounts this component
               so the centered camera instruction and cancel action remain available. */}
-          {layout === 'desktop' || dockAssist.active ? (
+          {layout === 'desktop' || dockTelemetry.cameraLocked || dockTelemetry.dockingError ? (
             <DockAssistAction
-              active={dockAssist.active}
+              active={dockTelemetry.cameraLocked}
               pending={pending}
               controlsDisabled={!canControl}
-              error={error}
+              error={dockTelemetry.dockingError}
               dockKeyLabel={dockKeyLabel}
               onDock={startDocking}
-              onCancel={dockAssist.exitAssist}
+              onCancel={() => actions.sendCommand('cancelDocking')}
               cornerOffsetClass={cornerOffsetClass}
               open={dockExpansionOpen}
               onOpenChange={setDockExpansionOpen}

webui/src/components/MobileControls/AuxColumn.jsx

@@ -7,7 +7,6 @@ import './mobileControls.css';
 import { useControlActions, useControlSelector } from '../../controls/index.js';
 import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
 import { dockTelemetryEqual, selectDockTelemetry } from '../../context/telemetryViews.js';
-import { useManualDockAssist } from '../../features/manualDockAssist/useManualDockAssist.js';
 import useCanControlRover from '../../hooks/useCanControlRover.js';
 import HornControl from '../HornControl/index.jsx';
 import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
@@ -33,7 +32,6 @@ function AuxColumnContent({ accessoriesAvailable, onShowAccessories }) {
   const laserState = useControlSelector((control) => control.pipeline?.laserState);
   const pipelineHorn = useControlSelector((control) => control.pipeline?.horn);
   const { setServoAngle, setHeadlight, setLaser, setAuxMotors, startHorn, stopHorn } = useControlActions();
-  const dockAssist = useManualDockAssist();
   const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
   const canControl = useCanControlRover(roverId);
   // Turn ownership is the common mutation boundary for every control in this
@@ -44,7 +42,8 @@ function AuxColumnContent({ accessoriesAvailable, onShowAccessories }) {
   const drivingMode = String(dockTelemetry?.oiModeLabel || '').toLowerCase() === 'full';
   const vacuumDisabled = controlsDisabled
     || docked
-    || (!drivingMode && !dockAssist.active);
+    || dockTelemetry.movementLocked
+    || (!drivingMode && !dockTelemetry.cameraLocked);
   const activeAuxButtonRef = useRef(null);
   const cameraConfig = camera?.config;
   const cameraEnabled = Boolean(roverId && camera?.enabled && cameraConfig);
@@ -60,7 +59,7 @@ function AuxColumnContent({ accessoriesAvailable, onShowAccessories }) {
       : typeof cameraConfig?.homeAngle === 'number'
         ? cameraConfig.homeAngle
         : (cameraMin + cameraMax) / 2;
-  const cameraDisabled = Boolean(controlsDisabled || dockAssist.cameraLocked);
+  const cameraDisabled = Boolean(controlsDisabled || dockTelemetry.cameraLocked);
   /*
     The mobile tilt track shares the same precision flag as desktop tilt. This
     keeps the servo fine-step behavior tied to the selected movement mode rather

webui/src/components/MobileControls/MovementColumn.jsx

@@ -2,19 +2,18 @@
 // Purpose: Assembles the mobile movement column, which is the right column by default.
 // Scope: Owns movement availability while the in-video DockingHud owns every dock/undock action.
 import { FaChargingStation } from 'react-icons/fa';
-import { useControlSelector } from '../../controls/index.js';
+import { useControlActions, useControlSelector } from '../../controls/index.js';
 import { useSessionSelector } from '../../context/SessionContext.jsx';
 import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
 import { dockTelemetryEqual, selectDockTelemetry } from '../../context/telemetryViews.js';
-import { useManualDockAssist } from '../../features/manualDockAssist/useManualDockAssist.js';
 import useCanControlRover from '../../hooks/useCanControlRover.js';
 import { triggerTouchHaptic } from '../../lib/touchHaptics.js';
 import ControlPadPanel from './ControlPadPanel.jsx';
 
 function MovementColumnContent({ layout }) {
   const roverId = useControlSelector((control) => control.state.roverId);
+  const actions = useControlActions();
   const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
-  const dockAssist = useManualDockAssist();
   const canControl = useCanControlRover(roverId);
   const batteryState = useSessionSelector((state) => {
     const rover = (state.session?.roster || []).find((entry) => String(entry.id) === String(roverId));
@@ -25,20 +24,11 @@ function MovementColumnContent({ layout }) {
   const docked = Boolean(dockTelemetry?.homeBase);
   const drivingMode = String(dockTelemetry?.oiModeLabel || '').toLowerCase() === 'full';
 
-  /*
-    DockingHud is now the only place where a user starts driving or enters docking
-    assist. This column only decides whether touch movement is safe. Requiring both
-    an undocked rover and full OI mode keeps the pad inert throughout the undock
-    macro, including the interval where dock contact and OI mode change separately.
-
-    Manual assist is the deliberate exception to the normal OI-mode requirement:
-    its entire purpose is to let the user drive onto the dock under the assist speed
-    cap. Actual dock contact still disables movement immediately.
-  */
   const movementDisabled = !roverId
     || !canControl
     || docked
-    || (!drivingMode && !dockAssist.active);
+    || dockTelemetry.movementLocked
+    || (!drivingMode && !dockTelemetry.cameraLocked);
   // Both controls share the same hardware-availability boundary. Dock assist
   // changes their actions, not who is allowed to command the rover.
   const dockActionDisabled = movementDisabled;
@@ -49,10 +39,10 @@ function MovementColumnContent({ layout }) {
     // Mobile deliberately enters assist on the first tap. The centered video HUD
     // provides the resulting instruction, so a confirmation modal would only add
     // friction between intent and the camera-guided docking task.
-    if (dockAssist.active) {
-      dockAssist.exitAssist();
+    if (dockTelemetry.cameraLocked) {
+      actions.sendCommand('cancelDocking');
     } else {
-      dockAssist.enterAssist();
+      actions.sendCommand('enterDocking');
     }
   };
 
@@ -63,7 +53,7 @@ function MovementColumnContent({ layout }) {
         disabled={dockActionDisabled}
         onClick={handleDockAction}
         className={`mobile-touch-control flex min-h-18 shrink-0 items-center justify-center gap-1.5 rounded-xl border-2 px-2 text-base font-semibold shadow-md transition disabled:cursor-not-allowed disabled:opacity-50 ${
-          dockAssist.active
+          dockTelemetry.cameraLocked
             ? 'border-cyan-300/70 bg-cyan-900 text-cyan-50'
             : batteryUrgent
               ? 'border-red-300/80 bg-red-950 text-red-50'
@@ -75,7 +65,7 @@ function MovementColumnContent({ layout }) {
         <FaChargingStation className="text-lg" aria-hidden="true" />
         {/* The mobile column remains the only mobile entry point into dock assist. Battery
             severity makes that existing action more obvious without adding another control. */}
-        <span>{dockAssist.active ? 'Exit dock assist' : batteryUrgent ? 'Dock now' : batteryLow ? 'Dock and charge soon' : 'Dock and charge'}</span>
+        <span>{dockTelemetry.cameraLocked ? 'Exit dock assist' : batteryUrgent ? 'Dock now' : batteryLow ? 'Dock and charge soon' : 'Dock and charge'}</span>
       </button>
       {/* Keeping the pad mounted prevents the mobile columns from changing size while
           the centered video HUD explains and performs dock-related transitions. */}

webui/src/components/PtzLiveVideo/index.jsx

@@ -5,6 +5,7 @@ import { useEffect, useRef, useState } from 'react';
 import { useVideoRequests } from '../../hooks/useVideoRequests.js';
 import { startWhepPlayback } from '../../lib/whepPlayback.js';
 import useWhepRestart from '../../hooks/useWhepRestart.js';
+import useServerUrl from '../../hooks/useServerUrl.js';
 
 export const PTZ_CAMERA_ID = 'ptz-camera';
 
@@ -23,6 +24,7 @@ export default function PtzLiveVideo({
   statusClassName = 'pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]',
   fallback = null,
 }) {
+  const serverUrl = useServerUrl();
   const videoRef = useRef(null);
   const playTimerRef = useRef(null);
   const [status, setStatus] = useState('idle');
@@ -44,7 +46,7 @@ export default function PtzLiveVideo({
     if (!enabled || shouldUseFallback || !source?.url || !videoRef.current) return undefined;
     const video = videoRef.current;
     const stop = startWhepPlayback({
-      url: source.url,
+      url: serverUrl(source.url),
       token: source.token,
       video: videoRef.current,
       startMuted,
@@ -76,7 +78,7 @@ export default function PtzLiveVideo({
       video.removeEventListener('error', handleError);
       stop();
     };
-  }, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted]);
+  }, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted, serverUrl]);
 
   useEffect(() => {
     const video = videoRef.current;

webui/src/components/ReplaySourcesPanel/index.jsx

@@ -46,7 +46,7 @@ export default function ReplaySourcesPanel({
   const assignmentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
   const roster = useSessionSelector((state) => state.session?.roster ?? []);
   const replayState = useSessionSelector((state) => state.session?.replay || null);
-  const replayStatus = useSessionSelector((state) => state.replayStatus);
+  const replayStatus = replayState?.status;
   const socket = useSocket();
   const { triggerReplay } = useSessionActions();
   const sources = useMemo(() => normalizeSources(replaySources || []), [replaySources]);
@@ -161,48 +161,25 @@ export default function ReplaySourcesPanel({
     };
   }, [sources]);
 
-  const hasReplayCooldown = Boolean(replayState?.lastTriggeredAt && replayState?.cooldownMs);
-  const replayCooldownEndsAt = hasReplayCooldown
-    ? replayState.lastTriggeredAt + replayState.cooldownMs
+  const cooldownNow = useSharedClock(1000, Boolean(replayState?.lastTriggeredAt));
+  const remainingMs = replayState?.lastTriggeredAt
+    ? Math.max(0, replayState.lastTriggeredAt + replayState.cooldownMs - cooldownNow)
     : 0;
-  const cooldownNow = useSharedClock(1000, hasReplayCooldown);
-  const remainingMs = useMemo(() => {
-    if (!hasReplayCooldown) return 0;
-    /*
-      The button only shows whole seconds, so a shared one-second clock gives the
-      same useful information without each mounted replay panel owning a 250ms
-      interval. The exact server cooldown still decides whether the action is
-      accepted; this value is only the local disabled-state/display estimate.
-    */
-    const next = replayCooldownEndsAt - cooldownNow;
-    return Math.max(0, next);
-  }, [cooldownNow, hasReplayCooldown, replayCooldownEndsAt]);
-
-  const replayDisabled = busy || mode === 'lockdown' || remainingMs > 0 || !selected.length;
+  const atJobLimit = replayState?.atJobLimit === true;
+  const replaySourceKeys = replayState?.activeSourceKeys;
+  const activeSourceKeys = useMemo(() => new Set(replaySourceKeys || []), [replaySourceKeys]);
+  const replayDisabled = busy || mode === 'lockdown' || remainingMs > 0 || atJobLimit || !selected.length;
   const selectedSet = useMemo(() => {
     // Checkbox rendering asks the same membership question for every source.
     // A Set avoids repeated linear scans and, more importantly, gives memoized
     // child lists a stable value while the selected keys have not changed.
     return new Set(selected);
   }, [selected]);
-  const replayStatusText = useMemo(() => {
-    if (!replayStatus?.status) return null;
-    const titleText = replayStatus.title ? `: ${replayStatus.title}` : '';
-    switch (replayStatus.status) {
-      case 'accepted':
-        return `Replay accepted${titleText}`;
-      case 'building':
-        return `Replay building${titleText}`;
-      case 'uploading':
-        return `Replay uploading${titleText}`;
-      case 'ready':
-        return `Replay ready${titleText}`;
-      case 'failed':
-        return replayStatus.message || `Replay failed${titleText}`;
-      default:
-        return replayStatus.message || `Replay ${replayStatus.status}${titleText}`;
-    }
-  }, [replayStatus]);
+  const replayStatusText = replayStatus?.status
+    ? replayStatus.status === 'failed' && replayStatus.message
+      ? replayStatus.message
+      : `Replay ${replayStatus.status}${replayStatus.title ? `: ${replayStatus.title}` : ''}`
+    : null;
   const toggleKey = useCallback((key) => {
     setSelected((prev) => {
       return prev.includes(key) ? prev.filter((value) => value !== key) : [...prev, key];
@@ -250,13 +227,13 @@ export default function ReplaySourcesPanel({
             current desktop sidebar. They only collapse when the card becomes
             narrower than the supported new-drive desktop column. */}
         <div className={`grid grid-cols-2 gap-0.5 @max-[15rem]:grid-cols-1 ${listWrapClass}`}>
-          <GroupList title="Rovers" items={grouped.rovers} selectedSet={selectedSet} onToggle={toggleKey} />
-          <GroupList title="Room Cams" items={grouped.rooms} selectedSet={selectedSet} onToggle={toggleKey} />
+          <GroupList title="Rovers" items={grouped.rovers} selectedSet={selectedSet} activeSourceKeys={activeSourceKeys} onToggle={toggleKey} />
+          <GroupList title="Room Cams" items={grouped.rooms} selectedSet={selectedSet} activeSourceKeys={activeSourceKeys} onToggle={toggleKey} />
         </div>
         <div className="space-y-0.5">
           {error ? <div className="text-xs text-amber-400">{error}</div> : null}
           {replayStatusText ? (
-            <div className={`text-xs ${replayStatus?.status === 'failed' ? 'text-amber-400' : 'text-emerald-300'}`}>
+            <div className={`text-xs ${replayStatus.status === 'failed' ? 'text-amber-400' : 'text-emerald-300'}`}>
               {replayStatusText}
             </div>
           ) : null}
@@ -279,11 +256,6 @@ export default function ReplaySourcesPanel({
                 saveSettings((current) => ({ ...(current || {}), [titleSettingKey]: next }));
               }}
               onKeyDown={(event) => {
-                // Enter is the keyboard equivalent of clicking Replay. Ignore
-                // composition events so confirming an IME candidate cannot
-                // accidentally submit a replay before the title is complete.
-                // handleReplay remains the single authority for cooldown,
-                // lockdown, busy, and empty-source checks.
                 if (event.key !== 'Enter' || event.nativeEvent?.isComposing) return;
                 event.preventDefault();
                 handleReplay();
@@ -314,7 +286,7 @@ export default function ReplaySourcesPanel({
               onClick={handleReplay}
               disabled={replayDisabled}
             >
-              {remainingMs > 0 ? `Replay (${Math.ceil(remainingMs / 1000)}s)` : busy ? 'Replay…' : 'Replay'}
+              {atJobLimit ? 'Replay (limit reached)' : remainingMs > 0 ? `Replay (${Math.ceil(remainingMs / 1000)}s)` : busy ? 'Replay…' : 'Replay'}
             </button>
           </div>
         </div>
@@ -323,14 +295,14 @@ export default function ReplaySourcesPanel({
   );
 }
 
-const GroupList = React.memo(function GroupList({ title, items, selectedSet, onToggle }) {
+const GroupList = React.memo(function GroupList({ title, items, selectedSet, activeSourceKeys, onToggle }) {
   if (!items.length) return null;
   return (
     <div className="space-y-0.5">
       <div className="panel-muted text-xs">{title}</div>
       <div className="space-y-0.5">
         {items.map((item) => (
-          <label key={item.key} className="surface flex items-center gap-0.5 text-xs">
+          <label key={item.key} className={`surface flex items-center gap-0.5 text-xs ${activeSourceKeys.has(item.key) ? 'animate-pulse' : ''}`}>
             <input
               type="checkbox"
               checked={selectedSet.has(item.key)}

webui/src/components/RoomAudioPlayer/index.jsx

@@ -0,0 +1,71 @@
+import { useCallback, useEffect, useRef } from 'react';
+import { useSessionSelector } from '../../context/SessionContext.jsx';
+import { useSocket } from '../../context/SocketContext.jsx';
+import { useSettingsNamespace } from '../../settings/index.js';
+import { AUDIO_SETTINGS_DEFAULTS } from '../../settings/namespaces.js';
+import { RESTART_DELAY_MS, startWhepPlayback } from '../../lib/whepPlayback.js';
+import useWhepRestart from '../../hooks/useWhepRestart.js';
+import useServerUrl from '../../hooks/useServerUrl.js';
+
+function StreamPlayer({ id, volume }) {
+  const socket = useSocket();
+  const serverUrl = useServerUrl();
+  const audio = useRef(null);
+  const { restartToken, scheduleRestart } = useWhepRestart();
+  useEffect(() => {
+    let active = true;
+    let stop;
+    let token;
+    socket.timeout(5000).emit('video:request', { type: 'roomAudio', id }, (error, response = {}) => {
+      if (!active) {
+        if (response.token && socket.connected) socket.emit('roomAudio:release', { token: response.token });
+        return;
+      }
+      token = response.token;
+      if (error || response.error || !response.url) { scheduleRestart(); return; }
+      stop = startWhepPlayback({
+        ...response, url: serverUrl(response.url), video: audio.current, audioOnly: true,
+        onStatus: () => {}, onError: () => {}, scheduleRestart,
+      });
+    });
+    return () => {
+      active = false; stop?.();
+      if (token && socket.connected) socket.emit('roomAudio:release', { token });
+    };
+  }, [id, socket, restartToken, scheduleRestart, serverUrl]);
+  useEffect(() => { if (audio.current) audio.current.volume = Math.max(0, Math.min(1, volume)); }, [volume]);
+  return <audio ref={audio} autoPlay />;
+}
+
+export default function RoomAudioPlayer() {
+  const container = useRef(null);
+  const streams = useSessionSelector((state) => state.session?.roomAudioStreams);
+  const connected = useSessionSelector((state) => state.connected);
+  const { value } = useSettingsNamespace('audio', AUDIO_SETTINGS_DEFAULTS);
+  const volume = (value?.masterVolume ?? 1) * (value?.roomVolume ?? 1);
+  const play = useCallback(() => {
+    const root = container.current;
+    if (!root) return;
+    const elements = [...root.querySelectorAll('audio')].filter((audio) => audio.srcObject);
+    if (!elements.length) return;
+    // Call play during the gesture; blocked playback retries on the interval.
+    elements.forEach((audio) => { audio.play().catch(() => {}); });
+  }, []);
+  useEffect(() => {
+    if (!connected) return undefined;
+    const timer = setInterval(play, RESTART_DELAY_MS);
+    window.addEventListener('pointerdown', play);
+    window.addEventListener('keydown', play);
+    return () => {
+      clearInterval(timer);
+      window.removeEventListener('pointerdown', play);
+      window.removeEventListener('keydown', play);
+    };
+  }, [connected, play]);
+  if (!connected) return null;
+  return (
+    <div ref={container} className="contents">
+      {(streams || []).map((stream) => <StreamPlayer key={stream.id} id={stream.id} volume={volume} />)}
+    </div>
+  );
+}

webui/src/components/RoverMediaPlayer/index.jsx

@@ -1,6 +1,7 @@
 import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
 import { startWhepPlayback } from '../../lib/whepPlayback.js';
 import useWhepRestart from '../../hooks/useWhepRestart.js';
+import useServerUrl from '../../hooks/useServerUrl.js';
 import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
 import {
   mainBrushAudioTelemetryEqual,
@@ -130,6 +131,7 @@ export default function RoverMediaPlayer({
   pauseVideoWhenHidden = null,
 }) {
   const assignedRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
+  const serverUrl = useServerUrl();
   const effectiveRoverId = roverId ?? assignedRoverId;
   const mode = useSessionSelector((state) => state.session?.mode || null);
   const configuredPauseHiddenRoverVideo = useSessionSelector(
@@ -415,7 +417,7 @@ export default function RoverMediaPlayer({
     }
     const resetMuteId = setTimeout(() => setMuted(true), 0);
     const stop = startWhepPlayback({
-      url: resolvedSessionInfo.url,
+      url: serverUrl(resolvedSessionInfo.url),
       token: resolvedSessionInfo.token,
       video: videoRef.current,
       receiveAudio: !hasDedicatedAudio,
@@ -446,6 +448,7 @@ export default function RoverMediaPlayer({
     hasDedicatedAudio,
     logAudio,
     effectiveRoverId,
+    serverUrl,
   ]);
 
   useEffect(() => {
@@ -510,7 +513,7 @@ export default function RoverMediaPlayer({
       return undefined;
     }
     return startWhepPlayback({
-      url: resolvedAudioSessionInfo.url,
+      url: serverUrl(resolvedAudioSessionInfo.url),
       token: resolvedAudioSessionInfo.token,
       video: audioRef.current,
       audioOnly: true,
@@ -533,6 +536,7 @@ export default function RoverMediaPlayer({
     audioRestartToken,
     scheduleAudioRestart,
     logAudio,
+    serverUrl,
   ]);
 
   useEffect(() => {

webui/src/components/SettingsPanel/index.jsx

@@ -159,6 +159,7 @@ export default function SettingsPanel() {
   const masterVolume = Number.isFinite(audioSettings?.masterVolume) ? audioSettings.masterVolume : AUDIO_SETTINGS_DEFAULTS.masterVolume;
   const alertVolume = Number.isFinite(audioSettings?.alertVolume) ? audioSettings.alertVolume : AUDIO_SETTINGS_DEFAULTS.alertVolume;
   const roverVolume = Number.isFinite(audioSettings?.roverVolume) ? audioSettings.roverVolume : AUDIO_SETTINGS_DEFAULTS.roverVolume;
+  const roomVolume = Number.isFinite(audioSettings?.roomVolume) ? audioSettings.roomVolume : AUDIO_SETTINGS_DEFAULTS.roomVolume;
   const mainBrushDuckEnabled =
     typeof audioSettings?.mainBrushDuckEnabled === 'boolean'
       ? audioSettings.mainBrushDuckEnabled
@@ -411,7 +412,7 @@ export default function SettingsPanel() {
                 <span className="font-semibold text-white">Swap control columns (put joystick on the left)</span>
               </SettingRow>
             </CardFrame>
-            <CardFrame title="Macros" bodyClassName="space-y-1 p-1 text-sm">
+            <CardFrame title="Driving" bodyClassName="space-y-1 p-1 text-sm">
               <SettingRow inline>
                 <input
                   type="checkbox"
@@ -419,7 +420,7 @@ export default function SettingsPanel() {
                   checked={driveMacroBackoffEnabled}
                   onChange={handleDriveMacroBackoffEnabled}
                 />
-                <span className="font-semibold text-white">Enable backward bump in drive macro</span>
+                <span className="font-semibold text-white">Back away from the dock when undocking</span>
               </SettingRow>
             </CardFrame>
             <CardFrame title="Audio" className="col-span-full" bodyClassName="space-y-1 p-1 text-sm">
@@ -440,6 +441,11 @@ export default function SettingsPanel() {
                 value={roverVolume}
                 onChange={handleAudioRange('roverVolume')}
               />
+              <RangeSetting
+                label="Room audio"
+                value={roomVolume}
+                onChange={handleAudioRange('roomVolume')}
+              />
               <SettingRow>
                 <span className="font-semibold text-white">Main brush ducking</span>
                 <input

webui/src/components/vip/VipAudioUploadCard/VipAudioUploadCardContent.jsx

@@ -1,19 +1,16 @@
 // Vip Audio Upload Card Content
 // Purpose: Defines the Vip Audio Upload Card Content module and the local helpers/components used in this file.
 // Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
-import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
+import { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react';
 import { fieldClass } from '../constants.js';
 import { useControlSelector } from '../../../controls/index.js';
 import ControlHint from '../../ControlHint/index.jsx';
 import { useSettingsNamespace } from '../../../settings/index.js';
-import { MAX_UPLOAD_BYTES, TARGET_SAMPLE_RATE, RTC_CONFIG } from './constants.js';
-import { bytesToBase64, buildAuthHeader } from './base64.js';
-import {
-  waitForIceGatheringComplete,
-  waitForPeerConnected,
-  configureSenderForLowLatency,
-  waitForOutboundAudioFlow,
-} from './whipTransport.js';
+import { MAX_UPLOAD_BYTES, TARGET_SAMPLE_RATE } from './constants.js';
+import { bytesToBase64 } from './base64.js';
+import { waitForOutboundAudioFlow } from './whipTransport.js';
+import MediaMTXWebRTCPublisher from '../../../../.cache/mediamtx/publisher.js';
+import useServerUrl from '../../../hooks/useServerUrl.js';
 import { mergeFloatChunks, encodeWavMono16 } from './audioCodec.js';
 import StatusIndicator from './StatusIndicator.jsx';
 import KeyPill from './KeyPill.jsx';
@@ -28,6 +25,7 @@ export default function VipAudioUploadCard({
   readyMicWhip,
   stopMicWhip,
 }) {
+  const serverUrl = useServerUrl();
   const pttActive = useControlSelector((control) => Boolean(control.state.mic?.pttActive));
   const { value: vipAudio, save: saveVipAudio } = useSettingsNamespace('vipAudio', {
     openMicEnabled: false,
@@ -47,7 +45,8 @@ export default function VipAudioUploadCard({
 
   const streamRef = useRef(null);
   const audioTrackRef = useRef(null);
-  const whipPcRef = useRef(null);
+  const micAttemptRef = useRef(null);
+  const micEnabledRef = useRef(false);
   const micActiveRef = useRef(false);
   const activeRoverRef = useRef('');
 
@@ -146,19 +145,12 @@ export default function VipAudioUploadCard({
       const target = String(targetRoverId || activeRoverRef.current || '').trim();
       micActiveRef.current = false;
       setMicState('idle');
-      if (whipPcRef.current) {
-        try {
-          whipPcRef.current.getSenders().forEach((sender) => sender.track?.stop());
-        } catch {
-          // noop
-        }
-        try {
-          whipPcRef.current.close();
-        } catch {
-          // noop
-        }
-      }
-      whipPcRef.current = null;
+      const publisher = micAttemptRef.current?.publisher;
+      micAttemptRef.current = null;
+      activeRoverRef.current = '';
+      publisher?.close();
+      // The upstream client stops transport retries; release its HTTP session separately.
+      if (publisher?.sessionUrl) fetch(publisher.sessionUrl, { method: 'DELETE', keepalive: true }).catch(() => {});
       if (streamRef.current) {
         try {
           streamRef.current.getTracks().forEach((track) => track.stop());
@@ -175,14 +167,15 @@ export default function VipAudioUploadCard({
           // noop
         }
       }
-      activeRoverRef.current = '';
     },
     [stopMicWhip],
   );
 
   const startWhipMic = useCallback(
-    async (target) => {
+    async (target, attempt) => {
+      const isCurrent = () => micAttemptRef.current === attempt && activeRoverRef.current === target;
       const startPayload = await startMicWhip?.(target);
+      if (!isCurrent()) return;
       const whipUrl = String(startPayload?.whipUrl || '').trim();
       const token = String(startPayload?.token || '').trim();
       if (!whipUrl || !token) {
@@ -198,12 +191,16 @@ export default function VipAudioUploadCard({
           autoGainControl: false,
         },
       });
+      if (!isCurrent()) {
+        stream.getTracks().forEach((track) => track.stop());
+        return;
+      }
       streamRef.current = stream;
 
       const track = stream.getAudioTracks()?.[0];
       audioTrackRef.current = track || null;
       if (track) {
-        track.enabled = Boolean(openMicEnabled || pttActive);
+        track.enabled = micEnabledRef.current;
       }
       if (track?.applyConstraints) {
         try {
@@ -219,47 +216,42 @@ export default function VipAudioUploadCard({
         }
       }
 
-      const pc = new RTCPeerConnection(RTC_CONFIG);
-      whipPcRef.current = pc;
-      pc.onconnectionstatechange = () => {
-        if (!micActiveRef.current) return;
-        const state = pc.connectionState;
-        if (state === 'connected') {
-          setMicState('live');
-          return;
-        }
-        if (state === 'failed' || state === 'disconnected' || state === 'closed') {
-          setMicState('error');
-          setMessage(`WHIP transport ${state}.`);
-        }
-      };
-      stream.getAudioTracks().forEach((audioTrack) => {
-        const sender = pc.addTrack(audioTrack, stream);
-        configureSenderForLowLatency(sender);
-      });
-
-      const offer = await pc.createOffer({ offerToReceiveAudio: false, offerToReceiveVideo: false });
-      await pc.setLocalDescription(offer);
-      await waitForIceGatheringComplete(pc, 1800);
-
-      const response = await fetch(whipUrl, {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/sdp',
-          ...buildAuthHeader(token),
+      if (!isCurrent()) return;
+      attempt.publisher = new MediaMTXWebRTCPublisher({
+        url: serverUrl(whipUrl),
+        user: token,
+        pass: token,
+        stream,
+        audioCodec: 'opus',
+        audioBitrate: 64,
+        audioVoice: true,
+        onConnected: async () => {
+          if (!isCurrent()) return;
+          const pc = attempt.publisher.pc;
+          try {
+            // Report ready only after audio is flowing, including on transport reconnection.
+            await waitForOutboundAudioFlow(pc);
+            if (!isCurrent() || attempt.publisher.pc !== pc) return;
+            await readyMicWhip?.(target);
+            if (!isCurrent() || attempt.publisher.pc !== pc) return;
+            setMicState('live');
+            setMessage('');
+          } catch (error) {
+            if (!isCurrent() || attempt.publisher.pc !== pc) return;
+            await stopMicCapture(target);
+            if (micAttemptRef.current) return;
+            setMicState('error');
+            setMessage(error.message || 'Failed to start mic forwarding.');
+          }
+        },
+        onError: (message) => {
+          if (!isCurrent()) return;
+          setMicState('starting');
+          setMessage(message);
         },
-        body: pc.localDescription?.sdp || offer.sdp,
       });
-      if (!response.ok) {
-        throw new Error(`WHIP request failed: ${response.status}`);
-      }
-      const answerSdp = await response.text();
-      await pc.setRemoteDescription({ type: 'answer', sdp: answerSdp });
-      await waitForPeerConnected(pc, 10000);
-      await waitForOutboundAudioFlow(pc, 6000);
-      await readyMicWhip?.(target);
     },
-    [openMicEnabled, pttActive, readyMicWhip, startMicWhip],
+    [readyMicWhip, startMicWhip, stopMicCapture, serverUrl],
   );
 
   const teardownClipPipeline = useCallback(async () => {
@@ -410,8 +402,6 @@ export default function VipAudioUploadCard({
   }, [ensureClipPipeline, roverId]);
 
   useEffect(() => {
-    let cancelled = false;
-
     async function syncLiveMic() {
       if (clipMode) {
         await stopMicCapture(activeRoverRef.current || roverId);
@@ -425,34 +415,31 @@ export default function VipAudioUploadCard({
       if (micActiveRef.current && activeRoverRef.current === roverId) return;
       if (!openMicEnabled && !pttActive) return;
 
+      const attempt = {};
+      micAttemptRef.current = attempt;
       try {
         setMicState('starting');
         setMessage('');
         micActiveRef.current = true;
         activeRoverRef.current = roverId;
-        await startWhipMic(roverId);
-        if (!cancelled) {
-          setMicState('live');
-        }
+        await startWhipMic(roverId, attempt);
       } catch (err) {
-        if (!cancelled) {
+        if (micAttemptRef.current === attempt) {
+          await stopMicCapture(roverId);
+          if (micAttemptRef.current) return;
           setMicState('error');
           setMessage(err?.message || 'Failed to start mic forwarding.');
         }
-        await stopMicCapture(roverId);
       }
     }
 
     syncLiveMic();
-    return () => {
-      cancelled = true;
-    };
   }, [clipMode, openMicEnabled, pttActive, roverId, startWhipMic, stopMicCapture]);
 
-  useEffect(() => {
+  useLayoutEffect(() => {
+    micEnabledRef.current = Boolean(!clipMode && (openMicEnabled || pttActive));
     const track = audioTrackRef.current;
-    if (!track) return;
-    track.enabled = Boolean(!clipMode && (openMicEnabled || pttActive));
+    if (track) track.enabled = micEnabledRef.current;
   }, [clipMode, openMicEnabled, pttActive]);
 
   useEffect(() => {

webui/src/components/vip/VipAudioUploadCard/base64.js

@@ -10,14 +10,3 @@ export function bytesToBase64(bytes) {
   }
   return btoa(binary);
 }
-
-function encodeBase64(value) {
-  if (typeof btoa === 'function') return btoa(value);
-  return '';
-}
-
-export function buildAuthHeader(token) {
-  if (!token) return {};
-  const encoded = encodeBase64(`${token}:${token}`);
-  return encoded ? { Authorization: `Basic ${encoded}` } : {};
-}

webui/src/components/vip/VipAudioUploadCard/constants.js

@@ -3,8 +3,3 @@
 // Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
 export const MAX_UPLOAD_BYTES = 8 * 1024 * 1024;
 export const TARGET_SAMPLE_RATE = 16000;
-export const RTC_CONFIG = {
-  iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
-  bundlePolicy: 'max-bundle',
-  rtcpMuxPolicy: 'require',
-};

webui/src/components/vip/VipAudioUploadCard/whipTransport.js

@@ -1,91 +1,6 @@
 // whip Transport
 // Purpose: Defines the whip Transport module and the local helpers/components used in this file.
 // Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
-export function waitForIceGatheringComplete(pc, timeoutMs = 1500) {
-  return new Promise((resolve) => {
-    if (!pc || pc.iceGatheringState === 'complete') {
-      resolve();
-      return;
-    }
-    const timer = setTimeout(() => {
-      pc.removeEventListener('icegatheringstatechange', onChange);
-      resolve();
-    }, timeoutMs);
-    function onChange() {
-      if (pc.iceGatheringState === 'complete') {
-        clearTimeout(timer);
-        pc.removeEventListener('icegatheringstatechange', onChange);
-        resolve();
-      }
-    }
-    pc.addEventListener('icegatheringstatechange', onChange);
-  });
-}
-
-function isPeerTransportReady(pc) {
-  if (!pc) return false;
-  const conn = pc.connectionState;
-  const ice = pc.iceConnectionState;
-  if (conn === 'connected') return true;
-  if (ice === 'connected' || ice === 'completed') return true;
-  return false;
-}
-
-export function waitForPeerConnected(pc, timeoutMs = 10000) {
-  return new Promise((resolve, reject) => {
-    if (!pc) {
-      reject(new Error('Peer connection missing'));
-      return;
-    }
-    if (isPeerTransportReady(pc)) {
-      resolve();
-      return;
-    }
-    const timer = setTimeout(() => {
-      cleanup();
-      reject(new Error('Peer connection timeout'));
-    }, timeoutMs);
-    const onState = () => {
-      if (isPeerTransportReady(pc)) {
-        cleanup();
-        resolve();
-      } else if (
-        pc.connectionState === 'failed' ||
-        pc.connectionState === 'closed' ||
-        pc.iceConnectionState === 'failed'
-      ) {
-        cleanup();
-        reject(new Error(`Peer connection ${pc.connectionState || pc.iceConnectionState}`));
-      }
-    };
-    function cleanup() {
-      clearTimeout(timer);
-      pc.removeEventListener('connectionstatechange', onState);
-      pc.removeEventListener('iceconnectionstatechange', onState);
-    }
-    pc.addEventListener('connectionstatechange', onState);
-    pc.addEventListener('iceconnectionstatechange', onState);
-  });
-}
-
-export async function configureSenderForLowLatency(sender) {
-  if (!sender?.getParameters || !sender?.setParameters) return;
-  const params = sender.getParameters() || {};
-  const first = (params.encodings && params.encodings[0]) || {};
-  params.encodings = [
-    {
-      ...first,
-      maxBitrate: 64000,
-      dtx: 'disabled',
-    },
-  ];
-  try {
-    await sender.setParameters(params);
-  } catch {
-    // Browser support varies; keep defaults if rejected.
-  }
-}
-
 export function waitForOutboundAudioFlow(pc, timeoutMs = 6000) {
   return new Promise((resolve, reject) => {
     if (!pc) {

webui/src/components/vip/VipRoverLearningCard.jsx

@@ -133,7 +133,7 @@ function LearningCard({ roverId }) {
       <div className="flex flex-wrap items-center justify-center gap-1">
         <select aria-label="Rover model" className={fieldClass} value={chosen?.id || ''} onChange={(event) => setSelectedId(event.target.value)} disabled={working || !models.length}>
           {!models.length && <option value="">No published models yet</option>}
-          {models.map((model) => <option key={model.id} value={model.id}>{favorites.includes(model.id) ? '★ ' : ''}{model.name}{model.experimental ? ' (experimental)' : ''}{model.compatible ? '' : model.formatVersion !== 3 ? ' (older model format)' : ' (incompatible)'}</option>)}
+          {models.map((model) => <option key={model.id} value={model.id}>{favorites.includes(model.id) ? '★ ' : ''}{model.name}{model.experimental ? ' (experimental)' : ''}{model.compatible ? '' : model.formatVersion !== 5 ? ' (older model format)' : ' (incompatible)'}</option>)}
         </select>
         <button type="button" className="button-dark text-sm disabled:opacity-50" aria-pressed={Boolean(chosen && favorites.includes(chosen.id))} disabled={!chosen} onClick={favorite}>
           {chosen && favorites.includes(chosen.id) ? '★ Favorite' : '☆ Favorite'}
@@ -220,6 +220,10 @@ function LearningCard({ roverId }) {
             <Row label="Latest update status">{training?.publication?.reason || 'Collecting data'}</Row>
             <Row label="Held-out evaluation">{training?.evaluation?.status || 'Waiting for held-out data'}</Row>
             <Row label="Held-out precision / recall">{training?.evaluation?.precision == null ? 'Unavailable' : `${training.evaluation.precision.toFixed(3)} / ${training.evaluation.recall.toFixed(3)}`}</Row>
+            <Row label="Evaluated training step">{display(training?.evaluatedSteps)}</Row>
+            <Row label="Mean / peak score on human commands">{training?.evaluation?.scoreOnHumanCommands?.mean == null ? 'Unavailable' : `${training.evaluation.scoreOnHumanCommands.mean.toFixed(3)} / ${training.evaluation.scoreOnHumanCommands.max.toFixed(3)}`}</Row>
+            <Row label="Training sample recall before / after">{training?.diagnostics?.trainingAudit?.after?.recall == null ? 'Unavailable' : `${training.diagnostics.trainingAudit.before.recall?.toFixed(3) ?? 'Unavailable'} / ${training.diagnostics.trainingAudit.after.recall.toFixed(3)}`}</Row>
+            <Row label="Memorization probe recall / steps">{training?.diagnostics?.trainingAudit?.learningProbe?.evaluation?.recall == null ? 'Unavailable' : `${training.diagnostics.trainingAudit.learningProbe.evaluation.recall.toFixed(3)} / ${training.diagnostics.trainingAudit.learningProbe.steps}`}</Row>
             <Row label="Steps between Latest updates">{display(limits?.training.checkpointEverySteps)}</Row>
           </dl>{training?.error && <p className="break-words text-amber-300">{training.error}</p>}
             {training?.reason && <p>{training.reason}</p>}

webui/src/context/SessionContext.jsx

@@ -16,7 +16,6 @@ const INITIAL_STATE = {
   overseerMemory: null,
   alerts: [],
   latestReplay: null,
-  replayStatus: null,
   duplicateIdentityBlock: null,
   roverRemovalNotice: null,
 };
@@ -199,23 +198,6 @@ export function SessionProvider({ children }) {
         ],
       }));
     }
-    function handleReplayStatus(payload = {}) {
-      if (!payload?.status) return;
-      setState((prev) => ({
-        ...prev,
-        /*
-          Replay creation is serialized by the server's cooldown and workflow,
-          so the browser needs one current progress value rather than a map of
-          jobs. The server remains authoritative for job identity and execution;
-          this scalar exists only to present accepted/building/uploading/ready/
-          failed progress in the web UI.
-        */
-        replayStatus: {
-          ...payload,
-          receivedAt: Date.now(),
-        },
-      }));
-    }
     function handleReplayReady(payload = {}) {
       if (!payload?.jobId || !payload?.url) return;
       setState((prev) => ({
@@ -316,7 +298,6 @@ export function SessionProvider({ children }) {
     socket.on('overseer:state', handleOverseerState);
     socket.on('overseer:memory', handleOverseerMemory);
     socket.on('alert:new', handleAlertNew);
-    socket.on('replay:status', handleReplayStatus);
     socket.on('replay:ready', handleReplayReady);
     socket.on('replay:failed', handleReplayFailed);
     socket.on('session:duplicateIdentity', handleDuplicateIdentity);
@@ -331,7 +312,6 @@ export function SessionProvider({ children }) {
       socket.off('overseer:state', handleOverseerState);
       socket.off('overseer:memory', handleOverseerMemory);
       socket.off('alert:new', handleAlertNew);
-      socket.off('replay:status', handleReplayStatus);
       socket.off('replay:ready', handleReplayReady);
       socket.off('replay:failed', handleReplayFailed);
       socket.off('session:duplicateIdentity', handleDuplicateIdentity);

webui/src/context/telemetryViews.js

@@ -11,6 +11,10 @@ const EMPTY_BATTERY_TELEMETRY = Object.freeze({
 });
 
 const EMPTY_DOCK_TELEMETRY = Object.freeze({
+  dockingPhase: 'idle',
+  movementLocked: true,
+  cameraLocked: false,
+  dockingError: '',
   oiModeLabel: 'Unknown',
   chargingStateLabel: '',
   homeBase: false,
@@ -127,6 +131,10 @@ export function selectDockTelemetry(frame) {
   const sensors = frame?.sensors;
   if (!sensors) return EMPTY_DOCK_TELEMETRY;
   return {
+    dockingPhase: sensors.docking?.phase || 'idle',
+    movementLocked: sensors.docking?.movementLocked ?? true,
+    cameraLocked: Boolean(sensors.docking?.cameraLocked),
+    dockingError: sensors.docking?.error?.message || '',
     oiModeLabel: sensors.oiMode?.label || 'Unknown',
     chargingStateLabel: sensors.chargingState?.label || '',
     // OI packet 34 is the authoritative charging-sources packet. Charging

webui/src/controls/ControlContext.jsx

@@ -17,13 +17,11 @@ import { computeDifferentialSpeeds, clamp } from './controlMath.js';
 import { useCommandPipeline } from './commandPipeline.js';
 import {
   DEFAULT_KEYMAP,
-  DEFAULT_MACROS,
   HORN_HEAT_COOL_PER_SEC,
   HORN_HEAT_RESUME_THRESHOLD,
   HORN_HEAT_UP_PER_SEC,
   HORN_MAX_FREQUENCY,
   HORN_MAX_MS,
-  MANUAL_DOCK_ASSIST_MAX_SPEED,
   SONG_DEFAULT_NOTE,
 } from './constants.js';
 import { canonicalizeKeyInput } from './keymapUtils.js';
@@ -32,6 +30,8 @@ import { useSessionActions, useSessionSelector } from '../context/SessionContext
 import { HORN_SETTINGS_DEFAULTS } from '../settings/namespaces.js';
 import { useOvercurrentLimiter } from './overcurrentLimiter.js';
 import { usePtzControlAdapter } from './ptzControlAdapter.js';
+import { useTelemetrySelector } from '../context/TelemetryContext.jsx';
+import { selectDockTelemetry, dockTelemetryEqual } from '../context/telemetryViews.js';
 import { trackAnalyticsEvent } from '../analytics/index.js';
 
 const ControlSystemContext = createContext(null);
@@ -49,7 +49,8 @@ const CONTROL_ACTION_NAMES = [
   'setCameraAxisIntent',
   'goServoHome',
   'setCameraPrecisionMode',
-  'runMacro',
+  'undock',
+  'sendCommand',
   'stopAllMotion',
   'sendOiCommand',
   'setSensorStream',
@@ -60,8 +61,6 @@ const CONTROL_ACTION_NAMES = [
   'updateKeyBinding',
   'resetKeyBindings',
   'registerInputState',
-  'setManualDockAssistActive',
-  'toggleManualDockAssist',
   'setSongNote',
   'sendSong',
   'startHorn',
@@ -83,23 +82,6 @@ function clampServoAngle(config, value) {
   return clamp(value, min, max);
 }
 
-function removeDriveSequenceBackoff(steps = []) {
-  if (!Array.isArray(steps) || steps.length < 3) return steps;
-  const last3 = steps.slice(-3);
-  const [backoffStep, pauseStep, stopStep] = last3;
-  const isBackoffStep =
-    backoffStep?.type === 'drive' &&
-    Number(backoffStep?.speeds?.left) === -300 &&
-    Number(backoffStep?.speeds?.right) === -300;
-  const isPauseStep = pauseStep?.type === 'pause' && Number(pauseStep?.duration) === 600;
-  const isStopStep =
-    stopStep?.type === 'drive' &&
-    Number(stopStep?.speeds?.left) === 0 &&
-    Number(stopStep?.speeds?.right) === 0;
-  if (!isBackoffStep || !isPauseStep || !isStopStep) return steps;
-  return steps.slice(0, -3);
-}
-
 export function ControlSystemProvider({ children }) {
   const [state, dispatch] = useReducer(controlReducer, initialControlState);
   const snapshotRef = useRef(null);
@@ -158,7 +140,7 @@ export function ControlSystemProvider({ children }) {
   const {
     value: controlSettings,
     save: saveControlSettings,
-  } = useSettingsNamespace('controls', { keymap: DEFAULT_KEYMAP, macros: DEFAULT_MACROS });
+  } = useSettingsNamespace('controls', { keymap: DEFAULT_KEYMAP });
   const { value: pageSettings } = useSettingsNamespace('page', { driveMacroBackoffEnabled: true });
   const { value: hornSettings } = useSettingsNamespace('horn', HORN_SETTINGS_DEFAULTS);
   const driveMacroBackoffEnabled =
@@ -166,6 +148,8 @@ export function ControlSystemProvider({ children }) {
       ? pageSettings.driveMacroBackoffEnabled
       : true;
   const roverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
+  const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
+  const cameraLocked = Boolean(dockTelemetry.cameraLocked);
   const homeAssistantEntities = useSessionSelector((state) => state.session?.homeAssistant?.entities ?? []);
   const roomLightsLocked = useSessionSelector((state) =>
     Boolean(state.session?.homeAssistant?.lightPolicy?.locked || state.session?.homeAssistant?.lightPolicy?.lockedOn),
@@ -238,10 +222,7 @@ export function ControlSystemProvider({ children }) {
   useEffect(() => {
     const mergedKeymap = { ...DEFAULT_KEYMAP, ...(controlSettings?.keymap || {}) };
     dispatch({ type: 'control/set-keymap', payload: mergedKeymap });
-    if (controlSettings?.macros) {
-      dispatch({ type: 'control/set-macros', payload: controlSettings.macros });
-    }
-  }, [controlSettings?.keymap, controlSettings?.macros]);
+  }, [controlSettings?.keymap]);
 
   useEffect(() => {
     const config = pipeline.servoConfig;
@@ -268,11 +249,10 @@ export function ControlSystemProvider({ children }) {
       typeof state.camera.angle === 'number' ? state.camera.angle : servoAngleRef.current;
   }, [state.camera.angle]);
 
+  const { enableSensorStream } = pipeline;
   useEffect(() => {
-    if (pipeline.roverId) {
-      pipeline.enableSensorStream();
-    }
-  }, [pipeline.roverId, pipeline.enableSensorStream]);
+    if (roverId) enableSensorStream();
+  }, [roverId, enableSensorStream]);
 
   const setMode = useCallback(
     (mode) => {
@@ -295,21 +275,6 @@ export function ControlSystemProvider({ children }) {
   const setDriveVector = useCallback(
     (vector, meta = {}) => {
       const speedOptions = { ...(meta.speedOptions || {}) };
-      if (state.manualDockAssist?.active) {
-        const capped = Math.max(1, Math.min(MANUAL_DOCK_ASSIST_MAX_SPEED, 500));
-        speedOptions.maxSpeed = Math.min(
-          typeof speedOptions.maxSpeed === 'number' ? speedOptions.maxSpeed : capped,
-          capped,
-        );
-        speedOptions.baseSpeed = Math.min(
-          typeof speedOptions.baseSpeed === 'number' ? speedOptions.baseSpeed : capped,
-          capped,
-        );
-        speedOptions.boostSpeed = Math.min(
-          typeof speedOptions.boostSpeed === 'number' ? speedOptions.boostSpeed : capped,
-          capped,
-        );
-      }
       const computed = computeDifferentialSpeeds(vector, speedOptions);
       const isMoving = computed.speeds.left !== 0 || computed.speeds.right !== 0;
       if (pipeline.roverId && isMoving && driveEngagementRoverRef.current !== pipeline.roverId) {
@@ -327,7 +292,7 @@ export function ControlSystemProvider({ children }) {
       if (ptzControls.applyDriveVector(vector, meta)) return;
       pipeline.sendDriveDirect(computed.speeds);
     },
-    [pipeline, ptzControls, recordControlIntent, state.manualDockAssist?.active],
+    [pipeline, ptzControls, recordControlIntent],
   );
 
   const setAuxMotors = useCallback(
@@ -369,23 +334,22 @@ export function ControlSystemProvider({ children }) {
   }, [saveControlSettings]);
 
   const setServoAngle = useCallback(
-    (value, options = {}) => {
+    (value) => {
       /*
         Absolute servo positions belong only to rover hardware. PTZ zoom now
         enters through setCameraAxisIntent as a signed held velocity, so this
         function must not infer zoom direction by comparing unrelated absolute
-        angle values from gamepad/manual-dock callers.
+        angle values from camera controls.
       */
       if (!pipeline.servoConfig) return;
-      const force = Boolean(options?.force);
-      if (state.manualDockAssist?.active && !force) return;
+      if (cameraLocked) return;
       const clamped = clampServoAngle(pipeline.servoConfig, value);
       dispatch({ type: 'control/set-camera-angle', payload: clamped });
       pipeline.sendServoAngle(clamped);
       servoAngleRef.current = clamped;
       recordControlIntent();
     },
-    [pipeline, recordControlIntent, state.manualDockAssist?.active],
+    [pipeline, recordControlIntent, cameraLocked],
   );
 
   const nudgeServo = useCallback(
@@ -457,35 +421,19 @@ export function ControlSystemProvider({ children }) {
     dispatch({ type: 'control/set-camera-precision-mode', payload: Boolean(active) });
   }, []);
 
-  const runMacro = useCallback(
-    async (macroId) => {
-      const macro = state.macros.find((item) => item.id === macroId) || null;
-      if (!macro) return;
-      let macroToRun = macro;
-      if (macroId === 'drive-sequence') {
-        turnOnAllLights();
-        if (!homeAssistantEntities?.length) {
-          pendingLightsRef.current = true;
-        }
-        recordControlIntent();
-        if (!driveMacroBackoffEnabled) {
-          macroToRun = {
-            ...macro,
-            steps: removeDriveSequenceBackoff(macro.steps),
-          };
-        }
-      }
-      await pipeline.runMacroSteps(macroToRun);
-    },
-    [
-      driveMacroBackoffEnabled,
-      pipeline,
-      recordControlIntent,
-      homeAssistantEntities,
-      state.macros,
-      turnOnAllLights,
-    ],
-  );
+  const sendCommand = useCallback((type, data = {}) => {
+    recordControlIntent();
+    pipeline.emitCommand({ type, data }, (result) => {
+      if (result?.error) alert(result.error);
+    });
+  }, [pipeline, recordControlIntent]);
+
+  const undock = useCallback(() => {
+    dispatch({ type: 'control/set-mode', payload: 'drive' });
+    turnOnAllLights();
+    if (!homeAssistantEntities?.length) pendingLightsRef.current = true;
+    sendCommand('undock', { backoff: driveMacroBackoffEnabled });
+  }, [driveMacroBackoffEnabled, homeAssistantEntities, turnOnAllLights, sendCommand]);
 
   const stopAllMotion = useCallback(() => {
     dispatch({
@@ -691,32 +639,6 @@ export function ControlSystemProvider({ children }) {
     return () => clearInterval(interval);
   }, [dispatch, hornNeedsTick]);
 
-  const setManualDockAssistActive = useCallback(
-    (active) => {
-      const nextActive = Boolean(active);
-      const prevActive = Boolean(state.manualDockAssist?.active);
-      if (prevActive === nextActive) return;
-      dispatch({ type: 'control/set-manual-dock-assist', payload: nextActive });
-      if (!nextActive) {
-        pipeline.sendSong([{ note: 83, duration: 10 }, { note: 76, duration: 10 }], { slot: 0 });
-        setServoAngle(0, { force: true });
-        return;
-      }
-      const minAngle =
-        typeof pipeline.servoConfig?.minAngle === 'number'
-          ? pipeline.servoConfig.minAngle
-          : -45;
-      setServoAngle(minAngle, { force: true });
-      pipeline.sendSong([{ note: 76, duration: 10 }, { note: 83, duration: 10 }], { slot: 0 });
-      recordControlIntent();
-    },
-    [pipeline, recordControlIntent, setServoAngle, state.manualDockAssist?.active],
-  );
-
-  const toggleManualDockAssist = useCallback(() => {
-    setManualDockAssistActive(!state.manualDockAssist?.active);
-  }, [setManualDockAssistActive, state.manualDockAssist?.active]);
-
   const registerInputState = useCallback((source, data) => {
     dispatch({ type: 'control/register-input-state', payload: { source, state: data } });
   }, []);
@@ -758,7 +680,8 @@ export function ControlSystemProvider({ children }) {
       setCameraAxisIntent,
       goServoHome,
       setCameraPrecisionMode,
-      runMacro,
+      undock,
+      sendCommand,
       stopAllMotion,
       sendOiCommand,
       setSensorStream,
@@ -769,8 +692,6 @@ export function ControlSystemProvider({ children }) {
       updateKeyBinding,
       resetKeyBindings,
       registerInputState,
-      setManualDockAssistActive,
-      toggleManualDockAssist,
       setSongNote,
       sendSong,
       startHorn,
@@ -787,7 +708,8 @@ export function ControlSystemProvider({ children }) {
       setCameraAxisIntent,
       goServoHome,
       setCameraPrecisionMode,
-      runMacro,
+      undock,
+      sendCommand,
       stopAllMotion,
       sendOiCommand,
       setSensorStream,
@@ -798,8 +720,6 @@ export function ControlSystemProvider({ children }) {
       updateKeyBinding,
       resetKeyBindings,
       registerInputState,
-      setManualDockAssistActive,
-      toggleManualDockAssist,
       setSongNote,
       sendSong,
       startHorn,

webui/src/controls/commandPipeline.js

@@ -5,13 +5,12 @@ import { useSocket } from '../context/SocketContext.jsx';
 import { useSessionSelector } from '../context/SessionContext.jsx';
 import {
   AUX_LIMITS,
-  COMMAND_DELAY_MS,
   OI_COMMANDS,
   SONG_DEFAULT_DURATION,
   SONG_DEFAULT_NOTE,
   SONG_NOTE_RANGE,
 } from './constants.js';
-import { bytesToBase64, clampRange, sleep } from './controlMath.js';
+import { bytesToBase64, clampRange } from './controlMath.js';
 
 export function useCommandPipeline(options = {}) {
   const { driveTransform, auxTransform } = options;
@@ -53,8 +52,17 @@ export function useCommandPipeline(options = {}) {
   const laserState = useMemo(() => rosterEntry?.laser?.state ?? null, [rosterEntry]);
   const emitCommand = useCallback(
     (payload, cb) => {
-      if (!roverId) return;
-      socket.emit('command', { roverId, ...payload }, cb);
+      if (!roverId || !socket.connected) {
+        cb?.({ error: 'Rover control connection unavailable' });
+        return;
+      }
+      if (cb) {
+        socket.timeout(5000).emit('command', { roverId, ...payload }, (error, result) => {
+          cb(error ? { error: 'Rover command acknowledgement timed out' } : result);
+        });
+      } else {
+        socket.emit('command', { roverId, ...payload });
+      }
     },
     [socket, roverId],
   );
@@ -142,41 +150,6 @@ export function useCommandPipeline(options = {}) {
     [emitCommand, enableSensorStream, roverId],
   );
 
-  const runMacroSteps = useCallback(
-    async (macro) => {
-      if (!macro || !Array.isArray(macro.steps) || !roverId) return;
-      for (const step of macro.steps) {
-        if (!roverId) break;
-        switch (step.type) {
-          case 'oi':
-            sendOiCommand(step.command);
-            break;
-          case 'drive':
-            sendDriveDirect(step.speeds ?? { left: 0, right: 0 });
-            break;
-          case 'motors':
-            sendAuxMotors(step.values ?? {});
-            break;
-          case 'servo':
-            sendServoAngle(step.angle);
-            break;
-          case 'pause':
-            await sleep(step.duration ?? COMMAND_DELAY_MS); // eslint-disable-line no-await-in-loop
-            break;
-          default:
-            break;
-        }
-        if (step.delay || step.delayMs) {
-          const delay = step.delayMs ?? step.delay;
-          if (typeof delay === 'number' && delay > 0) {
-            await sleep(delay); // eslint-disable-line no-await-in-loop
-          }
-        }
-      }
-    },
-    [roverId, sendOiCommand, sendDriveDirect, sendAuxMotors, sendServoAngle],
-  );
-
   const sendHeadlight = useCallback(
     (action = 'toggle') => {
       if (!roverId || !headlight) return null;
@@ -277,7 +250,6 @@ export function useCommandPipeline(options = {}) {
       sendHorn,
       sendPeripheralControl,
       sendSong,
-      runMacroSteps,
     }),
     [
       roverId,
@@ -300,7 +272,6 @@ export function useCommandPipeline(options = {}) {
       sendHorn,
       sendPeripheralControl,
       sendSong,
-      runMacroSteps,
     ],
   );
 }

webui/src/controls/constants.js

@@ -12,10 +12,6 @@ export const DRIVE_LIMITS = {
   boostSpeed: 400,
 };
 
-export const MANUAL_DOCK_ASSIST_MAX_SPEED = 60;
-
-export const COMMAND_DELAY_MS = 100;
-
 export const SONG_NOTE_RANGE = [31, 127];
 export const SONG_DEFAULT_NOTE = 60;
 export const SONG_DEFAULT_DURATION = 8;
@@ -64,23 +60,3 @@ export const DEFAULT_KEYMAP = {
   homeAssistantOn: ['ArrowRight'],
   homeAssistantOff: ['ArrowLeft'],
 };
-
-export const DEFAULT_MACROS = [
-  {
-    id: 'drive-sequence',
-    label: 'Drive',
-    description: 'Start, undock, and full command sequence used by the drive button.',
-    steps: [
-      { type: 'servo', angle: 0 },
-      { type: 'oi', command: 'start' },
-      { type: 'pause', duration: COMMAND_DELAY_MS },
-      { type: 'oi', command: 'dock' },
-      { type: 'pause', duration: COMMAND_DELAY_MS },
-      { type: 'oi', command: 'full' },
-      { type: 'pause', duration: 300 },
-      { type: 'drive', speeds: { left: -300, right: -300 } },
-      { type: 'pause', duration: 600 },
-      { type: 'drive', speeds: { left: 0, right: 0 } },
-    ],
-  },
-];

webui/src/controls/controlMath.js

@@ -55,7 +55,3 @@ export function bytesToBase64(bytes) {
   }
   throw new Error('No base64 encoder available');
 }
-
-export function sleep(ms) {
-  return new Promise((resolve) => setTimeout(resolve, ms));
-}

webui/src/controls/controlReducer.js

@@ -1,6 +1,6 @@
 // Control State Reducer
 // Purpose: Implements reducer transitions for control-system runtime state. Scope: Centralizes deterministic state updates for input, mode, and dispatch events.
-import { DEFAULT_KEYMAP, DEFAULT_MACROS, SONG_DEFAULT_NOTE } from './constants.js';
+import { DEFAULT_KEYMAP, SONG_DEFAULT_NOTE } from './constants.js';
 
 function createDriveState() {
   return {
@@ -49,12 +49,6 @@ function createMicState() {
   };
 }
 
-function createManualDockAssistState() {
-  return {
-    active: false,
-  };
-}
-
 export const initialControlState = {
   roverId: null,
   mode: 'drive',
@@ -64,9 +58,7 @@ export const initialControlState = {
   song: createSongState(),
   horn: createHornState(),
   mic: createMicState(),
-  manualDockAssist: createManualDockAssistState(),
   lastControlIntentAt: 0,
-  macros: DEFAULT_MACROS,
   keymap: DEFAULT_KEYMAP,
   inputs: {},
   // Generic accessory controls do not currently report state back from roverd.
@@ -86,7 +78,6 @@ export function controlReducer(state, action) {
         song: action.payload ? state.song : createSongState(),
         horn: action.payload ? state.horn : createHornState(),
         mic: action.payload ? state.mic : createMicState(),
-        manualDockAssist: action.payload ? state.manualDockAssist : createManualDockAssistState(),
         lastControlIntentAt: action.payload ? state.lastControlIntentAt : 0,
       };
     case 'control/set-mode':
@@ -169,11 +160,6 @@ export function controlReducer(state, action) {
         ...state,
         keymap: { ...state.keymap, ...(action.payload ?? {}) },
       };
-    case 'control/set-macros':
-      return {
-        ...state,
-        macros: Array.isArray(action.payload) ? action.payload : state.macros,
-      };
     case 'control/reset':
       return {
         ...state,
@@ -222,14 +208,6 @@ export function controlReducer(state, action) {
           pttActive: Boolean(action.payload),
         },
       };
-    case 'control/set-manual-dock-assist':
-      return {
-        ...state,
-        manualDockAssist: {
-          ...(state.manualDockAssist || createManualDockAssistState()),
-          active: Boolean(action.payload),
-        },
-      };
     case 'control/set-peripheral-value': {
       const roverId = String(action.payload?.roverId || '');
       const peripheralId = String(action.payload?.peripheralId || '');

webui/src/controls/inputs/GamepadInputManager.jsx

@@ -1,3 +1,5 @@
+import { dockTelemetryEqual, selectDockTelemetry } from '../../context/telemetryViews.js';
+import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
 // Gamepad Input Manager
 // Purpose: Converts polled gamepad state into normalized control actions/commands. Scope: Integrates bindings, deadzone math, and dispatch callbacks for driving.
 import { useCallback, useEffect, useLayoutEffect, useMemo, useRef } from 'react';
@@ -17,7 +19,6 @@ import {
 } from './gamepadBindings.js';
 import { subscribeGamepadHub } from './gamepadHub.js';
 import { isTextEntryActive } from './inputFocusUtils.js';
-import { useManualDockAssist } from '../../features/manualDockAssist/useManualDockAssist.js';
 import {
   isControllerControlLocked,
   markControllerDisconnected,
@@ -113,12 +114,12 @@ function cycleHomeAssistant(latest, targetState) {
 
 export default function GamepadInputManager() {
   const {
-    setMode,
     setDriveVector,
     setAuxMotors,
     setServoAngle,
     setCameraAxisIntent,
-    runMacro,
+    undock,
+    sendCommand,
     toggleHeadlight,
     toggleLaser,
     registerInputState,
@@ -131,7 +132,7 @@ export default function GamepadInputManager() {
   const cameraAngle = useControlSelector((control) => control.state.camera?.angle);
   const cameraConfig = useControlSelector((control) => control.state.camera?.config);
   const roverId = useControlSelector((control) => control.state.roverId);
-  const dockAssist = useManualDockAssist();
+  const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
   const { focusChat } = useChatActions();
   const { isChatFocused } = useChatFocus();
   const { homeAssistantSetState, pushAlert } = useSessionActions();
@@ -283,7 +284,7 @@ export default function GamepadInputManager() {
         role === 'lockdown' || (role === 'admin' && sessionMode !== 'lockdown'),
       cameraAngle,
       cameraConfig,
-      dockAssist,
+      dockTelemetry,
       focusChat,
       gamepadSettings,
       homeAssistant,
@@ -292,7 +293,8 @@ export default function GamepadInputManager() {
       pushAlert,
       registerInputState,
       roverId,
-      runMacro,
+      undock,
+      sendCommand,
       saveGamepadSettings,
       saveVideoSettings,
       sendSong,
@@ -300,8 +302,7 @@ export default function GamepadInputManager() {
       setCameraAxisIntent,
       setDriveVector,
       setMicPttActive,
-      setMode,
-      setSongNote,
+        setSongNote,
       setServoAngle,
       songNote,
       startHorn,
@@ -462,15 +463,13 @@ export default function GamepadInputManager() {
       }
 
       if (outputs.buttons.driveMacro && handleButtonEdge('driveMacro', true)) {
-        latest.dockAssist.exitAssist();
-        latest.setMode('drive');
-        latest.runMacro('drive-sequence');
+        latest.undock();
       } else if (!outputs.buttons.driveMacro) {
         handleButtonEdge('driveMacro', false);
       }
 
       if (outputs.buttons.dockMacro && handleButtonEdge('dockMacro', true)) {
-        latest.dockAssist.toggleAssist();
+        latest.sendCommand(latest.dockTelemetry.cameraLocked ? 'cancelDocking' : 'enterDocking');
       } else if (!outputs.buttons.dockMacro) {
         handleButtonEdge('dockMacro', false);
       }