From b2bbaf2085ee12a9c9ad5035601945a24f9766a7 Mon Sep 17 00:00:00 2001 From: legop3 Date: Tue, 29 Sep 2026 22:14:57 -0400 Subject: [PATCH] slop --- plans/rover learning service.md | 70 ++++++++++++++----- .../roverLearningService/capabilities.js | 4 +- .../services/roverLearningService/driving.js | 2 +- .../services/roverLearningService/models.js | 4 +- .../services/roverLearningService/training.js | 4 +- .../roverLearningService/workers/dataset.py | 39 ++++++++--- .../roverLearningService/workers/policy.py | 50 +++++++++---- .../roverLearningService/workers/train.py | 27 ++++--- .../components/vip/VipRoverLearningCard.jsx | 2 +- 9 files changed, 142 insertions(+), 60 deletions(-) diff --git a/plans/rover learning service.md b/plans/rover learning service.md index 806770a8..d8104f55 100644 --- a/plans/rover learning service.md +++ b/plans/rover learning service.md @@ -2,27 +2,21 @@ ## Resume here -Latest change: format 3 command-event policy replaces format 2 continuous wheel -speeds. All supported controls, including drive and recorded zero-speed stops, -use scored command templates and the same threshold/checked command pipe. The -network produces scores and numeric values; it never generates JSON text. +Latest change (2026-09-29 IPI): format 4 seeds known user-control command shapes +before training. Generic peripherals are excluded; sensorStream is ignored without +invalidating footage. Drive includes zero-speed drive, with no separate stop type. +Drive, camera, brushes/vacuum, headlight, laser, horn, songs and operating modes +remain supported. Raw recordings and the existing checked command pipe stay intact. -Implemented: remove wheel-speed telemetry and previous drive-command history -from inputs (raw recordings unchanged); preserve exact numeric zeros; reuse -recordings with a fresh resume-v4.pt / schedule-v4.json lineage. Startup deletes recognized older-format snapshots/checkpoints and old training -lineages; recordings and current-format models are preserved. One ongoing learner, gated -Latest/Previous, and admin-created permanent snapshots remain. +Held-out data cannot expand the schema. Unknown control variants exclude nearby +windows only, with command/count/reason diagnostics and a coverage gate that blocks +promotion. Supported portions still receive evaluation. Current training lineage +is resume-v5.pt / schedule-v5.json. Startup removes obsolete format 1-3 models and +old lineages through v4; recordings and current-format models are preserved. -Main card shows decoded command parameters, score, applied threshold marker, and -submission results, including below-threshold proposals. Diagnostics reportVersion -3 includes command predictions/submissions and event evaluation. Manual inputs -still never stop or pause autonomy. Disabled remains dormant/off by default. - -Focused Node syntax, Python production worker entry points, and card ESLint -passed. New policy: 803,440 parameters, 30 sensor fields, stateSize 700. No live -recordings/camera/rover pipeline or authenticated browser were available locally. -Actual training, stop behavior, concurrent performance and visual appearance -remain unverified. No tests were added and no processes were left running. +Verification for this increment is recorded at the end. Earlier version-specific +notes are historical. Manual inputs never pause/cancel autonomy; disabled service +remains dormant. Actual autonomous performance has not been established. This document preserves the conversation's decisions across compactions and usage cutoffs. Keep the checklist and handoff section current during authorized @@ -930,3 +924,41 @@ Cleanup runs even when training is disabled, but not when the whole service is o Production deletion occurs on the next enabled-service startup after deployment. Verification: focused Node syntax and diff whitespace checks passed. No production storage was accessed; actual deletion awaits enabled-service startup on the server. + +### User-control scope and evaluation coverage (2026-09-29) + +Authorized by latest IPI after user clarified that controls include camera, brushes, +lights, laser, horn and songs but NOT generic peripherals. No new command pipe or +separate stop command. sensorStream is neither input/target nor a boundary. Actual +reset/reboot/update remain temporal boundaries because they interrupt observations. + +Format 4 seeds drive and motor zero/nonzero field combinations, servo, light/laser +actions, the two four-frequency horn waveforms and horn release, OI start/safe/full/ +dock bytes and default song. Seed values describe shapes, not artificial training +examples. Horn frequencies are numeric parameters; integral JSON numbers share +shapes with equivalent floating representations for drive/motors/servo/horn. Songs +remain exact demonstrated note arrays; other song/raw variants can extend the +training schema within its existing 64-slot bound. No promise to pre-enumerate all +possible tunes. Seeded slots receive negative examples even in sessions where that +control was not used, avoiding untrained random seeded outputs. + +Evaluation freezes the training schema. Unsupported commands are reported with +payload, reason, occurrence count (up to 32 distinct descriptions), total count and +affected-window count. The affected observation history/target interval is excluded +without labeling the unknown command as inaction or skipping the entire recording. +Any unsupported held-out command blocks promotion; metrics describe only supported +windows. If none survive, dataset diagnostics still report the coverage problem. +Existing permission checks, threshold mechanics, recording and manual input behavior +are unchanged. Generic peripherals are also removed from execution allowlist and +rover compatibility profile. Previous policy models are automatically cleaned up; +no configuration edit is required. + +Verification completed: train.py --check-dependencies loaded the production +worker and constructed format 4 with 803,440 parameters and predefined command +shapes. infer.py --help, Node syntax checks for the four changed service modules, +focused VipRoverLearningCard ESLint and diff whitespace checks passed. +No live server data/hardware/browser available locally. Actual training, driving, +and browser appearance remain unverified. No new tests or background processes. +Deploying starts a new learner using existing recordings; enabled-service startup +removes older policy models and optimizer/scheduler lineages through v4. Current +lineage is v5. No settings changes or prescribed human driving are required. diff --git a/server/src/services/roverLearningService/capabilities.js b/server/src/services/roverLearningService/capabilities.js index 977ae895..a3c2b0f1 100644 --- a/server/src/services/roverLearningService/capabilities.js +++ b/server/src/services/roverLearningService/capabilities.js @@ -1,9 +1,9 @@ // Match actuator configuration, not a rover's display name or changing telemetry. const { isDeepStrictEqual } = require('node:util'); -const POLICY_VERSION = 3; +const POLICY_VERSION = 4; function controlProfile(meta = {}) { - return Object.fromEntries(['cameraServo', 'peripherals', 'headlight', 'laser', 'roomba', 'platform'] + return Object.fromEntries(['cameraServo', 'headlight', 'laser', 'roomba', 'platform'] .map((key) => [key, meta?.[key] ?? null])); } diff --git a/server/src/services/roverLearningService/driving.js b/server/src/services/roverLearningService/driving.js index 23df69c9..e4ffded3 100644 --- a/server/src/services/roverLearningService/driving.js +++ b/server/src/services/roverLearningService/driving.js @@ -9,7 +9,7 @@ const { recordCommandRequest } = require('../commandService/recording'); const { acquireModel } = require('./models'); const { compatible } = require('./capabilities'); -const OPERATING = new Set(['drive', 'motors', 'servo', 'headlight', 'laser', 'horn', 'peripheral', 'song', 'raw']); +const OPERATING = new Set(['drive', 'motors', 'servo', 'headlight', 'laser', 'horn', 'song', 'raw']); const clamp = (value, min, max) => Math.max(min, Math.min(max, Number(value) || 0)); function canControl(socket, roverId) { diff --git a/server/src/services/roverLearningService/models.js b/server/src/services/roverLearningService/models.js index 92e4d22f..ef891c28 100644 --- a/server/src/services/roverLearningService/models.js +++ b/server/src/services/roverLearningService/models.js @@ -142,8 +142,8 @@ function removeObsoleteModels(root) { else { await fs.rm(target, { recursive: true, force: true }); retired.delete(target); } } // These filenames belong to superseded optimizer/scheduler lineages, not - // policy versions. The current command policy uses lineage v4. - for (const suffix of ['', '-v1', '-v2', '-v3']) { + // policy versions. The current command policy uses lineage v5. + for (const suffix of ['', '-v1', '-v2', '-v3', '-v4']) { for (const filename of [`resume${suffix}.pt`, `resume${suffix}.tmp`, `schedule${suffix}.json`, `schedule${suffix}.json.tmp`]) { await fs.rm(path.join(root, 'training', filename), { force: true }); diff --git a/server/src/services/roverLearningService/training.js b/server/src/services/roverLearningService/training.js index fcf97c23..bb412a4c 100644 --- a/server/src/services/roverLearningService/training.js +++ b/server/src/services/roverLearningService/training.js @@ -33,7 +33,7 @@ function createTrainer({ root, config, actionThreshold, logger }) { } } try { - const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v4.json'), 'utf8')); + const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v5.json'), 'utf8')); ledger = saved.sessions; state = { ...saved.progress, status: 'waiting' }; if (!ledger || typeof ledger !== 'object') throw new Error('Invalid training schedule'); @@ -46,7 +46,7 @@ function createTrainer({ root, config, actionThreshold, logger }) { } async function saveLedger() { - const filename = path.join(trainingRoot, 'schedule-v4.json'); + const filename = path.join(trainingRoot, 'schedule-v5.json'); await fs.writeFile(`${filename}.tmp`, JSON.stringify({ sessions: ledger, progress: state })); await fs.rename(`${filename}.tmp`, filename); } diff --git a/server/src/services/roverLearningService/workers/dataset.py b/server/src/services/roverLearningService/workers/dataset.py index fcb851d6..b43ddf1b 100644 --- a/server/src/services/roverLearningService/workers/dataset.py +++ b/server/src/services/roverLearningService/workers/dataset.py @@ -47,7 +47,7 @@ def load_events(directory): return metadata, sorted(events, key=lambda e: e['ts']) -def prepare(directory, schema, stats): +def prepare(directory, schema, stats, extend): metadata, events = load_events(directory) requests = {} accepted = set() @@ -72,14 +72,27 @@ def prepare(directory, schema, stats): if not sensors: raise ValueError('No sensor observations') candidate = copy.deepcopy(schema) - history, targets = [], [] + history, targets, unsupported_times = [], [], [] + unsupported = {} stats.update({'sensorFrames': len(sensors), 'dispatches': len(dispatches), 'failedCommands': len(failed), 'sensorPresentFraction': np.mean([row[1][len(SENSORS):] for row in sensors], axis=0).tolist()}) for data in dispatches: command = data.get('command') or {} - if command.get('type') in ('reset', 'reboot', 'update', 'sensorStream'): + if command.get('type') in ('reset', 'reboot', 'update'): boundaries.append(data['ts']) - encoded = encode_command(command, candidate, extend=True) + try: + encoded = encode_command(command, candidate, extend=extend) + except ValueError as error: + # Unrepresentable controls invalidate only nearby observations/targets, + # not the rest of a recording. Never turn them into negative labels. + request = requests.get(data.get('requestId')) + unsupported_times.extend([data['ts'], request['ts'] if request else data['ts']]) + key = json.dumps(command, sort_keys=True) + if key not in unsupported and len(unsupported) < 32: + unsupported[key] = {'command': command, 'reason': str(error), 'count': 0} + if key in unsupported: + unsupported[key]['count'] += 1 + continue if encoded is None: continue stamp = data['ts'] @@ -95,24 +108,25 @@ def prepare(directory, schema, stats): # Server safety overrides belong in history, never demonstration # targets. Exclude windows around them rather than teaching inaction. boundaries.append(stamp) - if not targets: + stats['unsupportedCommands'] = list(unsupported.values()) + stats['unsupportedCommandEvents'] = len(unsupported_times) // 2 + if not targets and not unsupported_times: raise ValueError('No accepted client operating commands') schema[:] = candidate - return metadata, sensors, sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries) + return metadata, sensors, sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries), sorted(unsupported_times) -def windows(directory, schema, cancelled=lambda: False, stats=None): +def windows(directory, schema, cancelled=lambda: False, stats=None, extend=True): directory = Path(directory) stats = stats if stats is not None else {} - metadata, sensors, history, targets, boundaries = prepare(directory, schema, stats) + metadata, sensors, history, targets, boundaries, unsupported_times = prepare(directory, schema, stats, extend) stats.update({'usableWindows': 0, 'commandWindows': 0, 'stopWindows': 0, 'noCommandWindows': 0, 'staleSensorWindows': 0, 'boundaryWindows': 0, - 'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0}) + 'unsupportedWindows': 0, 'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0}) sensor_times = [row[0] for row in sensors] target_times = [row[0] for row in targets] available = np.zeros(SLOTS, dtype=np.float32) - for _, slot, _ in history: - available[slot] = 1 + available[:len(schema)] = 1 command_values = np.zeros((SLOTS, VALUES), dtype=np.float32) command_times = np.full(SLOTS, -np.inf) cursor = 0 @@ -172,6 +186,9 @@ def windows(directory, schema, cancelled=lambda: False, stats=None): chosen.append(options[-1]) if len(chosen) != 4 or stamp + 100 > metadata['endedAt']: continue + if bisect.bisect_right(unsupported_times, stamp + 100) > bisect.bisect_left(unsupported_times, chosen[0][0]): + stats['unsupportedWindows'] += 1 + continue if bisect.bisect_right(boundaries, stamp + 100) > bisect.bisect_left(boundaries, chosen[0][0]): stats['boundaryWindows'] += 1 continue diff --git a/server/src/services/roverLearningService/workers/policy.py b/server/src/services/roverLearningService/workers/policy.py index 8a98dc8d..543c0d83 100644 --- a/server/src/services/roverLearningService/workers/policy.py +++ b/server/src/services/roverLearningService/workers/policy.py @@ -4,6 +4,8 @@ Slots describe command shapes, not arbitrary executable instructions. Numeric leaves are regressed; string/bool/list values remain exact template constants. No command is sent by this module. Execution must use server authorization. """ +import base64 +import itertools import copy import json import math @@ -12,12 +14,12 @@ import numpy as np import torch from torch import nn -VERSION = 3 +VERSION = 4 SLOTS = 64 VALUES = 8 FRAMES = 4 HEIGHT, WIDTH = 120, 160 -OPERATING_TYPES = {"drive", "motors", "servo", "headlight", "laser", "horn", "peripheral", "song", "raw"} +OPERATING_TYPES = {"drive", "motors", "servo", "headlight", "laser", "horn", "song", "raw"} SENSORS = [ ("bumpsAndWheelDrops.bumpLeft", 1), ("bumpsAndWheelDrops.bumpRight", 1), ("bumpsAndWheelDrops.wheelDropLeft", 1), ("bumpsAndWheelDrops.wheelDropRight", 1), @@ -59,18 +61,20 @@ def describe_command(command): values, fields = [], [] def visit(value, path=()): + if isinstance(value, list) and command['type'] == 'horn' and path == ('horn', 'freqs'): + return [visit(item, (*path, index)) for index, item in enumerate(value)] if isinstance(value, dict): return {key: visit(item, (*path, key)) for key, item in sorted(value.items())} - # Exact zeros preserve demonstrated releases/stops. Identity fields must - # also stay exact: regression must never invent a peripheral address. - if value == 0 or (path and path[-1] in ('id', 'control')): + # Exact actuator zeros preserve demonstrated releases. Horn frequency + # zeros are parameters so changing a chord does not require a new slot. + if value == 0 and command['type'] != 'horn': return copy.deepcopy(value) if isinstance(value, (int, float)) and not isinstance(value, bool): if not math.isfinite(value) or len(values) >= VALUES: raise ValueError('Nonfinite command or too many numeric control fields') - scale = {'drive': 500, 'motors': 255, 'servo': 180, 'peripheral': 255, + scale = {'drive': 500, 'motors': 255, 'servo': 180, 'horn': 1000, 'song': 127}.get(command['type'], 1) - fields.append({'path': list(path), 'scale': scale, 'integer': isinstance(value, int)}) + fields.append({'path': list(path), 'scale': scale, 'integer': command['type'] in ('drive', 'motors', 'servo', 'horn') or isinstance(value, int)}) values.append(normalize(value, scale)) return {'$number': len(values) - 1} # Arrays (e.g. songs) are discrete exact demonstrations, not variable @@ -92,7 +96,7 @@ def encode_command(command, schema, extend=False): if not extend: raise ValueError('Command shape missing from model schema') if len(schema) >= SLOTS: - raise ValueError('64 command shapes exhausted; recording skipped, not silently truncated') + raise ValueError('64 command shapes exhausted; affected windows excluded') slot = len(schema) schema.append(descriptor) vector = np.zeros(VALUES, dtype=np.float32) @@ -100,10 +104,32 @@ def encode_command(command, schema, extend=False): return slot, vector +def initial_schema(): + # Match commandPipeline.js envelopes, OI_COMMANDS and the four-frequency + # horn control. Values are examples describing fields, not training labels. + commands = [] + for left, right in itertools.product((0, 1), repeat=2): + commands.append({'type': 'drive', 'driveDirect': {'left': left, 'right': right}}) + for main, side, vacuum in itertools.product((0, 1), repeat=3): + commands.append({'type': 'motors', 'motorPwm': {'main': main, 'side': side, 'vacuum': vacuum}}) + for angle in (0, 1): + commands.append({'type': 'servo', 'servo': {'angle': angle}}) + for kind in ('headlight', 'laser'): + for action in ('on', 'off', 'toggle'): + commands.append({'type': kind, kind: {'action': action}}) + for waveform in ('sine', 'saw'): + commands.append({'type': 'horn', 'horn': {'action': 'start', 'waveform': waveform, 'freqs': [1, 1, 1, 1]}}) + commands.append({'type': 'horn', 'horn': {'action': 'stop'}}) + for opcode in (128, 131, 132, 143): + commands.append({'type': 'raw', 'raw': base64.b64encode(bytes([opcode])).decode()}) + commands.append({'type': 'song', 'song': {'notes': [{'note': 60, 'duration': 8}]}}) + schema = [] + for command in commands: + encode_command(command, schema, extend=True) + return schema + + def command_family(command): - if command['type'] == 'peripheral': - peripheral = command.get('peripheral') or {} - return ('peripheral', str(peripheral.get('id')), str(peripheral.get('control'))) return (command['type'],) @@ -162,7 +188,7 @@ class Policy(nn.Module): def specification(schema): - return {'version': VERSION, 'architecture': 'spatial-cnn-gru-events-v3', + return {'version': VERSION, 'architecture': 'spatial-cnn-gru-events-v4', 'frames': FRAMES, 'height': HEIGHT, 'width': WIDTH, 'historyStrideMs': 300, 'decisionIntervalMs': 100, 'stateSize': STATE_SIZE, 'sensorFields': SENSORS, diff --git a/server/src/services/roverLearningService/workers/train.py b/server/src/services/roverLearningService/workers/train.py index 0f3f75ed..df533108 100644 --- a/server/src/services/roverLearningService/workers/train.py +++ b/server/src/services/roverLearningService/workers/train.py @@ -41,11 +41,11 @@ def main(): import torch from torch.nn import functional as functional from dataset import windows - from policy import Policy, VERSION, SLOTS, VALUES, specification, is_stop, select_slots + from policy import Policy, VERSION, SLOTS, VALUES, specification, is_stop, select_slots, initial_schema if args.check_dependencies: emit('dependencies', torch=str(torch.__version__), numpy=str(np.__version__), av=str(av.__version__), - parameters=sum(p.numel() for p in Policy().parameters()), specification=specification([])) + parameters=sum(p.numel() for p in Policy().parameters()), specification=specification(initial_schema())) return if not args.job: parser.error('--job is required unless --check-dependencies is used') @@ -72,8 +72,8 @@ def main(): optimizer = torch.optim.AdamW(model.parameters(), lr=0.0001) # The command-event policy has different inputs, targets and weights. Keep its # optimizer lineage separate; reuse recordings with their unchanged partition. - resume_path = root / 'training' / 'resume-v4.pt' - schema, steps, examples, last_published = [], 0, 0, 0 + resume_path = root / 'training' / 'resume-v5.pt' + schema, steps, examples, last_published = initial_schema(), 0, 0, 0 trained_rovers = set() control_profiles = [] distinct_windows = {} @@ -113,7 +113,7 @@ def main(): held_out = is_held_out(session_id) try: directory = root / 'recordings' / session_id - for sample in windows(directory, candidate_schema, cancelled, stats): + for sample in windows(directory, candidate_schema, cancelled, stats, extend=not held_out): seen += 1 # Keep evaluation naturally distributed. Training reserves space # for recorded commands and releases so periods without commands @@ -137,15 +137,15 @@ def main(): if cancelled(): break # Do not call a partially decoded recording validated. if not samples: + if held_out and stats.get('unsupportedCommandEvents'): + dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': 0, + 'skipReason': 'No supported aligned windows'}) + continue raise ValueError('No complete, aligned observation/action windows') # Entire sessions stay in one partition across all jobs. Neighboring # video windows must never straddle training and validation. held_out = is_held_out(session_id) if held_out: - if candidate_schema != schema: - dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': len(samples), - 'skipReason': 'Vocabulary absent from training data'}) - continue # Retry after training data introduces these command shapes. validation.extend(samples) else: schema = candidate_schema @@ -219,7 +219,12 @@ def main(): evaluation_started = time.monotonic() distinct_windows.update(observed_windows) unique_count = sum(distinct_windows.values()) - evaluation = {'status': 'waiting for held-out sessions', 'windows': len(validation)} + coverage = [row for row in dataset_stats if row.get('partition') == 'heldOut'] + unsupported_events = sum(row.get('unsupportedCommandEvents', 0) for row in coverage) + unsupported_windows = sum(row.get('unsupportedWindows', 0) for row in coverage) + evaluation = {'status': 'No supported held-out windows' if unsupported_events and not validation else 'waiting for held-out sessions', + 'windows': len(validation), 'unsupportedCommandEvents': unsupported_events, + 'unsupportedWindows': unsupported_windows} if validation and not cancelled(): model.eval() true_positive = predicted_positive = actual_positive = 0 @@ -276,6 +281,7 @@ def main(): held_out_sessions = sum(row.get('partition') == 'heldOut' and not row.get('skipReason') for row in dataset_stats) gate_reasons = [] for condition, reason in [ + (unsupported_events == 0, 'Held-out controls are not fully represented; see unsupportedCommands'), (evaluated == len(validation), 'Evaluation interrupted'), (held_out_sessions >= 2, 'Need two usable held-out recordings'), (evaluated >= 128, 'Need 128 held-out windows'), @@ -291,6 +297,7 @@ def main(): if not condition: gate_reasons.append(reason) evaluation = {'status': 'imitation gate not met' if gate_reasons else 'passed imitation gate', + 'unsupportedCommandEvents': unsupported_events, 'unsupportedWindows': unsupported_windows, 'method': 'Held-out 100ms command events; not a closed-loop rollout', 'threshold': job['actionThreshold'], 'heldOutSessions': held_out_sessions, 'windows': evaluated, 'precision': precision, 'recall': recall, diff --git a/webui/src/components/vip/VipRoverLearningCard.jsx b/webui/src/components/vip/VipRoverLearningCard.jsx index f7e7060d..935767bc 100644 --- a/webui/src/components/vip/VipRoverLearningCard.jsx +++ b/webui/src/components/vip/VipRoverLearningCard.jsx @@ -133,7 +133,7 @@ function LearningCard({ roverId }) {