diff --git a/perf/SPECTATE_PERFORMANCE_SUMMARY.md b/perf/SPECTATE_PERFORMANCE_SUMMARY.md new file mode 100644 index 00000000..b2a682ca --- /dev/null +++ b/perf/SPECTATE_PERFORMANCE_SUMMARY.md @@ -0,0 +1,147 @@ +# `/spectate` CPU Performance Summary + +This is the `/spectate` companion to the `/` performance writeups. The testing target +was the deployed page: + +```sh +https://rover.otter.land/spectate +``` + +The most useful artifact for this pass is: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +That run used: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +## High-Level Result + +`/spectate` is CPU-heavy for a different reason than `/`. + +The `/` page's biggest confirmed issue is global input listener churn. `/spectate` does +not show that same dominant pattern. The fresh `/spectate` audit had 379 event listeners, +mostly normal media/image listeners, and no giant add/remove loop. + +The `/spectate` hot path is instead a live-display fan-out problem: + +- several rover media players are mounted at once +- each rover card renders live HUD telemetry and a top-down SVG map +- camera/snapshot frames create new blobs and `img.src` values +- the route subscribes to broad session/log/chat state while also rendering the whole + spectator layout +- the page maintains several WebRTC players at the same time + +## Key Measurements + +Fresh `/spectate` runtime audit, 30s, mobile viewport, 6x CPU throttle: + +```txt +TaskDuration: 28.48s over a 30.9s sample +ScriptDuration: 12.11s +Long tasks: 81 +Worst long task: 375ms +Average frame gap: 113.9ms +p95 frame gap: 233.3ms +p99 frame gap: 300.0ms +Nodes: 7,059 +Layout objects: 6,150 +RTCPeerConnections: 6 +LayoutCount: 243 +RecalcStyleCount: 248 +``` + +The 8x CPU-throttled profile from the earlier route comparison was worse: + +```txt +Task delta: 49.13s over a 52.85s sample +Script delta: 19.58s +Layout delta: 516 +Recalc delta: 522 +Node delta: +3,815 +``` + +That profile also put `Blob`, `decodeString`, and `setAttribute` in the CPU stack, which +matches the runtime evidence for camera blobs, socket payload parsing, and SVG/image +attribute churn. + +## Socket And DOM Pressure + +The fresh audit captured these live events during the 30s window: + +```txt +sensorFrame: 1,354 events, 2,531,090 bytes +log:entry: 634 events, 150,179 bytes +unlabeled binary: 126 events, 11,048,477 bytes +roomCamera:frame: 117 events +session:sync: 54 events, 914,000 bytes +commandAck: 68 events +roverHostStats: 17 events +roverSnapshot:frame: 7 events +``` + +The most important DOM mutations were: + +```txt +path d: 2,688 mutations +path fill: 2,682 mutations +img.h-full.w-full src: 117 mutations +span.h-2.w-2 class: 117 mutations +``` + +The `path` mutations point strongly at the per-rover `TopDownMap` SVG overlays. The +`img.src` and blink class mutations point at room camera snapshot rendering. + +## Ordered Issue List + +1. [Telemetry fan-out and always-on HUD maps](./spectate-issues/001-telemetry-fanout-hud-maps.md) +2. [Multi-rover WebRTC media fan-out](./spectate-issues/002-multi-rover-webrtc-media-fanout.md) +3. [Room camera and snapshot object URL churn](./spectate-issues/003-room-camera-snapshot-object-url-churn.md) +4. [Broad spectator session rerenders](./spectate-issues/004-broad-spectator-session-rerenders.md) +5. [Shared log/session stream pressure](./spectate-issues/005-shared-log-session-stream-pressure.md) +6. [Spectator role handshake retries](./spectate-issues/006-spectator-role-handshake-retries.md) + +## Upstream Dependency Map + +Fixing the `/` backlog first is a good plan. + +Likely improved by `/` fixes: + +- telemetry visual throttling +- log batching/gating +- chat/nickname churn +- session selector/store improvements +- generic timer cleanup + +Not likely to be fixed by `/` alone: + +- several simultaneous WHEP video/audio players +- per-rover spectator HUD maps +- room camera snapshot blob/object URL churn +- spectator role/subscribe handshake behavior +- mobile policy for how many live feeds should be active at once + +## Suggested Re-Test + +After each `/spectate` fix, run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +The most useful numbers to compare: + +- `TaskDuration` +- `ScriptDuration` +- long task count and worst duration +- frame average/p95/p99 +- `RTCPeerConnections` +- `mutationTargets` for `path d`, `path fill`, and `img src` +- `webSocketEvents` for `sensorFrame`, `log:entry`, `session:sync`, and binary frame volume + diff --git a/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json b/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json new file mode 100644 index 00000000..9380bec9 --- /dev/null +++ b/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json @@ -0,0 +1,858 @@ +{ + "url": "https://rover.otter.land/spectate", + "runDir": "perf/results/2026-06-12T04-55-16-888Z-root-runtime", + "environment": { + "cpuThrottle": 6, + "viewport": { + "width": 390, + "height": 844 + }, + "isMobile": true, + "sampleMs": 30000 + }, + "metrics": { + "Timestamp": 29302.703615, + "AudioHandlers": 0, + "AudioWorkletProcessors": 0, + "Documents": 1, + "Frames": 1, + "JSEventListeners": 379, + "LayoutObjects": 6150, + "MediaKeySessions": 0, + "MediaKeys": 0, + "Nodes": 7059, + "Resources": 40, + "ContextLifecycleStateObservers": 21, + "V8PerContextDatas": 2, + "WorkerGlobalScopes": 0, + "UACSSResources": 0, + "RTCPeerConnections": 6, + "ResourceFetchers": 1, + "AdSubframes": 0, + "DetachedScriptStates": 0, + "ArrayBufferContents": 28, + "LayoutCount": 243, + "RecalcStyleCount": 248, + "LayoutDuration": 0.840302, + "RecalcStyleDuration": 1.699961, + "DevToolsCommandDuration": 0.000155, + "ScriptDuration": 12.114687, + "V8CompileDuration": 0, + "TaskDuration": 28.48339, + "TaskOtherDuration": 13.828285, + "ThreadTime": 30.190528, + "ProcessTime": 39.47, + "JSHeapUsedSize": 22447504, + "JSHeapTotalSize": 42467328, + "FirstMeaningfulPaint": 0, + "DomContentLoaded": 29272.696156, + "NavigationStart": 29272.010802 + }, + "snapshot": { + "elapsedMs": 30914.699999999255, + "listeners": [ + { + "key": "add:audio:canplay", + "count": 9 + }, + { + "key": "add:audio:ended", + "count": 9 + }, + { + "key": "add:audio:error", + "count": 9 + }, + { + "key": "add:audio:pause", + "count": 9 + }, + { + "key": "add:audio:play", + "count": 9 + }, + { + "key": "add:audio:stalled", + "count": 9 + }, + { + "key": "add:audio:waiting", + "count": 9 + }, + { + "key": "add:window:online", + "count": 4 + }, + { + "key": "add:document:visibilitychange", + "count": 4 + }, + { + "key": "add:img:error", + "count": 4 + }, + { + "key": "add:img:load", + "count": 4 + }, + { + "key": "add:input:invalid", + "count": 3 + }, + { + "key": "remove:window:online", + "count": 3 + }, + { + "key": "remove:document:visibilitychange", + "count": 3 + }, + { + "key": "add:audio:abort", + "count": 3 + }, + { + "key": "add:audio:canplaythrough", + "count": 3 + }, + { + "key": "add:audio:durationchange", + "count": 3 + }, + { + "key": "add:audio:emptied", + "count": 3 + }, + { + "key": "add:audio:encrypted", + "count": 3 + }, + { + "key": "add:audio:loadeddata", + "count": 3 + }, + { + "key": "add:audio:loadedmetadata", + "count": 3 + }, + { + "key": "add:audio:loadstart", + "count": 3 + }, + { + "key": "add:audio:playing", + "count": 3 + }, + { + "key": "add:audio:progress", + "count": 3 + }, + { + "key": "add:audio:ratechange", + "count": 3 + }, + { + "key": "add:audio:resize", + "count": 3 + }, + { + "key": "add:audio:seeked", + "count": 3 + }, + { + "key": "add:audio:seeking", + "count": 3 + }, + { + "key": "add:audio:suspend", + "count": 3 + }, + { + "key": "add:audio:timeupdate", + "count": 3 + }, + { + "key": "add:audio:volumechange", + "count": 3 + }, + { + "key": "add:video:abort", + "count": 3 + }, + { + "key": "add:video:canplay", + "count": 3 + }, + { + "key": "add:video:canplaythrough", + "count": 3 + }, + { + "key": "add:video:durationchange", + "count": 3 + }, + { + "key": "add:video:emptied", + "count": 3 + }, + { + "key": "add:video:encrypted", + "count": 3 + }, + { + "key": "add:video:ended", + "count": 3 + }, + { + "key": "add:video:error", + "count": 3 + }, + { + "key": "add:video:loadeddata", + "count": 3 + } + ], + "timers": [ + { + "key": "interval:3000", + "count": 12 + }, + { + "key": "timeout:12000", + "count": 6 + }, + { + "key": "timeout:1200", + "count": 4 + }, + { + "key": "interval:60000", + "count": 3 + }, + { + "key": "timeout:0", + "count": 3 + }, + { + "key": "interval:200", + "count": 2 + }, + { + "key": "timeout:15000", + "count": 1 + }, + { + "key": "interval:2000", + "count": 1 + } + ], + "timeouts": 14, + "intervals": 18, + "timeoutFires": 7, + "intervalFires": 112, + "webSocketMessageCount": 1000, + "webSocketBytes": 4207474, + "webSocketEvents": [ + { + "event": "sensorFrame", + "count": 1354, + "bytes": 2531090 + }, + { + "event": "log:entry", + "count": 634, + "bytes": 150179 + }, + { + "event": "", + "count": 126, + "bytes": 11048477 + }, + { + "event": "roomCamera:frame", + "count": 117, + "bytes": 10413 + }, + { + "event": "commandAck", + "count": 68, + "bytes": 6412 + }, + { + "event": "session:sync", + "count": 54, + "bytes": 914000 + }, + { + "event": "roverHostStats", + "count": 17, + "bytes": 13058 + }, + { + "event": "roverSnapshot:frame", + "count": 7, + "bytes": 682 + }, + { + "event": "2", + "count": 5, + "bytes": 5 + }, + { + "event": "auth:role", + "count": 4, + "bytes": 139 + }, + { + "event": "roomCamera:status", + "count": 4, + "bytes": 284 + }, + { + "event": "roverSnapshot:status", + "count": 3, + "bytes": 238 + }, + { + "event": "alert", + "count": 2, + "bytes": 256 + }, + { + "event": "alert:new", + "count": 2, + "bytes": 330 + }, + { + "event": "0{\"sid\":\"3a5", + "count": 1, + "bytes": 106 + }, + { + "event": "40{\"sid\":\"lE", + "count": 1, + "bytes": 32 + }, + { + "event": "mode", + "count": 1, + "bytes": 27 + }, + { + "event": "controlGranted", + "count": 1, + "bytes": 41 + }, + { + "event": "rovers", + "count": 1, + "bytes": 5485 + }, + { + "event": "chat:init", + "count": 1, + "bytes": 77415 + }, + { + "event": "overseer:memory", + "count": 1, + "bytes": 520 + }, + { + "event": "log:init", + "count": 1, + "bytes": 44530 + }, + { + "event": "barcode:state", + "count": 1, + "bytes": 1627 + }, + { + "event": "kinect:pointCloudFrame", + "count": 1, + "bytes": 262 + }, + { + "event": "kinect:colorFrame", + "count": 1, + "bytes": 169 + }, + { + "event": "430[{\"succes", + "count": 1, + "bytes": 40 + }, + { + "event": "431[{\"succes", + "count": 1, + "bytes": 41 + }, + { + "event": "432[{\"succes", + "count": 1, + "bytes": 194 + }, + { + "event": "433[{\"succes", + "count": 1, + "bytes": 40 + }, + { + "event": "434[{\"url\":\"", + "count": 1, + "bytes": 133 + }, + { + "event": "435[{\"url\":\"", + "count": 1, + "bytes": 145 + }, + { + "event": "WALL-E", + "count": 1, + "bytes": 40 + }, + { + "event": "437[{\"url\":\"", + "count": 1, + "bytes": 133 + }, + { + "event": "438[{\"url\":\"", + "count": 1, + "bytes": 145 + }, + { + "event": "Freaky", + "count": 1, + "bytes": 40 + }, + { + "event": "4310[{\"url\":", + "count": 1, + "bytes": 136 + }, + { + "event": "4311[{\"url\":", + "count": 1, + "bytes": 148 + }, + { + "event": "Bweeble", + "count": 1, + "bytes": 42 + }, + { + "event": "carpet", + "count": 1, + "bytes": 68 + }, + { + "event": "4314[{\"succe", + "count": 1, + "bytes": 41 + }, + { + "event": "4315[{\"succe", + "count": 1, + "bytes": 22 + }, + { + "event": "4316[{\"succe", + "count": 1, + "bytes": 22 + }, + { + "event": "4317[{\"succe", + "count": 1, + "bytes": 195 + }, + { + "event": "4318[{\"succe", + "count": 1, + "bytes": 22 + }, + { + "event": "4319[{\"succe", + "count": 1, + "bytes": 22 + } + ], + "mutationsTotal": { + "records": 5868, + "added": 440, + "removed": 440, + "attrs": 4921, + "text": 67 + }, + "mutationTargets": [ + { + "key": "path.[object.SVGAnimatedString]:attributes:d", + "count": 2688 + }, + { + "key": "path.[object.SVGAnimatedString]:attributes:fill", + "count": 2682 + }, + { + "key": "Text:characterData:", + "count": 164 + }, + { + "key": "img.h-full.w-full:attributes:src", + "count": 117 + }, + { + "key": "span.h-2.w-2:attributes:class", + "count": 117 + }, + { + "key": "input.field-input.flex-1:attributes:name", + "count": 102 + }, + { + "key": "input.field-input.chat-composer-input:attributes:name", + "count": 102 + }, + { + "key": "path.[object.SVGAnimatedString]:attributes:stroke", + "count": 18 + }, + { + "key": "section.panel-section.border:attributes:style", + "count": 11 + }, + { + "key": "span.rounded.px-1.5:attributes:class", + "count": 9 + }, + { + "key": "header.flex.items-center:attributes:style", + "count": 8 + }, + { + "key": "article.surface.px-0.5:attributes:class", + "count": 8 + }, + { + "key": "li.surface.flex:attributes:class", + "count": 3 + }, + { + "key": "video.h-full.w-full:attributes:playsinline", + "count": 3 + }, + { + "key": "video.h-full.w-full:attributes:autoplay", + "count": 3 + }, + { + "key": "video.h-full.w-full:attributes:disableremoteplayback", + "count": 3 + }, + { + "key": "audio:attributes:autoplay", + "count": 3 + }, + { + "key": "audio:attributes:disableremoteplayback", + "count": 3 + }, + { + "key": "input.accent-cyan-500:attributes:name", + "count": 2 + }, + { + "key": "div.panel-section.flex:attributes:style", + "count": 2 + }, + { + "key": "span.rounded.px-1:attributes:class", + "count": 2 + }, + { + "key": "input.accent-cyan-500:attributes:type", + "count": 1 + }, + { + "key": "input.field-input.flex-1:attributes:value", + "count": 1 + } + ], + "mutationBursts": [ + { + "at": 13575, + "records": 119, + "added": 57, + "removed": 58, + "attrs": 4, + "text": 0 + }, + { + "at": 18089, + "records": 117, + "added": 52, + "removed": 53, + "attrs": 12, + "text": 0 + }, + { + "at": 27878, + "records": 113, + "added": 52, + "removed": 52, + "attrs": 8, + "text": 1 + }, + { + "at": 22882, + "records": 102, + "added": 49, + "removed": 49, + "attrs": 4, + "text": 0 + }, + { + "at": 9538, + "records": 100, + "added": 48, + "removed": 48, + "attrs": 4, + "text": 0 + }, + { + "at": 14782, + "records": 84, + "added": 37, + "removed": 35, + "attrs": 12, + "text": 0 + }, + { + "at": 15222, + "records": 83, + "added": 39, + "removed": 39, + "attrs": 5, + "text": 0 + }, + { + "at": 9811, + "records": 68, + "added": 28, + "removed": 28, + "attrs": 12, + "text": 0 + }, + { + "at": 13733, + "records": 36, + "added": 13, + "removed": 13, + "attrs": 10, + "text": 0 + }, + { + "at": 19439, + "records": 26, + "added": 10, + "removed": 10, + "attrs": 6, + "text": 0 + }, + { + "at": 23034, + "records": 18, + "added": 3, + "removed": 3, + "attrs": 12, + "text": 0 + }, + { + "at": 23113, + "records": 18, + "added": 4, + "removed": 4, + "attrs": 10, + "text": 0 + }, + { + "at": 28037, + "records": 18, + "added": 4, + "removed": 4, + "attrs": 10, + "text": 0 + }, + { + "at": 10169, + "records": 16, + "added": 4, + "removed": 4, + "attrs": 8, + "text": 0 + }, + { + "at": 18258, + "records": 16, + "added": 4, + "removed": 4, + "attrs": 8, + "text": 0 + }, + { + "at": 13903, + "records": 15, + "added": 4, + "removed": 4, + "attrs": 7, + "text": 0 + }, + { + "at": 14108, + "records": 15, + "added": 2, + "removed": 2, + "attrs": 10, + "text": 1 + }, + { + "at": 16376, + "records": 15, + "added": 2, + "removed": 2, + "attrs": 11, + "text": 0 + }, + { + "at": 11700, + "records": 14, + "added": 0, + "removed": 0, + "attrs": 14, + "text": 0 + }, + { + "at": 13012, + "records": 14, + "added": 0, + "removed": 0, + "attrs": 13, + "text": 1 + } + ], + "frames": { + "count": 264, + "average": 113.8840909090909, + "p95": 233.29999999999927, + "p99": 300 + }, + "longTasks": { + "count": 81, + "top": [ + { + "startTime": 311.5999999977648, + "duration": 375, + "name": "self" + }, + { + "startTime": 1978.699999999255, + "duration": 299, + "name": "self" + }, + { + "startTime": 1254.5999999977648, + "duration": 229, + "name": "self" + }, + { + "startTime": 1772, + "duration": 206, + "name": "self" + }, + { + "startTime": 1485.3999999985099, + "duration": 170, + "name": "self" + }, + { + "startTime": 2646.800000000745, + "duration": 165, + "name": "self" + }, + { + "startTime": 2507.5, + "duration": 139, + "name": "self" + }, + { + "startTime": 693.8000000007451, + "duration": 125, + "name": "self" + }, + { + "startTime": 3026.800000000745, + "duration": 119, + "name": "self" + }, + { + "startTime": 4148.39999999851, + "duration": 113, + "name": "self" + }, + { + "startTime": 3461.099999997765, + "duration": 109, + "name": "self" + }, + { + "startTime": 9812.89999999851, + "duration": 106, + "name": "self" + }, + { + "startTime": 14821.39999999851, + "duration": 100, + "name": "self" + }, + { + "startTime": 3145.800000000745, + "duration": 97, + "name": "self" + }, + { + "startTime": 4447.39999999851, + "duration": 96, + "name": "self" + }, + { + "startTime": 9551, + "duration": 96, + "name": "self" + }, + { + "startTime": 9939.300000000745, + "duration": 95, + "name": "self" + }, + { + "startTime": 18100.89999999851, + "duration": 92, + "name": "self" + }, + { + "startTime": 23114.89999999851, + "duration": 92, + "name": "self" + }, + { + "startTime": 18302.800000000745, + "duration": 88, + "name": "self" + } + ] + }, + "consoleErrors": [] + } +} \ No newline at end of file diff --git a/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime.png b/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime.png new file mode 100644 index 00000000..c2b0179f Binary files /dev/null and b/perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime.png differ diff --git a/perf/spectate-issues/001-telemetry-fanout-hud-maps.md b/perf/spectate-issues/001-telemetry-fanout-hud-maps.md new file mode 100644 index 00000000..2d494190 --- /dev/null +++ b/perf/spectate-issues/001-telemetry-fanout-hud-maps.md @@ -0,0 +1,219 @@ +# Issue 001: Telemetry Fan-Out And Always-On HUD Maps + +## Summary + +`/spectate` renders multiple rover cards at once, and every rover card includes a live HUD +with an always-on top-down SVG map. During the 30s throttled audit, the page received +1,354 `sensorFrame` events and produced more than 5,300 SVG `path` attribute mutations. + +This is the strongest measured `/spectate` CPU problem. + +## Severity + +High. + +This is high-frequency, always-on work. It scales with the number of visible rovers and +hits exactly the devices we care about: mobile browsers and weaker computers. + +## Upstream Likelihood + +Medium. + +The `/` telemetry issue will likely help if it introduces a central throttled visual +telemetry path. However, `/spectate` has a unique multiplier: it renders several rover +HUDs and several `TopDownMap` instances at the same time. Even after upstream telemetry +throttling, `/spectate` probably still needs a policy for how many maps update and how +often. + +Related upstream issue: + +```txt +perf/issues/004-sensor-telemetry-render-frequency.md +``` + +## Affected Files + +- `webui/src/components/SpectateVideo/index.jsx` +- `webui/src/components/HudOverlays/HudOverlay/index.jsx` +- `webui/src/components/HudOverlays/HudOverlay/HudMapOverlay.jsx` +- `webui/src/components/TopDownMap/TopDownMapContent.jsx` +- `webui/src/components/TopDownMap/visuals.jsx` +- `webui/src/context/TelemetryContext.jsx` +- `webui/src/components/RoverMediaPlayer/index.jsx` +- `webui/src/components/HudOverlays/OvercurrentOverlay/index.jsx` +- `webui/src/components/HudOverlays/LowBatteryOverlay/index.jsx` +- `webui/src/components/HudOverlays/VerticalBatteryOverlay/index.jsx` + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +30s sample, mobile viewport, 6x CPU throttle: + +```txt +sensorFrame events: 1,354 +sensorFrame bytes: 2,531,090 +path d mutations: 2,688 +path fill mutations: 2,682 +Long tasks: 81 +Worst long task: 375ms +Average frame gap: 113.9ms +p95 frame gap: 233.3ms +``` + +The path mutation pattern is the important clue. Normal text/status updates do not mutate +thousands of SVG path `d` and `fill` attributes. `TopDownMap` does. + +## Current Code Path + +`SpectateVideo` mounts a full media and overlay stack for each rover: + +```txt +webui/src/components/SpectateVideo/index.jsx:17 +RoverMediaPlayer + +webui/src/components/SpectateVideo/index.jsx:26 +HudOverlay variant="spectator" + +webui/src/components/SpectateVideo/index.jsx:33 +OvercurrentOverlay + +webui/src/components/SpectateVideo/index.jsx:35 +VerticalBatteryOverlay +``` + +`HudOverlay` subscribes to telemetry: + +```txt +webui/src/components/HudOverlays/HudOverlay/index.jsx:27 +const frame = useTelemetryFrame(effectiveRoverId); +``` + +For spectator mode, it forces the top-down map on: + +```txt +webui/src/components/HudOverlays/HudOverlay/index.jsx:50 +variant === 'spectator' ? true : ... +``` + +Then `HudMapOverlay` renders: + +```txt +webui/src/components/HudOverlays/HudOverlay/HudMapOverlay.jsx:35 + +``` + +There is also duplicated telemetry subscription pressure. `RoverMediaPlayer` subscribes to +telemetry for audio ducking and media state, while `HudOverlay`, battery overlays, and +warning overlays can subscribe for the same rover. + +## Why This Matters + +The raw socket message rate is not automatically a problem. The expensive part is waking +multiple React subscribers, recomputing visual sensor geometry, and mutating SVG +attributes for every visual update. + +On `/`, there is generally one primary driver view. On `/spectate`, the same sensor stream +can be multiplied across several rover cards. + +## Fix Strategies + +### Option A: Add A Throttled Visual Telemetry Channel + +This is the most reusable upstream-friendly fix. + +Keep `TelemetryContext` storing the latest frame immediately, but notify visual consumers +at a lower cadence: + +```txt +useTelemetryFrameRaw(roverId) high-frequency, only where truly needed +useTelemetryFrameVisual(roverId) throttled display data +``` + +Use visual telemetry for: + +- `TopDownMap` +- `HudOverlay` +- `SpectatorTelemetryOverlay` +- battery overlays +- warning overlays where 100-200ms delay is acceptable + +Suggested rates: + +```txt +desktop visual telemetry: 10-15Hz +mobile visual telemetry: 5-10Hz +``` + +### Option B: Pass One Telemetry Frame Through `SpectateVideo` + +Right now, each overlay can subscribe independently. Instead, let `SpectateVideo` subscribe +once per rover and pass `sensors` down: + +```jsx +const frame = useTelemetryFrameVisual(roverId); +const sensors = frame?.sensors ?? null; + + + + + + +``` + +This may require small prop additions to the warning/battery overlays. + +### Option C: Make Spectator HUD Maps Adaptive + +Spectator maps are currently always on. For low-end/mobile: + +- hide maps by default on portrait mobile +- update maps at a lower cadence than text warnings +- only animate the focused/first rover's map +- show static/minimal maps for secondary rovers +- make map visibility a spectator setting + +### Option D: Memoize TopDownMap Geometry + +This is useful after throttling. Avoid recalculating path data when the sensor fields that +drive a particular path did not change. + +## Recommended Path + +Do this after the `/` telemetry issue if that work touches `TelemetryContext`. + +Best first implementation: + +1. Add `useTelemetryFrameVisual(roverId)` with a 100-150ms visual notification cadence. +2. Switch `HudOverlay` and `TopDownMap` consumers to visual telemetry. +3. In `SpectateVideo`, subscribe once and pass sensors down where practical. +4. Add a spectator/mobile map policy so only the most important maps update live. + +## Validation + +Run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- `path d` and `path fill` mutations should drop sharply. +- `ScriptDuration` should drop. +- Long task count should drop. +- Frame p95/p99 should improve. +- `sensorFrame` socket count may stay the same if only rendering is throttled. + +## Risks + +- Do not throttle control/safety logic that needs immediate data. +- Keep battery/overcurrent warnings responsive enough to be trusted. +- If map updates become too slow, operators may perceive the HUD as stale. A 5-10Hz map is + usually enough for spectators, but verify by watching the live page. + diff --git a/perf/spectate-issues/002-multi-rover-webrtc-media-fanout.md b/perf/spectate-issues/002-multi-rover-webrtc-media-fanout.md new file mode 100644 index 00000000..a8656899 --- /dev/null +++ b/perf/spectate-issues/002-multi-rover-webrtc-media-fanout.md @@ -0,0 +1,190 @@ +# Issue 002: Multi-Rover WebRTC Media Fan-Out + +## Summary + +`/spectate` mounts several `RoverMediaPlayer` instances at once. In the fresh runtime +audit, Chromium reported 6 active `RTCPeerConnections`. That is a large baseline for +low-end clients, especially while the page is also handling telemetry, snapshots, logs, +chat, and SVG HUD maps. + +## Severity + +High. + +This may not show up as React mutations, but it consumes CPU in browser media pipelines, +WebRTC negotiation, decoding, audio handling, and event dispatch. It also scales with the +number of rovers. + +## Upstream Likelihood + +Low to medium. + +General `RoverMediaPlayer` improvements can help both `/` and `/spectate`, but the core +problem is spectate-specific: `/spectate` intentionally displays multiple rover media +players at once. The fix needs a spectator policy for active feeds. + +## Affected Files + +- `webui/src/components/SpectateVideo/index.jsx` +- `webui/src/components/RoverMediaPlayer/index.jsx` +- `webui/src/hooks/useVideoRequests.js` +- `webui/src/lib/whepPlayer.js` +- `webui/src/spectate/SpectatorApp/components/RoverRow.jsx` +- server-side WHEP/video request handling, if feed quality/rate selection is added + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +30s sample: + +```txt +RTCPeerConnections: 6 +JSEventListeners: 379 +``` + +The listener list is dominated by normal media listeners: + +```txt +audio canplay/ended/error/pause/play/stalled/waiting: 9 each +video media events: 3 each +``` + +That pattern fits several rover media players, likely with video and dedicated audio +connections. + +The same audit also saw: + +```txt +TaskDuration: 28.48s +ScriptDuration: 12.11s +Long tasks: 81 +``` + +The earlier 8x throttled `/spectate` profile saw: + +```txt +Task delta: 49.13s +Script delta: 19.58s +``` + +## Current Code Path + +Each rover card mounts: + +```txt +webui/src/components/SpectateVideo/index.jsx:17 + +``` + +`RoverMediaPlayer` automatically creates video and audio request entries: + +```txt +webui/src/components/RoverMediaPlayer/index.jsx +autoEntries = [ + video rover entry, + optional dedicated audio entry, +] +``` + +It then requests sources through `useVideoRequests`, starts WHEP players, maintains +restart timers, maintains unmute/audio retry timers, and optionally falls back to +snapshots when no WHEP URL exists. + +## Why This Matters + +Six peer connections on a powerful desktop may feel fine. On weak laptops and mobile +devices, simultaneous WebRTC decode plus React UI churn can easily saturate the main +thread or media threads. The visible symptom is not necessarily one obvious function in +the CPU profile; it is degraded frame cadence, long tasks, and high browser-level media +work. + +## Fix Strategies + +### Option A: Cap Active Live Feeds On Mobile + +For mobile/low-end layouts: + +- one primary live WHEP feed +- secondary rover cards use snapshots or lower-rate previews +- promote a secondary rover to live only when selected/visible/focused + +This is the highest-impact spectator-specific design change. + +### Option B: Pause Offscreen Feeds + +Use `IntersectionObserver` around each `SpectateVideo` or `RoverSpectatorCard`: + +- start WHEP when the card is visible +- stop WHEP when the card is far offscreen +- keep a small grace period to avoid flapping while scrolling + +This matters especially because portrait `/spectate` can scroll. + +### Option C: Disable Dedicated Audio For Non-Focused Rovers + +Dedicated audio is useful, but not necessarily for every rover at the same time. + +Possible policy: + +- audio enabled only for the selected/primary rover +- audio disabled for muted secondary rovers +- audio WHEP starts only after user interaction + +This should reduce peer connections and audio retry timers. + +### Option D: Request Lower Quality For Spectator Secondary Feeds + +If the server/media stack supports it, add spectator quality tiers: + +```txt +primary: normal FPS/resolution +secondary: low FPS/resolution or snapshot-only +mobile: lower default bitrate/resolution +``` + +### Option E: Stop Retry Loops When Hidden + +When document visibility is hidden or a card is offscreen: + +- stop restart timers +- stop audio retry intervals +- avoid requesting fresh WHEP URLs + +## Recommended Path + +After the HUD/telemetry issue, add a spectator feed activity policy: + +1. Define "primary" versus "secondary" rover cards. +2. Keep only primary feeds fully live on mobile. +3. Use `IntersectionObserver` to avoid live playback for offscreen cards. +4. Disable secondary audio by default. + +## Validation + +Run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- `RTCPeerConnections` should drop below 6 on mobile. +- Media listener counts should drop. +- `TaskDuration` should drop. +- Frame p95/p99 should improve. +- User-visible video should still start reliably for the primary rover. + +## Risks + +- Spectators may expect all feeds live all the time on desktop. +- Switching feeds can introduce startup delay if WHEP is torn down too aggressively. +- Audio policy needs to avoid surprising users. Make the focused rover's audio behavior + clear and consistent. + diff --git a/perf/spectate-issues/003-room-camera-snapshot-object-url-churn.md b/perf/spectate-issues/003-room-camera-snapshot-object-url-churn.md new file mode 100644 index 00000000..04c884e2 --- /dev/null +++ b/perf/spectate-issues/003-room-camera-snapshot-object-url-churn.md @@ -0,0 +1,202 @@ +# Issue 003: Room Camera And Snapshot Object URL Churn + +## Summary + +Room camera and rover snapshot frames are delivered as binary payloads. The client creates +a new `Blob`, creates a new object URL, revokes the previous URL, and updates React state +for every frame. In the 30s `/spectate` audit, this caused 117 `img.src` mutations and +11MB of binary socket payloads. + +## Severity + +Medium-high. + +This is not the biggest measured CPU source, but it is steady work and very visible on +low-end devices because JPEG decoding, Blob allocation, object URL churn, image source +updates, and React state updates all happen together. + +## Upstream Likelihood + +Low. + +This is mostly `/spectate` and camera-panel specific. It will not be fixed by the `/` +input, control context, or log work. It may benefit indirectly if broad rerenders are +reduced, but the frame handling itself needs targeted changes. + +## Affected Files + +- `webui/src/hooks/useRoomCameraSnapshots.js` +- `webui/src/hooks/useRoverSnapshots.js` +- `webui/src/components/RoomCameraPanel/index.jsx` +- `webui/src/components/RoomCameraFeed/index.jsx` +- `server/src/services/roomCameraService/socketGateway.js` +- `server/src/services/roverSnapshotService/socketGateway.js` + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +30s sample: + +```txt +unlabeled binary socket events: 126 +unlabeled binary bytes: 11,048,477 +roomCamera:frame events: 117 +roverSnapshot:frame events: 7 +img src mutations: 117 +blink class mutations: 117 +``` + +The 8x throttled profile also had these CPU entries: + +```txt +Blob +decodeString +setAttribute +``` + +Those line up with binary socket parsing, Blob/object URL creation, and `img.src` updates. + +## Current Code Path + +`useRoomCameraSnapshots` handles each frame: + +```txt +webui/src/hooks/useRoomCameraSnapshots.js:78 +const blob = new Blob([buffer], { type: 'image/jpeg' }); + +webui/src/hooks/useRoomCameraSnapshots.js:79 +const url = URL.createObjectURL(blob); + +webui/src/hooks/useRoomCameraSnapshots.js:82 +URL.revokeObjectURL(prevUrl); + +webui/src/hooks/useRoomCameraSnapshots.js:85 +setFeeds(...) +``` + +`useRoverSnapshots` has the same pattern. + +`RoomCameraFeed` writes that object URL into an image: + +```txt +webui/src/components/RoomCameraFeed/index.jsx:24 + +``` + +It also toggles blink state for each frame: + +```txt +webui/src/components/RoomCameraFeed/index.jsx:12 +setBlink((prev) => !prev); +``` + +Server-side frame rates: + +```txt +server/src/services/roomCameraService/socketGateway.js:11 +STREAM_INTERVAL_MS = 1000 + +server/src/services/roverSnapshotService/socketGateway.js:11 +STREAM_INTERVAL_MS = 333 +``` + +## Why This Matters + +The page is doing memory allocation and DOM/image decode work on every camera frame. At +one room camera per second this can be acceptable, but `/spectate` is already busy. When +combined with live WebRTC and telemetry maps, this becomes another steady CPU drain. + +The blink dot is also doing one React state update and class mutation per frame. It is +small, but it is pure extra work. + +## Fix Strategies + +### Option A: Drop Frames When The Client Is Behind + +Keep only the latest pending frame per camera. If the previous object URL has not been +painted or decoded yet, replace the pending buffer instead of forcing every frame through +React. + +### Option B: Throttle Display Updates On Mobile + +Render room camera frames at a lower client-side display rate: + +```txt +desktop: 1fps as today, or configured +mobile: 0.2-0.5fps for secondary room cameras +``` + +The server can still emit at its existing rate for other clients, but this client can +choose not to render every frame. + +### Option C: Lazy Mount Or Collapse Room Cameras On Mobile + +On portrait mobile, `SecondaryRow` can defer `RoomCameraPanel` until: + +- the user scrolls near it +- the user opens the panel +- the page is idle after initial rover feeds have settled + +### Option D: Remove Per-Frame React Blink State + +Replace the blink toggle with CSS animation or a timestamp updated less often. + +For example: + +- keep status text +- use a CSS pulse class while status is `playing` +- avoid toggling React state on every frame + +### Option E: Stabilize Room Camera Source Lists + +`RoomCameraPanel` currently passes a newly created array of objects into the snapshot hook: + +```txt +webui/src/components/RoomCameraPanel/index.jsx:37 +useRoomCameraSnapshots(cameras.map((camera) => ({ id: camera.id }))) +``` + +The hook ultimately keys by IDs, so this is not the main issue, but it still creates extra +memo/effect churn on rerender. Prefer a memoized array of IDs: + +```jsx +const cameraIds = useMemo(() => cameras.map((camera) => camera.id), [cameras]); +const feedMap = useRoomCameraSnapshots(cameraIds); +``` + +## Recommended Path + +Start with the cheap fixes: + +1. Remove the per-frame blink state. +2. Memoize camera IDs in `RoomCameraPanel`. +3. Add client-side render throttling/drop-latest behavior in `useRoomCameraSnapshots`. +4. Consider lazy mounting/collapsing room cameras in mobile spectator layout. + +## Validation + +Run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- `img.h-full.w-full:attributes:src` mutations should drop if display is throttled. +- `span.h-2.w-2:attributes:class` mutations should disappear if blink state is removed. +- `ScriptDuration` and long tasks should improve modestly. +- Binary socket bytes may stay the same unless server subscription/rate is changed. + +## Risks + +- Reducing room camera update rate can make the room view feel stale. +- Object URL lifecycle must stay correct. Do not leak old URLs. +- If lazy mounting unsubscribes too aggressively, first frame display may feel delayed. + diff --git a/perf/spectate-issues/004-broad-spectator-session-rerenders.md b/perf/spectate-issues/004-broad-spectator-session-rerenders.md new file mode 100644 index 00000000..fb447a8c --- /dev/null +++ b/perf/spectate-issues/004-broad-spectator-session-rerenders.md @@ -0,0 +1,202 @@ +# Issue 004: Broad Spectator Session Rerenders + +## Summary + +`SpectatorContent` reads the whole session object with `useSession()`. That means broad +session changes can rerender the top-level spectator layout, including the sidebar, rover +row, secondary row, chat, logs, room cameras, and overlays. + +The audit saw 54 `session:sync` events in 30s, carrying about 914KB, while the DOM also +showed repeated added/removed-node bursts. + +## Severity + +Medium-high. + +This is likely a multiplier. It may not be the single hottest path, but it can cause +otherwise independent components to rerender together. + +## Upstream Likelihood + +Medium to high. + +If the `/` backlog changes `SessionContext` to provide better selectors, better structural +sharing, batched log/session updates, or more stable action references, this issue may be +partly fixed upstream. However, `SpectatorContent` itself still needs to stop reading the +whole session object. + +Related upstream issues: + +```txt +perf/issues/003-high-volume-log-stream.md +perf/issues/006-chat-and-nickname-churn.md +perf/issues/007-timers-and-polling.md +``` + +## Affected Files + +- `webui/src/spectate/SpectatorApp/SpectatorContent.jsx` +- `webui/src/hooks/useSpectatorMode.js` +- `webui/src/context/SessionContext.jsx` +- `webui/src/spectate/SpectatorApp/components/RoverRow.jsx` +- `webui/src/spectate/SpectatorApp/components/SecondaryRow.jsx` +- sidebar components mounted by `SpectatorContent` + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +30s sample: + +```txt +session:sync events: 54 +session:sync bytes: 914,000 +DOM mutation records: 5,868 +DOM nodes added: 440 +DOM nodes removed: 440 +``` + +Largest mutation bursts: + +```txt +119 records, 57 added, 58 removed +117 records, 52 added, 53 removed +113 records, 52 added, 52 removed +102 records, 49 added, 49 removed +100 records, 48 added, 48 removed +``` + +Those bursts suggest periodic component/list updates, not just single text changes. + +## Current Code Path + +`SpectatorContent` subscribes to the whole session: + +```txt +webui/src/spectate/SpectatorApp/SpectatorContent.jsx:22 +const { session } = useSession(); +``` + +It then derives: + +```txt +inLockdown = session?.mode === 'lockdown' +roster = session?.roster ?? [] +``` + +But because it reads the whole session object, changes to logs, users, active drivers, +room cameras, replay state, chat-related session fields, or any other session branch can +potentially re-render the whole spectator layout. + +`useSpectatorMode` also reads the whole session: + +```txt +webui/src/hooks/useSpectatorMode.js:7 +const { session, setRole, subscribeAll, connected } = useSession(); +``` + +It only needs `mode`, `role`, `connected`, and actions. + +## Why This Matters + +`/spectate` is already rendering expensive children. A broad rerender at the top makes +other optimizations less effective because unrelated live data can still wake the route. + +For example: + +- a log entry should not rerender the rover media grid +- a chat composer state change should not rerender room camera feeds +- a session sync should not recreate layout props unless the selected fields changed + +## Fix Strategies + +### Option A: Replace Whole Session Reads With Selectors + +In `SpectatorContent`, use precise selectors: + +```jsx +const mode = useSessionSelector((state) => state.session?.mode ?? null); +const roster = useSessionSelector((state) => state.session?.roster ?? EMPTY_ROSTER); +``` + +Then: + +```jsx +const inLockdown = mode === 'lockdown'; +``` + +Make sure selector outputs are stable. If `roster` is rebuilt on every `session:sync`, +this still rerenders. The store may need structural sharing upstream. + +### Option B: Split Spectator Layout Into Memoized Regions + +Split the page into: + +- spectator sidebar +- rover grid +- secondary/camera row +- global overlays + +Then each region subscribes only to what it needs. + +### Option C: Narrow `useSpectatorMode` + +Replace: + +```jsx +const { session, setRole, subscribeAll, connected } = useSession(); +``` + +with selectors/actions: + +```jsx +const mode = useSessionSelector((state) => state.session?.mode ?? null); +const role = useSessionSelector((state) => state.session?.role ?? null); +const connected = useSessionSelector((state) => state.connected); +const { setRole, subscribeAll } = useSessionActions(); +``` + +### Option D: Batch Or Diff `session:sync` + +This may belong to the `/` backlog. If incoming sync payloads replace large object +branches each time, selector users will still rerender. Preserve references for unchanged +branches. + +## Recommended Path + +Do this after the `/` session/log work if that work changes `SessionContext`. + +Local `/spectate` changes: + +1. Remove whole-session `useSession()` from `SpectatorContent`. +2. Remove whole-session `useSession()` from `useSpectatorMode`. +3. Split the layout so rover media rows do not depend on sidebar data. +4. Re-test mutation bursts and React commit behavior. + +## Validation + +Run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- Fewer added/removed DOM mutation bursts. +- Lower `ScriptDuration`. +- Possibly lower `RecalcStyleCount` and `LayoutCount`. +- `session:sync` socket counts may stay the same unless server/store work changes them. + +## Risks + +- Selector equality matters. Returning new arrays/objects from selectors can erase the win. +- Be careful with lockdown mode; the full-page lockdown branch must still update promptly. +- If `roster` reference stability is poor upstream, this issue may need `SessionContext` + structural sharing before it fully improves. + diff --git a/perf/spectate-issues/005-shared-log-session-stream-pressure.md b/perf/spectate-issues/005-shared-log-session-stream-pressure.md new file mode 100644 index 00000000..4430919c --- /dev/null +++ b/perf/spectate-issues/005-shared-log-session-stream-pressure.md @@ -0,0 +1,177 @@ +# Issue 005: Shared Log And Session Stream Pressure + +## Summary + +`/spectate` receives a large volume of shared live socket traffic, especially `log:entry`, +`session:sync`, `sensorFrame`, and `commandAck`. Some of this will likely be fixed by the +`/` backlog, but it remains important for `/spectate` because the route keeps logs/chat +visible in the sidebar while also rendering multiple live media feeds. + +## Severity + +Medium. + +This is not as spectate-specific as the HUD map or WebRTC fan-out issues, but it adds +steady parsing, state, and render pressure. + +## Upstream Likelihood + +High for logs and shared session pressure. + +If the `/` work implements log batching, log virtualization, selector-based session +updates, and chat/nickname churn reduction, a lot of this issue should improve without +special `/spectate` changes. + +Related upstream issues: + +```txt +perf/issues/003-high-volume-log-stream.md +perf/issues/006-chat-and-nickname-churn.md +perf/issues/007-timers-and-polling.md +perf/issues/004-sensor-telemetry-render-frequency.md +``` + +## Affected Files + +- `webui/src/context/SessionContext.jsx` +- `webui/src/spectate/SpectatorApp/SpectatorContent.jsx` +- `webui/src/spectate/SpectatorApp/components/LogsRow.jsx` +- `webui/src/components/ChatPanel/index.jsx` +- `webui/src/components/RawUserPilePanel/index.jsx` +- `webui/src/components/RoverQueuesPanel/index.jsx` +- server log/session emitters + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +30s sample: + +```txt +log:entry events: 634 +log:entry bytes: 150,179 +session:sync events: 54 +session:sync bytes: 914,000 +commandAck events: 68 +chat:init bytes: 77,415 +log:init bytes: 44,530 +``` + +Text and input mutations also appeared: + +```txt +Text characterData mutations: 164 +input.field-input.flex-1 name mutations: 102 +input.chat-composer-input name mutations: 102 +``` + +The input-name churn matches the same type of sidebar/chat behavior observed on `/`. + +## Current Code Path + +`SpectatorContent` keeps these mounted: + +```txt +webui/src/spectate/SpectatorApp/SpectatorContent.jsx:88 + + +webui/src/spectate/SpectatorApp/SpectatorContent.jsx:91 + + +webui/src/spectate/SpectatorApp/SpectatorContent.jsx:79 + + +webui/src/spectate/SpectatorApp/SpectatorContent.jsx:68 + +``` + +That is useful spectator UI, but it means `/spectate` pays for general page state while +also paying for media and telemetry. + +## Why This Matters + +Logs and session syncs are deceptively expensive: + +- socket payloads must be parsed +- state must be updated +- arrays/lists may be copied +- list components may rerender +- text/input attributes may mutate + +This becomes worse on `/spectate` because the route is not just a dashboard. It is also a +multi-feed media surface. + +## Fix Strategies + +### Option A: Reuse The `/` Log Fix + +The `/` log issue should probably introduce: + +- batching log updates +- capping retained logs +- virtualizing or windowing visible rows +- pausing log rendering when collapsed/offscreen +- separating log ingestion from log rendering + +Apply the same path to `LogsRow`. + +### Option B: Make Spectator Sidebar Panels Independently Subscribed + +The rover media grid should not rerender because logs/chat/users changed. This overlaps +with issue 004. + +### Option C: Lower Or Gate Command/Log Visibility For Mobile + +On mobile spectator layout, logs are less important than video health. Options: + +- collapse logs by default +- batch logs while collapsed +- show only warning/error logs by default +- update visible logs at 1-2Hz + +### Option D: Reduce `session:sync` Payload Churn + +If the server sends full session snapshots frequently, consider: + +- event-specific deltas for high-frequency branches +- preserving client-side branch references for unchanged data +- moving logs/chat out of the broad session path if they are currently coupled + +## Recommended Path + +Do not start here if you are working the `/` list first. Fix the `/` log/session/chat +items, then rerun `/spectate` and reassess. + +If still hot on `/spectate`: + +1. Make `LogsRow` batch/virtualize/collapse. +2. Keep the media grid isolated from sidebar state. +3. Add a mobile spectator mode that reduces log rendering frequency. + +## Validation + +Run: + +```sh +CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \ + node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- Lower `Text:characterData` mutations. +- Fewer added/removed DOM mutation bursts. +- Lower `ScriptDuration`. +- `log:entry` socket count may remain high if only rendering is batched. + +## Risks + +- Logs are operationally useful; do not hide critical warnings. +- If batching is too aggressive, the UI can feel stale. +- Be careful not to mix log ingestion throttling with log display throttling. Usually, + display throttling is safer. + diff --git a/perf/spectate-issues/006-spectator-role-handshake-retries.md b/perf/spectate-issues/006-spectator-role-handshake-retries.md new file mode 100644 index 00000000..8ca736b3 --- /dev/null +++ b/perf/spectate-issues/006-spectator-role-handshake-retries.md @@ -0,0 +1,188 @@ +# Issue 006: Spectator Role Handshake Retries + +## Summary + +The spectator route sometimes attempts `subscribeAll` before the server has accepted the +socket as a spectator. In the 8x throttled profile, the console repeatedly showed: + +```txt +Failed to enter spectator mode Error: Spectator role required +``` + +This did not reproduce in the later 6x runtime audit, so it is intermittent. It is still +worth fixing because it can create startup retries, extra session/auth events, console +noise, and delayed media setup on slow devices. + +## Severity + +Medium-low. + +This is not the main steady-state CPU eater. Treat it as a correctness/startup reliability +issue that can make performance worse under throttling. + +## Upstream Likelihood + +Low. + +This is specific to `/spectate` startup and server role/subscription ordering. The `/` +performance backlog probably will not fix it. + +## Affected Files + +- `webui/src/hooks/useSpectatorMode.js` +- `webui/src/context/SocketContext.jsx` +- `webui/src/context/SessionContext.jsx` +- `server/src/services/authService/index.js` +- `server/src/services/roverManager/socketHandlers.js` + +## Evidence + +Artifact: + +```sh +perf/results/2026-06-11T03-14-34-168Z/report.json +``` + +That 8x CPU-throttled profile captured repeated console errors: + +```txt +Failed to enter spectator mode Error: Spectator role required +``` + +Fresh 6x runtime audit: + +```txt +consoleErrors: [] +auth:role events: 4 +``` + +So the race is not constant, but it exists under some timing conditions. + +## Current Code Path + +`useSpectatorMode` does this: + +```txt +webui/src/hooks/useSpectatorMode.js:18 +if (session?.role !== 'spectator') { + await setRole('spectator'); +} + +webui/src/hooks/useSpectatorMode.js:21 +await subscribeAll(); +``` + +The effect reruns when these change: + +```txt +webui/src/hooks/useSpectatorMode.js:36 +[connected, session?.mode, session?.role, setRole, subscribeAll] +``` + +The server rejects `subscribeAll` unless the socket role is already spectator: + +```txt +server/src/services/roverManager/socketHandlers.js:161 +if (socket.data?.role !== 'spectator') { + cb({ error: 'Spectator role required' }); +} +``` + +The auth service can initialize a socket as spectator only if the socket handshake query +requests it: + +```txt +server/src/services/authService/index.js:40 +const requestedRole = socket.handshake?.query?.role; + +server/src/services/authService/index.js:41 +const initialRole = requestedRole === 'spectator' ? 'spectator' : 'user'; +``` + +If the spectate page connects as a normal user and then switches role after connect, slow +timing can expose ordering issues. + +## Why This Matters + +The retry itself is not a huge CPU cost. The damage is indirect: + +- extra auth/session events +- repeated `subscribeAll` attempts +- delayed rover room joins +- delayed video/snapshot setup +- noisy console errors during profiling +- worse startup behavior on weak devices + +## Fix Strategies + +### Option A: Connect `/spectate` With `role=spectator` + +Make the spectator route establish its socket with a spectator role query from the start: + +```txt +io(..., { query: { role: 'spectator' } }) +``` + +This uses the existing server path in `authService`. + +### Option B: Wait For Confirmed Role Before `subscribeAll` + +After calling `setRole('spectator')`, wait until `auth:role` or session state confirms +`role === 'spectator'`, then call `subscribeAll`. + +Avoid calling `subscribeAll` in the same effect tick if the role update has not propagated. + +### Option C: Add A Server-Side Atomic Spectator Enter Event + +Create one event: + +```txt +session:enterSpectator +``` + +Server behavior: + +1. set socket role to spectator +2. join all visible rover rooms +3. return one ack + +This removes client-side ordering risk. + +### Option D: Debounce Or Guard Retries + +If `subscribeAll` fails with `Spectator role required`, do not retry in a tight loop. Wait +for an explicit role event or connection change. + +## Recommended Path + +The cleanest fix is Option A plus Option B: + +1. Make the `/spectate` socket connect as `role=spectator` when possible. +2. Keep `useSpectatorMode` as a fallback, but call `subscribeAll` only after confirmed + spectator role. + +If socket creation is shared in a way that makes route-specific query parameters awkward, +use the atomic server event instead. + +## Validation + +Run the throttled profile several times: + +```sh +CPU_THROTTLE=8 VIEWPORT=390x844 MOBILE=1 \ + node perf/live-cpu-profile.mjs https://rover.otter.land/spectate perf/results +``` + +Expected improvements: + +- No `Failed to enter spectator mode` console errors. +- Fewer `auth:role` events during startup. +- More consistent startup timing. + +## Risks + +- Be careful not to accidentally make ordinary `/` users spectators. +- If using route-specific socket query parameters, verify reconnects preserve the correct + role. +- In lockdown mode, spectator setup must still fail cleanly. + diff --git a/perf/spectate-issues/README.md b/perf/spectate-issues/README.md new file mode 100644 index 00000000..ad60bdd6 --- /dev/null +++ b/perf/spectate-issues/README.md @@ -0,0 +1,63 @@ +# `/spectate` CPU Performance Issue Index + +This folder breaks the live-site `/spectate` CPU investigation into separate, +work-ready issues. These are intentionally separate from `perf/issues`, which is the +current `/` backlog. + +The testing target was: + +```sh +https://rover.otter.land/spectate +``` + +Primary artifact: + +```sh +perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json +``` + +Secondary artifacts: + +```sh +perf/results/2026-06-11T03-14-34-168Z/report.json +perf/results/2026-06-11T03-11-39-596Z-route-sweep/route-sweep-report.json +``` + +## Ranked Issues + +1. [Telemetry fan-out and always-on HUD maps](./001-telemetry-fanout-hud-maps.md) +2. [Multi-rover WebRTC media fan-out](./002-multi-rover-webrtc-media-fanout.md) +3. [Room camera and snapshot object URL churn](./003-room-camera-snapshot-object-url-churn.md) +4. [Broad spectator session rerenders](./004-broad-spectator-session-rerenders.md) +5. [Shared log/session stream pressure](./005-shared-log-session-stream-pressure.md) +6. [Spectator role handshake retries](./006-spectator-role-handshake-retries.md) + +## Quick Read + +The biggest `/spectate` CPU issue is the number of live visual things being updated at +once. The page receives about 45 `sensorFrame` events/sec in the sampled run, and every +rover card has an always-on HUD map. That produced thousands of SVG `path` mutations in +30 seconds. + +The second issue is live media fan-out. The audit saw 6 `RTCPeerConnections`, which is a +lot for low-end laptops and mobile browsers, especially alongside React/HUD updates. + +The third issue is snapshot/camera image churn. Room camera frames created 117 `img.src` +mutations and the socket audit saw 11MB of binary payloads in 30 seconds. + +Issues 004 and 005 are where the `/` backlog overlaps most. If the `/` work adds better +session selectors, log batching, chat batching, and telemetry visual throttling, revisit +these spectate writeups before implementing them from scratch. + +Issue 006 is lower priority because it was intermittent, but it is worth fixing because +failed spectator setup can create extra retries, console noise, and unpredictable startup +work. + +## Upstream Labels + +Each issue includes an "Upstream Likelihood" section: + +- `High`: likely mostly fixed by the earlier `/` work. +- `Medium`: `/` work should help, but `/spectate` still needs targeted changes. +- `Low`: mostly spectate-specific. +