mirror of
https://github.com/legop3/MultiRoombaRover.git
synced 2026-09-16 01:21:20 -04:00
more performance sloptesting
This commit is contained in:
legop3
@@ -0,0 +1,219 @@
|
||||
# Issue 001: Telemetry Fan-Out And Always-On HUD Maps
|
||||
|
||||
## Summary
|
||||
|
||||
`/spectate` renders multiple rover cards at once, and every rover card includes a live HUD
|
||||
with an always-on top-down SVG map. During the 30s throttled audit, the page received
|
||||
1,354 `sensorFrame` events and produced more than 5,300 SVG `path` attribute mutations.
|
||||
|
||||
This is the strongest measured `/spectate` CPU problem.
|
||||
|
||||
## Severity
|
||||
|
||||
High.
|
||||
|
||||
This is high-frequency, always-on work. It scales with the number of visible rovers and
|
||||
hits exactly the devices we care about: mobile browsers and weaker computers.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
Medium.
|
||||
|
||||
The `/` telemetry issue will likely help if it introduces a central throttled visual
|
||||
telemetry path. However, `/spectate` has a unique multiplier: it renders several rover
|
||||
HUDs and several `TopDownMap` instances at the same time. Even after upstream telemetry
|
||||
throttling, `/spectate` probably still needs a policy for how many maps update and how
|
||||
often.
|
||||
|
||||
Related upstream issue:
|
||||
|
||||
```txt
|
||||
perf/issues/004-sensor-telemetry-render-frequency.md
|
||||
```
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/components/SpectateVideo/index.jsx`
|
||||
- `webui/src/components/HudOverlays/HudOverlay/index.jsx`
|
||||
- `webui/src/components/HudOverlays/HudOverlay/HudMapOverlay.jsx`
|
||||
- `webui/src/components/TopDownMap/TopDownMapContent.jsx`
|
||||
- `webui/src/components/TopDownMap/visuals.jsx`
|
||||
- `webui/src/context/TelemetryContext.jsx`
|
||||
- `webui/src/components/RoverMediaPlayer/index.jsx`
|
||||
- `webui/src/components/HudOverlays/OvercurrentOverlay/index.jsx`
|
||||
- `webui/src/components/HudOverlays/LowBatteryOverlay/index.jsx`
|
||||
- `webui/src/components/HudOverlays/VerticalBatteryOverlay/index.jsx`
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
30s sample, mobile viewport, 6x CPU throttle:
|
||||
|
||||
```txt
|
||||
sensorFrame events: 1,354
|
||||
sensorFrame bytes: 2,531,090
|
||||
path d mutations: 2,688
|
||||
path fill mutations: 2,682
|
||||
Long tasks: 81
|
||||
Worst long task: 375ms
|
||||
Average frame gap: 113.9ms
|
||||
p95 frame gap: 233.3ms
|
||||
```
|
||||
|
||||
The path mutation pattern is the important clue. Normal text/status updates do not mutate
|
||||
thousands of SVG path `d` and `fill` attributes. `TopDownMap` does.
|
||||
|
||||
## Current Code Path
|
||||
|
||||
`SpectateVideo` mounts a full media and overlay stack for each rover:
|
||||
|
||||
```txt
|
||||
webui/src/components/SpectateVideo/index.jsx:17
|
||||
RoverMediaPlayer
|
||||
|
||||
webui/src/components/SpectateVideo/index.jsx:26
|
||||
HudOverlay variant="spectator"
|
||||
|
||||
webui/src/components/SpectateVideo/index.jsx:33
|
||||
OvercurrentOverlay
|
||||
|
||||
webui/src/components/SpectateVideo/index.jsx:35
|
||||
VerticalBatteryOverlay
|
||||
```
|
||||
|
||||
`HudOverlay` subscribes to telemetry:
|
||||
|
||||
```txt
|
||||
webui/src/components/HudOverlays/HudOverlay/index.jsx:27
|
||||
const frame = useTelemetryFrame(effectiveRoverId);
|
||||
```
|
||||
|
||||
For spectator mode, it forces the top-down map on:
|
||||
|
||||
```txt
|
||||
webui/src/components/HudOverlays/HudOverlay/index.jsx:50
|
||||
variant === 'spectator' ? true : ...
|
||||
```
|
||||
|
||||
Then `HudMapOverlay` renders:
|
||||
|
||||
```txt
|
||||
webui/src/components/HudOverlays/HudOverlay/HudMapOverlay.jsx:35
|
||||
<TopDownMap sensors={sensors} size={240} overlay />
|
||||
```
|
||||
|
||||
There is also duplicated telemetry subscription pressure. `RoverMediaPlayer` subscribes to
|
||||
telemetry for audio ducking and media state, while `HudOverlay`, battery overlays, and
|
||||
warning overlays can subscribe for the same rover.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
The raw socket message rate is not automatically a problem. The expensive part is waking
|
||||
multiple React subscribers, recomputing visual sensor geometry, and mutating SVG
|
||||
attributes for every visual update.
|
||||
|
||||
On `/`, there is generally one primary driver view. On `/spectate`, the same sensor stream
|
||||
can be multiplied across several rover cards.
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Add A Throttled Visual Telemetry Channel
|
||||
|
||||
This is the most reusable upstream-friendly fix.
|
||||
|
||||
Keep `TelemetryContext` storing the latest frame immediately, but notify visual consumers
|
||||
at a lower cadence:
|
||||
|
||||
```txt
|
||||
useTelemetryFrameRaw(roverId) high-frequency, only where truly needed
|
||||
useTelemetryFrameVisual(roverId) throttled display data
|
||||
```
|
||||
|
||||
Use visual telemetry for:
|
||||
|
||||
- `TopDownMap`
|
||||
- `HudOverlay`
|
||||
- `SpectatorTelemetryOverlay`
|
||||
- battery overlays
|
||||
- warning overlays where 100-200ms delay is acceptable
|
||||
|
||||
Suggested rates:
|
||||
|
||||
```txt
|
||||
desktop visual telemetry: 10-15Hz
|
||||
mobile visual telemetry: 5-10Hz
|
||||
```
|
||||
|
||||
### Option B: Pass One Telemetry Frame Through `SpectateVideo`
|
||||
|
||||
Right now, each overlay can subscribe independently. Instead, let `SpectateVideo` subscribe
|
||||
once per rover and pass `sensors` down:
|
||||
|
||||
```jsx
|
||||
const frame = useTelemetryFrameVisual(roverId);
|
||||
const sensors = frame?.sensors ?? null;
|
||||
|
||||
<RoverMediaPlayer roverId={roverId} sensors={sensors} />
|
||||
<HudOverlay roverId={roverId} sensors={sensors} />
|
||||
<OvercurrentOverlay roverId={roverId} sensors={sensors} />
|
||||
<LowBatteryOverlay roverId={roverId} sensors={sensors} />
|
||||
<VerticalBatteryOverlay roverId={roverId} sensors={sensors} />
|
||||
```
|
||||
|
||||
This may require small prop additions to the warning/battery overlays.
|
||||
|
||||
### Option C: Make Spectator HUD Maps Adaptive
|
||||
|
||||
Spectator maps are currently always on. For low-end/mobile:
|
||||
|
||||
- hide maps by default on portrait mobile
|
||||
- update maps at a lower cadence than text warnings
|
||||
- only animate the focused/first rover's map
|
||||
- show static/minimal maps for secondary rovers
|
||||
- make map visibility a spectator setting
|
||||
|
||||
### Option D: Memoize TopDownMap Geometry
|
||||
|
||||
This is useful after throttling. Avoid recalculating path data when the sensor fields that
|
||||
drive a particular path did not change.
|
||||
|
||||
## Recommended Path
|
||||
|
||||
Do this after the `/` telemetry issue if that work touches `TelemetryContext`.
|
||||
|
||||
Best first implementation:
|
||||
|
||||
1. Add `useTelemetryFrameVisual(roverId)` with a 100-150ms visual notification cadence.
|
||||
2. Switch `HudOverlay` and `TopDownMap` consumers to visual telemetry.
|
||||
3. In `SpectateVideo`, subscribe once and pass sensors down where practical.
|
||||
4. Add a spectator/mobile map policy so only the most important maps update live.
|
||||
|
||||
## Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \
|
||||
node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- `path d` and `path fill` mutations should drop sharply.
|
||||
- `ScriptDuration` should drop.
|
||||
- Long task count should drop.
|
||||
- Frame p95/p99 should improve.
|
||||
- `sensorFrame` socket count may stay the same if only rendering is throttled.
|
||||
|
||||
## Risks
|
||||
|
||||
- Do not throttle control/safety logic that needs immediate data.
|
||||
- Keep battery/overcurrent warnings responsive enough to be trusted.
|
||||
- If map updates become too slow, operators may perceive the HUD as stale. A 5-10Hz map is
|
||||
usually enough for spectators, but verify by watching the live page.
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# Issue 002: Multi-Rover WebRTC Media Fan-Out
|
||||
|
||||
## Summary
|
||||
|
||||
`/spectate` mounts several `RoverMediaPlayer` instances at once. In the fresh runtime
|
||||
audit, Chromium reported 6 active `RTCPeerConnections`. That is a large baseline for
|
||||
low-end clients, especially while the page is also handling telemetry, snapshots, logs,
|
||||
chat, and SVG HUD maps.
|
||||
|
||||
## Severity
|
||||
|
||||
High.
|
||||
|
||||
This may not show up as React mutations, but it consumes CPU in browser media pipelines,
|
||||
WebRTC negotiation, decoding, audio handling, and event dispatch. It also scales with the
|
||||
number of rovers.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
Low to medium.
|
||||
|
||||
General `RoverMediaPlayer` improvements can help both `/` and `/spectate`, but the core
|
||||
problem is spectate-specific: `/spectate` intentionally displays multiple rover media
|
||||
players at once. The fix needs a spectator policy for active feeds.
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/components/SpectateVideo/index.jsx`
|
||||
- `webui/src/components/RoverMediaPlayer/index.jsx`
|
||||
- `webui/src/hooks/useVideoRequests.js`
|
||||
- `webui/src/lib/whepPlayer.js`
|
||||
- `webui/src/spectate/SpectatorApp/components/RoverRow.jsx`
|
||||
- server-side WHEP/video request handling, if feed quality/rate selection is added
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
30s sample:
|
||||
|
||||
```txt
|
||||
RTCPeerConnections: 6
|
||||
JSEventListeners: 379
|
||||
```
|
||||
|
||||
The listener list is dominated by normal media listeners:
|
||||
|
||||
```txt
|
||||
audio canplay/ended/error/pause/play/stalled/waiting: 9 each
|
||||
video media events: 3 each
|
||||
```
|
||||
|
||||
That pattern fits several rover media players, likely with video and dedicated audio
|
||||
connections.
|
||||
|
||||
The same audit also saw:
|
||||
|
||||
```txt
|
||||
TaskDuration: 28.48s
|
||||
ScriptDuration: 12.11s
|
||||
Long tasks: 81
|
||||
```
|
||||
|
||||
The earlier 8x throttled `/spectate` profile saw:
|
||||
|
||||
```txt
|
||||
Task delta: 49.13s
|
||||
Script delta: 19.58s
|
||||
```
|
||||
|
||||
## Current Code Path
|
||||
|
||||
Each rover card mounts:
|
||||
|
||||
```txt
|
||||
webui/src/components/SpectateVideo/index.jsx:17
|
||||
<RoverMediaPlayer roverId={roverId} />
|
||||
```
|
||||
|
||||
`RoverMediaPlayer` automatically creates video and audio request entries:
|
||||
|
||||
```txt
|
||||
webui/src/components/RoverMediaPlayer/index.jsx
|
||||
autoEntries = [
|
||||
video rover entry,
|
||||
optional dedicated audio entry,
|
||||
]
|
||||
```
|
||||
|
||||
It then requests sources through `useVideoRequests`, starts WHEP players, maintains
|
||||
restart timers, maintains unmute/audio retry timers, and optionally falls back to
|
||||
snapshots when no WHEP URL exists.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Six peer connections on a powerful desktop may feel fine. On weak laptops and mobile
|
||||
devices, simultaneous WebRTC decode plus React UI churn can easily saturate the main
|
||||
thread or media threads. The visible symptom is not necessarily one obvious function in
|
||||
the CPU profile; it is degraded frame cadence, long tasks, and high browser-level media
|
||||
work.
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Cap Active Live Feeds On Mobile
|
||||
|
||||
For mobile/low-end layouts:
|
||||
|
||||
- one primary live WHEP feed
|
||||
- secondary rover cards use snapshots or lower-rate previews
|
||||
- promote a secondary rover to live only when selected/visible/focused
|
||||
|
||||
This is the highest-impact spectator-specific design change.
|
||||
|
||||
### Option B: Pause Offscreen Feeds
|
||||
|
||||
Use `IntersectionObserver` around each `SpectateVideo` or `RoverSpectatorCard`:
|
||||
|
||||
- start WHEP when the card is visible
|
||||
- stop WHEP when the card is far offscreen
|
||||
- keep a small grace period to avoid flapping while scrolling
|
||||
|
||||
This matters especially because portrait `/spectate` can scroll.
|
||||
|
||||
### Option C: Disable Dedicated Audio For Non-Focused Rovers
|
||||
|
||||
Dedicated audio is useful, but not necessarily for every rover at the same time.
|
||||
|
||||
Possible policy:
|
||||
|
||||
- audio enabled only for the selected/primary rover
|
||||
- audio disabled for muted secondary rovers
|
||||
- audio WHEP starts only after user interaction
|
||||
|
||||
This should reduce peer connections and audio retry timers.
|
||||
|
||||
### Option D: Request Lower Quality For Spectator Secondary Feeds
|
||||
|
||||
If the server/media stack supports it, add spectator quality tiers:
|
||||
|
||||
```txt
|
||||
primary: normal FPS/resolution
|
||||
secondary: low FPS/resolution or snapshot-only
|
||||
mobile: lower default bitrate/resolution
|
||||
```
|
||||
|
||||
### Option E: Stop Retry Loops When Hidden
|
||||
|
||||
When document visibility is hidden or a card is offscreen:
|
||||
|
||||
- stop restart timers
|
||||
- stop audio retry intervals
|
||||
- avoid requesting fresh WHEP URLs
|
||||
|
||||
## Recommended Path
|
||||
|
||||
After the HUD/telemetry issue, add a spectator feed activity policy:
|
||||
|
||||
1. Define "primary" versus "secondary" rover cards.
|
||||
2. Keep only primary feeds fully live on mobile.
|
||||
3. Use `IntersectionObserver` to avoid live playback for offscreen cards.
|
||||
4. Disable secondary audio by default.
|
||||
|
||||
## Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \
|
||||
node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- `RTCPeerConnections` should drop below 6 on mobile.
|
||||
- Media listener counts should drop.
|
||||
- `TaskDuration` should drop.
|
||||
- Frame p95/p99 should improve.
|
||||
- User-visible video should still start reliably for the primary rover.
|
||||
|
||||
## Risks
|
||||
|
||||
- Spectators may expect all feeds live all the time on desktop.
|
||||
- Switching feeds can introduce startup delay if WHEP is torn down too aggressively.
|
||||
- Audio policy needs to avoid surprising users. Make the focused rover's audio behavior
|
||||
clear and consistent.
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
# Issue 003: Room Camera And Snapshot Object URL Churn
|
||||
|
||||
## Summary
|
||||
|
||||
Room camera and rover snapshot frames are delivered as binary payloads. The client creates
|
||||
a new `Blob`, creates a new object URL, revokes the previous URL, and updates React state
|
||||
for every frame. In the 30s `/spectate` audit, this caused 117 `img.src` mutations and
|
||||
11MB of binary socket payloads.
|
||||
|
||||
## Severity
|
||||
|
||||
Medium-high.
|
||||
|
||||
This is not the biggest measured CPU source, but it is steady work and very visible on
|
||||
low-end devices because JPEG decoding, Blob allocation, object URL churn, image source
|
||||
updates, and React state updates all happen together.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
Low.
|
||||
|
||||
This is mostly `/spectate` and camera-panel specific. It will not be fixed by the `/`
|
||||
input, control context, or log work. It may benefit indirectly if broad rerenders are
|
||||
reduced, but the frame handling itself needs targeted changes.
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/hooks/useRoomCameraSnapshots.js`
|
||||
- `webui/src/hooks/useRoverSnapshots.js`
|
||||
- `webui/src/components/RoomCameraPanel/index.jsx`
|
||||
- `webui/src/components/RoomCameraFeed/index.jsx`
|
||||
- `server/src/services/roomCameraService/socketGateway.js`
|
||||
- `server/src/services/roverSnapshotService/socketGateway.js`
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
30s sample:
|
||||
|
||||
```txt
|
||||
unlabeled binary socket events: 126
|
||||
unlabeled binary bytes: 11,048,477
|
||||
roomCamera:frame events: 117
|
||||
roverSnapshot:frame events: 7
|
||||
img src mutations: 117
|
||||
blink class mutations: 117
|
||||
```
|
||||
|
||||
The 8x throttled profile also had these CPU entries:
|
||||
|
||||
```txt
|
||||
Blob
|
||||
decodeString
|
||||
setAttribute
|
||||
```
|
||||
|
||||
Those line up with binary socket parsing, Blob/object URL creation, and `img.src` updates.
|
||||
|
||||
## Current Code Path
|
||||
|
||||
`useRoomCameraSnapshots` handles each frame:
|
||||
|
||||
```txt
|
||||
webui/src/hooks/useRoomCameraSnapshots.js:78
|
||||
const blob = new Blob([buffer], { type: 'image/jpeg' });
|
||||
|
||||
webui/src/hooks/useRoomCameraSnapshots.js:79
|
||||
const url = URL.createObjectURL(blob);
|
||||
|
||||
webui/src/hooks/useRoomCameraSnapshots.js:82
|
||||
URL.revokeObjectURL(prevUrl);
|
||||
|
||||
webui/src/hooks/useRoomCameraSnapshots.js:85
|
||||
setFeeds(...)
|
||||
```
|
||||
|
||||
`useRoverSnapshots` has the same pattern.
|
||||
|
||||
`RoomCameraFeed` writes that object URL into an image:
|
||||
|
||||
```txt
|
||||
webui/src/components/RoomCameraFeed/index.jsx:24
|
||||
<img src={feed.objectUrl} ... />
|
||||
```
|
||||
|
||||
It also toggles blink state for each frame:
|
||||
|
||||
```txt
|
||||
webui/src/components/RoomCameraFeed/index.jsx:12
|
||||
setBlink((prev) => !prev);
|
||||
```
|
||||
|
||||
Server-side frame rates:
|
||||
|
||||
```txt
|
||||
server/src/services/roomCameraService/socketGateway.js:11
|
||||
STREAM_INTERVAL_MS = 1000
|
||||
|
||||
server/src/services/roverSnapshotService/socketGateway.js:11
|
||||
STREAM_INTERVAL_MS = 333
|
||||
```
|
||||
|
||||
## Why This Matters
|
||||
|
||||
The page is doing memory allocation and DOM/image decode work on every camera frame. At
|
||||
one room camera per second this can be acceptable, but `/spectate` is already busy. When
|
||||
combined with live WebRTC and telemetry maps, this becomes another steady CPU drain.
|
||||
|
||||
The blink dot is also doing one React state update and class mutation per frame. It is
|
||||
small, but it is pure extra work.
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Drop Frames When The Client Is Behind
|
||||
|
||||
Keep only the latest pending frame per camera. If the previous object URL has not been
|
||||
painted or decoded yet, replace the pending buffer instead of forcing every frame through
|
||||
React.
|
||||
|
||||
### Option B: Throttle Display Updates On Mobile
|
||||
|
||||
Render room camera frames at a lower client-side display rate:
|
||||
|
||||
```txt
|
||||
desktop: 1fps as today, or configured
|
||||
mobile: 0.2-0.5fps for secondary room cameras
|
||||
```
|
||||
|
||||
The server can still emit at its existing rate for other clients, but this client can
|
||||
choose not to render every frame.
|
||||
|
||||
### Option C: Lazy Mount Or Collapse Room Cameras On Mobile
|
||||
|
||||
On portrait mobile, `SecondaryRow` can defer `RoomCameraPanel` until:
|
||||
|
||||
- the user scrolls near it
|
||||
- the user opens the panel
|
||||
- the page is idle after initial rover feeds have settled
|
||||
|
||||
### Option D: Remove Per-Frame React Blink State
|
||||
|
||||
Replace the blink toggle with CSS animation or a timestamp updated less often.
|
||||
|
||||
For example:
|
||||
|
||||
- keep status text
|
||||
- use a CSS pulse class while status is `playing`
|
||||
- avoid toggling React state on every frame
|
||||
|
||||
### Option E: Stabilize Room Camera Source Lists
|
||||
|
||||
`RoomCameraPanel` currently passes a newly created array of objects into the snapshot hook:
|
||||
|
||||
```txt
|
||||
webui/src/components/RoomCameraPanel/index.jsx:37
|
||||
useRoomCameraSnapshots(cameras.map((camera) => ({ id: camera.id })))
|
||||
```
|
||||
|
||||
The hook ultimately keys by IDs, so this is not the main issue, but it still creates extra
|
||||
memo/effect churn on rerender. Prefer a memoized array of IDs:
|
||||
|
||||
```jsx
|
||||
const cameraIds = useMemo(() => cameras.map((camera) => camera.id), [cameras]);
|
||||
const feedMap = useRoomCameraSnapshots(cameraIds);
|
||||
```
|
||||
|
||||
## Recommended Path
|
||||
|
||||
Start with the cheap fixes:
|
||||
|
||||
1. Remove the per-frame blink state.
|
||||
2. Memoize camera IDs in `RoomCameraPanel`.
|
||||
3. Add client-side render throttling/drop-latest behavior in `useRoomCameraSnapshots`.
|
||||
4. Consider lazy mounting/collapsing room cameras in mobile spectator layout.
|
||||
|
||||
## Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \
|
||||
node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- `img.h-full.w-full:attributes:src` mutations should drop if display is throttled.
|
||||
- `span.h-2.w-2:attributes:class` mutations should disappear if blink state is removed.
|
||||
- `ScriptDuration` and long tasks should improve modestly.
|
||||
- Binary socket bytes may stay the same unless server subscription/rate is changed.
|
||||
|
||||
## Risks
|
||||
|
||||
- Reducing room camera update rate can make the room view feel stale.
|
||||
- Object URL lifecycle must stay correct. Do not leak old URLs.
|
||||
- If lazy mounting unsubscribes too aggressively, first frame display may feel delayed.
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
# Issue 004: Broad Spectator Session Rerenders
|
||||
|
||||
## Summary
|
||||
|
||||
`SpectatorContent` reads the whole session object with `useSession()`. That means broad
|
||||
session changes can rerender the top-level spectator layout, including the sidebar, rover
|
||||
row, secondary row, chat, logs, room cameras, and overlays.
|
||||
|
||||
The audit saw 54 `session:sync` events in 30s, carrying about 914KB, while the DOM also
|
||||
showed repeated added/removed-node bursts.
|
||||
|
||||
## Severity
|
||||
|
||||
Medium-high.
|
||||
|
||||
This is likely a multiplier. It may not be the single hottest path, but it can cause
|
||||
otherwise independent components to rerender together.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
Medium to high.
|
||||
|
||||
If the `/` backlog changes `SessionContext` to provide better selectors, better structural
|
||||
sharing, batched log/session updates, or more stable action references, this issue may be
|
||||
partly fixed upstream. However, `SpectatorContent` itself still needs to stop reading the
|
||||
whole session object.
|
||||
|
||||
Related upstream issues:
|
||||
|
||||
```txt
|
||||
perf/issues/003-high-volume-log-stream.md
|
||||
perf/issues/006-chat-and-nickname-churn.md
|
||||
perf/issues/007-timers-and-polling.md
|
||||
```
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/spectate/SpectatorApp/SpectatorContent.jsx`
|
||||
- `webui/src/hooks/useSpectatorMode.js`
|
||||
- `webui/src/context/SessionContext.jsx`
|
||||
- `webui/src/spectate/SpectatorApp/components/RoverRow.jsx`
|
||||
- `webui/src/spectate/SpectatorApp/components/SecondaryRow.jsx`
|
||||
- sidebar components mounted by `SpectatorContent`
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
30s sample:
|
||||
|
||||
```txt
|
||||
session:sync events: 54
|
||||
session:sync bytes: 914,000
|
||||
DOM mutation records: 5,868
|
||||
DOM nodes added: 440
|
||||
DOM nodes removed: 440
|
||||
```
|
||||
|
||||
Largest mutation bursts:
|
||||
|
||||
```txt
|
||||
119 records, 57 added, 58 removed
|
||||
117 records, 52 added, 53 removed
|
||||
113 records, 52 added, 52 removed
|
||||
102 records, 49 added, 49 removed
|
||||
100 records, 48 added, 48 removed
|
||||
```
|
||||
|
||||
Those bursts suggest periodic component/list updates, not just single text changes.
|
||||
|
||||
## Current Code Path
|
||||
|
||||
`SpectatorContent` subscribes to the whole session:
|
||||
|
||||
```txt
|
||||
webui/src/spectate/SpectatorApp/SpectatorContent.jsx:22
|
||||
const { session } = useSession();
|
||||
```
|
||||
|
||||
It then derives:
|
||||
|
||||
```txt
|
||||
inLockdown = session?.mode === 'lockdown'
|
||||
roster = session?.roster ?? []
|
||||
```
|
||||
|
||||
But because it reads the whole session object, changes to logs, users, active drivers,
|
||||
room cameras, replay state, chat-related session fields, or any other session branch can
|
||||
potentially re-render the whole spectator layout.
|
||||
|
||||
`useSpectatorMode` also reads the whole session:
|
||||
|
||||
```txt
|
||||
webui/src/hooks/useSpectatorMode.js:7
|
||||
const { session, setRole, subscribeAll, connected } = useSession();
|
||||
```
|
||||
|
||||
It only needs `mode`, `role`, `connected`, and actions.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
`/spectate` is already rendering expensive children. A broad rerender at the top makes
|
||||
other optimizations less effective because unrelated live data can still wake the route.
|
||||
|
||||
For example:
|
||||
|
||||
- a log entry should not rerender the rover media grid
|
||||
- a chat composer state change should not rerender room camera feeds
|
||||
- a session sync should not recreate layout props unless the selected fields changed
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Replace Whole Session Reads With Selectors
|
||||
|
||||
In `SpectatorContent`, use precise selectors:
|
||||
|
||||
```jsx
|
||||
const mode = useSessionSelector((state) => state.session?.mode ?? null);
|
||||
const roster = useSessionSelector((state) => state.session?.roster ?? EMPTY_ROSTER);
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
```jsx
|
||||
const inLockdown = mode === 'lockdown';
|
||||
```
|
||||
|
||||
Make sure selector outputs are stable. If `roster` is rebuilt on every `session:sync`,
|
||||
this still rerenders. The store may need structural sharing upstream.
|
||||
|
||||
### Option B: Split Spectator Layout Into Memoized Regions
|
||||
|
||||
Split the page into:
|
||||
|
||||
- spectator sidebar
|
||||
- rover grid
|
||||
- secondary/camera row
|
||||
- global overlays
|
||||
|
||||
Then each region subscribes only to what it needs.
|
||||
|
||||
### Option C: Narrow `useSpectatorMode`
|
||||
|
||||
Replace:
|
||||
|
||||
```jsx
|
||||
const { session, setRole, subscribeAll, connected } = useSession();
|
||||
```
|
||||
|
||||
with selectors/actions:
|
||||
|
||||
```jsx
|
||||
const mode = useSessionSelector((state) => state.session?.mode ?? null);
|
||||
const role = useSessionSelector((state) => state.session?.role ?? null);
|
||||
const connected = useSessionSelector((state) => state.connected);
|
||||
const { setRole, subscribeAll } = useSessionActions();
|
||||
```
|
||||
|
||||
### Option D: Batch Or Diff `session:sync`
|
||||
|
||||
This may belong to the `/` backlog. If incoming sync payloads replace large object
|
||||
branches each time, selector users will still rerender. Preserve references for unchanged
|
||||
branches.
|
||||
|
||||
## Recommended Path
|
||||
|
||||
Do this after the `/` session/log work if that work changes `SessionContext`.
|
||||
|
||||
Local `/spectate` changes:
|
||||
|
||||
1. Remove whole-session `useSession()` from `SpectatorContent`.
|
||||
2. Remove whole-session `useSession()` from `useSpectatorMode`.
|
||||
3. Split the layout so rover media rows do not depend on sidebar data.
|
||||
4. Re-test mutation bursts and React commit behavior.
|
||||
|
||||
## Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \
|
||||
node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- Fewer added/removed DOM mutation bursts.
|
||||
- Lower `ScriptDuration`.
|
||||
- Possibly lower `RecalcStyleCount` and `LayoutCount`.
|
||||
- `session:sync` socket counts may stay the same unless server/store work changes them.
|
||||
|
||||
## Risks
|
||||
|
||||
- Selector equality matters. Returning new arrays/objects from selectors can erase the win.
|
||||
- Be careful with lockdown mode; the full-page lockdown branch must still update promptly.
|
||||
- If `roster` reference stability is poor upstream, this issue may need `SessionContext`
|
||||
structural sharing before it fully improves.
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
# Issue 005: Shared Log And Session Stream Pressure
|
||||
|
||||
## Summary
|
||||
|
||||
`/spectate` receives a large volume of shared live socket traffic, especially `log:entry`,
|
||||
`session:sync`, `sensorFrame`, and `commandAck`. Some of this will likely be fixed by the
|
||||
`/` backlog, but it remains important for `/spectate` because the route keeps logs/chat
|
||||
visible in the sidebar while also rendering multiple live media feeds.
|
||||
|
||||
## Severity
|
||||
|
||||
Medium.
|
||||
|
||||
This is not as spectate-specific as the HUD map or WebRTC fan-out issues, but it adds
|
||||
steady parsing, state, and render pressure.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
High for logs and shared session pressure.
|
||||
|
||||
If the `/` work implements log batching, log virtualization, selector-based session
|
||||
updates, and chat/nickname churn reduction, a lot of this issue should improve without
|
||||
special `/spectate` changes.
|
||||
|
||||
Related upstream issues:
|
||||
|
||||
```txt
|
||||
perf/issues/003-high-volume-log-stream.md
|
||||
perf/issues/006-chat-and-nickname-churn.md
|
||||
perf/issues/007-timers-and-polling.md
|
||||
perf/issues/004-sensor-telemetry-render-frequency.md
|
||||
```
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/context/SessionContext.jsx`
|
||||
- `webui/src/spectate/SpectatorApp/SpectatorContent.jsx`
|
||||
- `webui/src/spectate/SpectatorApp/components/LogsRow.jsx`
|
||||
- `webui/src/components/ChatPanel/index.jsx`
|
||||
- `webui/src/components/RawUserPilePanel/index.jsx`
|
||||
- `webui/src/components/RoverQueuesPanel/index.jsx`
|
||||
- server log/session emitters
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
30s sample:
|
||||
|
||||
```txt
|
||||
log:entry events: 634
|
||||
log:entry bytes: 150,179
|
||||
session:sync events: 54
|
||||
session:sync bytes: 914,000
|
||||
commandAck events: 68
|
||||
chat:init bytes: 77,415
|
||||
log:init bytes: 44,530
|
||||
```
|
||||
|
||||
Text and input mutations also appeared:
|
||||
|
||||
```txt
|
||||
Text characterData mutations: 164
|
||||
input.field-input.flex-1 name mutations: 102
|
||||
input.chat-composer-input name mutations: 102
|
||||
```
|
||||
|
||||
The input-name churn matches the same type of sidebar/chat behavior observed on `/`.
|
||||
|
||||
## Current Code Path
|
||||
|
||||
`SpectatorContent` keeps these mounted:
|
||||
|
||||
```txt
|
||||
webui/src/spectate/SpectatorApp/SpectatorContent.jsx:88
|
||||
<ChatPanel allowSpectatorInput ... />
|
||||
|
||||
webui/src/spectate/SpectatorApp/SpectatorContent.jsx:91
|
||||
<LogsRow ... />
|
||||
|
||||
webui/src/spectate/SpectatorApp/SpectatorContent.jsx:79
|
||||
<RoverQueuesPanel title="Rovers" />
|
||||
|
||||
webui/src/spectate/SpectatorApp/SpectatorContent.jsx:68
|
||||
<RawUserPilePanel ... />
|
||||
```
|
||||
|
||||
That is useful spectator UI, but it means `/spectate` pays for general page state while
|
||||
also paying for media and telemetry.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
Logs and session syncs are deceptively expensive:
|
||||
|
||||
- socket payloads must be parsed
|
||||
- state must be updated
|
||||
- arrays/lists may be copied
|
||||
- list components may rerender
|
||||
- text/input attributes may mutate
|
||||
|
||||
This becomes worse on `/spectate` because the route is not just a dashboard. It is also a
|
||||
multi-feed media surface.
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Reuse The `/` Log Fix
|
||||
|
||||
The `/` log issue should probably introduce:
|
||||
|
||||
- batching log updates
|
||||
- capping retained logs
|
||||
- virtualizing or windowing visible rows
|
||||
- pausing log rendering when collapsed/offscreen
|
||||
- separating log ingestion from log rendering
|
||||
|
||||
Apply the same path to `LogsRow`.
|
||||
|
||||
### Option B: Make Spectator Sidebar Panels Independently Subscribed
|
||||
|
||||
The rover media grid should not rerender because logs/chat/users changed. This overlaps
|
||||
with issue 004.
|
||||
|
||||
### Option C: Lower Or Gate Command/Log Visibility For Mobile
|
||||
|
||||
On mobile spectator layout, logs are less important than video health. Options:
|
||||
|
||||
- collapse logs by default
|
||||
- batch logs while collapsed
|
||||
- show only warning/error logs by default
|
||||
- update visible logs at 1-2Hz
|
||||
|
||||
### Option D: Reduce `session:sync` Payload Churn
|
||||
|
||||
If the server sends full session snapshots frequently, consider:
|
||||
|
||||
- event-specific deltas for high-frequency branches
|
||||
- preserving client-side branch references for unchanged data
|
||||
- moving logs/chat out of the broad session path if they are currently coupled
|
||||
|
||||
## Recommended Path
|
||||
|
||||
Do not start here if you are working the `/` list first. Fix the `/` log/session/chat
|
||||
items, then rerun `/spectate` and reassess.
|
||||
|
||||
If still hot on `/spectate`:
|
||||
|
||||
1. Make `LogsRow` batch/virtualize/collapse.
|
||||
2. Keep the media grid isolated from sidebar state.
|
||||
3. Add a mobile spectator mode that reduces log rendering frequency.
|
||||
|
||||
## Validation
|
||||
|
||||
Run:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=6 VIEWPORT=390x844 MOBILE=1 SAMPLE_MS=30000 \
|
||||
node perf/live-root-runtime-audit.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- Lower `Text:characterData` mutations.
|
||||
- Fewer added/removed DOM mutation bursts.
|
||||
- Lower `ScriptDuration`.
|
||||
- `log:entry` socket count may remain high if only rendering is batched.
|
||||
|
||||
## Risks
|
||||
|
||||
- Logs are operationally useful; do not hide critical warnings.
|
||||
- If batching is too aggressive, the UI can feel stale.
|
||||
- Be careful not to mix log ingestion throttling with log display throttling. Usually,
|
||||
display throttling is safer.
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
# Issue 006: Spectator Role Handshake Retries
|
||||
|
||||
## Summary
|
||||
|
||||
The spectator route sometimes attempts `subscribeAll` before the server has accepted the
|
||||
socket as a spectator. In the 8x throttled profile, the console repeatedly showed:
|
||||
|
||||
```txt
|
||||
Failed to enter spectator mode Error: Spectator role required
|
||||
```
|
||||
|
||||
This did not reproduce in the later 6x runtime audit, so it is intermittent. It is still
|
||||
worth fixing because it can create startup retries, extra session/auth events, console
|
||||
noise, and delayed media setup on slow devices.
|
||||
|
||||
## Severity
|
||||
|
||||
Medium-low.
|
||||
|
||||
This is not the main steady-state CPU eater. Treat it as a correctness/startup reliability
|
||||
issue that can make performance worse under throttling.
|
||||
|
||||
## Upstream Likelihood
|
||||
|
||||
Low.
|
||||
|
||||
This is specific to `/spectate` startup and server role/subscription ordering. The `/`
|
||||
performance backlog probably will not fix it.
|
||||
|
||||
## Affected Files
|
||||
|
||||
- `webui/src/hooks/useSpectatorMode.js`
|
||||
- `webui/src/context/SocketContext.jsx`
|
||||
- `webui/src/context/SessionContext.jsx`
|
||||
- `server/src/services/authService/index.js`
|
||||
- `server/src/services/roverManager/socketHandlers.js`
|
||||
|
||||
## Evidence
|
||||
|
||||
Artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-11T03-14-34-168Z/report.json
|
||||
```
|
||||
|
||||
That 8x CPU-throttled profile captured repeated console errors:
|
||||
|
||||
```txt
|
||||
Failed to enter spectator mode Error: Spectator role required
|
||||
```
|
||||
|
||||
Fresh 6x runtime audit:
|
||||
|
||||
```txt
|
||||
consoleErrors: []
|
||||
auth:role events: 4
|
||||
```
|
||||
|
||||
So the race is not constant, but it exists under some timing conditions.
|
||||
|
||||
## Current Code Path
|
||||
|
||||
`useSpectatorMode` does this:
|
||||
|
||||
```txt
|
||||
webui/src/hooks/useSpectatorMode.js:18
|
||||
if (session?.role !== 'spectator') {
|
||||
await setRole('spectator');
|
||||
}
|
||||
|
||||
webui/src/hooks/useSpectatorMode.js:21
|
||||
await subscribeAll();
|
||||
```
|
||||
|
||||
The effect reruns when these change:
|
||||
|
||||
```txt
|
||||
webui/src/hooks/useSpectatorMode.js:36
|
||||
[connected, session?.mode, session?.role, setRole, subscribeAll]
|
||||
```
|
||||
|
||||
The server rejects `subscribeAll` unless the socket role is already spectator:
|
||||
|
||||
```txt
|
||||
server/src/services/roverManager/socketHandlers.js:161
|
||||
if (socket.data?.role !== 'spectator') {
|
||||
cb({ error: 'Spectator role required' });
|
||||
}
|
||||
```
|
||||
|
||||
The auth service can initialize a socket as spectator only if the socket handshake query
|
||||
requests it:
|
||||
|
||||
```txt
|
||||
server/src/services/authService/index.js:40
|
||||
const requestedRole = socket.handshake?.query?.role;
|
||||
|
||||
server/src/services/authService/index.js:41
|
||||
const initialRole = requestedRole === 'spectator' ? 'spectator' : 'user';
|
||||
```
|
||||
|
||||
If the spectate page connects as a normal user and then switches role after connect, slow
|
||||
timing can expose ordering issues.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
The retry itself is not a huge CPU cost. The damage is indirect:
|
||||
|
||||
- extra auth/session events
|
||||
- repeated `subscribeAll` attempts
|
||||
- delayed rover room joins
|
||||
- delayed video/snapshot setup
|
||||
- noisy console errors during profiling
|
||||
- worse startup behavior on weak devices
|
||||
|
||||
## Fix Strategies
|
||||
|
||||
### Option A: Connect `/spectate` With `role=spectator`
|
||||
|
||||
Make the spectator route establish its socket with a spectator role query from the start:
|
||||
|
||||
```txt
|
||||
io(..., { query: { role: 'spectator' } })
|
||||
```
|
||||
|
||||
This uses the existing server path in `authService`.
|
||||
|
||||
### Option B: Wait For Confirmed Role Before `subscribeAll`
|
||||
|
||||
After calling `setRole('spectator')`, wait until `auth:role` or session state confirms
|
||||
`role === 'spectator'`, then call `subscribeAll`.
|
||||
|
||||
Avoid calling `subscribeAll` in the same effect tick if the role update has not propagated.
|
||||
|
||||
### Option C: Add A Server-Side Atomic Spectator Enter Event
|
||||
|
||||
Create one event:
|
||||
|
||||
```txt
|
||||
session:enterSpectator
|
||||
```
|
||||
|
||||
Server behavior:
|
||||
|
||||
1. set socket role to spectator
|
||||
2. join all visible rover rooms
|
||||
3. return one ack
|
||||
|
||||
This removes client-side ordering risk.
|
||||
|
||||
### Option D: Debounce Or Guard Retries
|
||||
|
||||
If `subscribeAll` fails with `Spectator role required`, do not retry in a tight loop. Wait
|
||||
for an explicit role event or connection change.
|
||||
|
||||
## Recommended Path
|
||||
|
||||
The cleanest fix is Option A plus Option B:
|
||||
|
||||
1. Make the `/spectate` socket connect as `role=spectator` when possible.
|
||||
2. Keep `useSpectatorMode` as a fallback, but call `subscribeAll` only after confirmed
|
||||
spectator role.
|
||||
|
||||
If socket creation is shared in a way that makes route-specific query parameters awkward,
|
||||
use the atomic server event instead.
|
||||
|
||||
## Validation
|
||||
|
||||
Run the throttled profile several times:
|
||||
|
||||
```sh
|
||||
CPU_THROTTLE=8 VIEWPORT=390x844 MOBILE=1 \
|
||||
node perf/live-cpu-profile.mjs https://rover.otter.land/spectate perf/results
|
||||
```
|
||||
|
||||
Expected improvements:
|
||||
|
||||
- No `Failed to enter spectator mode` console errors.
|
||||
- Fewer `auth:role` events during startup.
|
||||
- More consistent startup timing.
|
||||
|
||||
## Risks
|
||||
|
||||
- Be careful not to accidentally make ordinary `/` users spectators.
|
||||
- If using route-specific socket query parameters, verify reconnects preserve the correct
|
||||
role.
|
||||
- In lockdown mode, spectator setup must still fail cleanly.
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# `/spectate` CPU Performance Issue Index
|
||||
|
||||
This folder breaks the live-site `/spectate` CPU investigation into separate,
|
||||
work-ready issues. These are intentionally separate from `perf/issues`, which is the
|
||||
current `/` backlog.
|
||||
|
||||
The testing target was:
|
||||
|
||||
```sh
|
||||
https://rover.otter.land/spectate
|
||||
```
|
||||
|
||||
Primary artifact:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-12T04-55-16-888Z-root-runtime/root-runtime-report.json
|
||||
```
|
||||
|
||||
Secondary artifacts:
|
||||
|
||||
```sh
|
||||
perf/results/2026-06-11T03-14-34-168Z/report.json
|
||||
perf/results/2026-06-11T03-11-39-596Z-route-sweep/route-sweep-report.json
|
||||
```
|
||||
|
||||
## Ranked Issues
|
||||
|
||||
1. [Telemetry fan-out and always-on HUD maps](./001-telemetry-fanout-hud-maps.md)
|
||||
2. [Multi-rover WebRTC media fan-out](./002-multi-rover-webrtc-media-fanout.md)
|
||||
3. [Room camera and snapshot object URL churn](./003-room-camera-snapshot-object-url-churn.md)
|
||||
4. [Broad spectator session rerenders](./004-broad-spectator-session-rerenders.md)
|
||||
5. [Shared log/session stream pressure](./005-shared-log-session-stream-pressure.md)
|
||||
6. [Spectator role handshake retries](./006-spectator-role-handshake-retries.md)
|
||||
|
||||
## Quick Read
|
||||
|
||||
The biggest `/spectate` CPU issue is the number of live visual things being updated at
|
||||
once. The page receives about 45 `sensorFrame` events/sec in the sampled run, and every
|
||||
rover card has an always-on HUD map. That produced thousands of SVG `path` mutations in
|
||||
30 seconds.
|
||||
|
||||
The second issue is live media fan-out. The audit saw 6 `RTCPeerConnections`, which is a
|
||||
lot for low-end laptops and mobile browsers, especially alongside React/HUD updates.
|
||||
|
||||
The third issue is snapshot/camera image churn. Room camera frames created 117 `img.src`
|
||||
mutations and the socket audit saw 11MB of binary payloads in 30 seconds.
|
||||
|
||||
Issues 004 and 005 are where the `/` backlog overlaps most. If the `/` work adds better
|
||||
session selectors, log batching, chat batching, and telemetry visual throttling, revisit
|
||||
these spectate writeups before implementing them from scratch.
|
||||
|
||||
Issue 006 is lower priority because it was intermittent, but it is worth fixing because
|
||||
failed spectator setup can create extra retries, console noise, and unpredictable startup
|
||||
work.
|
||||
|
||||
## Upstream Labels
|
||||
|
||||
Each issue includes an "Upstream Likelihood" section:
|
||||
|
||||
- `High`: likely mostly fixed by the earlier `/` work.
|
||||
- `Medium`: `/` work should help, but `/spectate` still needs targeted changes.
|
||||
- `Low`: mostly spectate-specific.
|
||||
|
||||
Reference in New Issue
Block a user