Compare commits

..
Author SHA1 Message Date
legop3 ea4af2ebb3 add config option for room camera interval
Container image / image (push) Failing after 1m5s
2026-09-29 22:18:45 -04:00
legop3 b2bbaf2085 slop 2026-09-29 22:14:57 -04:00
legop3 18e7fb8d1b slop
Container image / image (push) Failing after 56s
2026-09-28 23:38:19 -04:00
legop3 ba71106544 slop 2026-09-28 23:22:15 -04:00
legop3 a395656056 slop
Container image / image (push) Failing after 54s
2026-09-28 18:36:29 -04:00
legop3 50c2c1f5db slop 2026-09-28 18:15:37 -04:00
legop3 b59389870e slop 2026-09-28 17:43:39 -04:00
legop3 cd4aee4e74 slop 2026-09-28 17:43:23 -04:00
legop3 9de0bfb1e7 Merge pull request #30 from legop3/FEAT-rover-learner-model
Feat rover learner model
2026-09-28 15:46:40 -04:00
legop3 a157ece816 Merge pull request #29 from legop3/REFACTOR-participation-mode-ptz
Container image / image (push) Failing after 1m1s
Refactor participation mode ptz
2026-09-27 18:18:49 -04:00
legop3 8cb7cc5bdc Merge branch 'main' into REFACTOR-participation-mode-ptz
Container image / image (push) Failing after 55s
2026-09-27 18:11:37 -04:00
legop3 34936ab05c more queue related convergence for the future. 2026-09-27 17:19:56 -04:00
legop3 7231809140 gonna break stuff now
Container image / image (push) Failing after 51s
2026-09-24 18:36:24 -04:00
legop3 8ca714cf1a pause ptz turns when no ones on 2026-09-24 18:06:24 -04:00
legop3 ffd34de56c improving the UI side of ptz... and improving the way turns works with UI.... 2026-09-24 17:54:46 -04:00
legop3 846c9dae55 new internal concept: operating mode. will allow for more UI and separate pages to be made to operate things other than rovers. Converted PTZ to new system. 2026-09-22 14:42:21 -04:00
133 changed files with 7371 additions and 4064 deletions
+191 -15
View File
@@ -2,18 +2,21 @@
## Resume here ## Resume here
Latest change: one ongoing learner, gated Latest/Previous checkpoints, and Latest change (2026-09-29 IPI): format 4 seeds known user-control command shapes
admin-created permanent snapshots replace automatic permanent publications. before training. Generic peripherals are excluded; sensorStream is ignored without
invalidating footage. Drive includes zero-speed drive, with no separate stop type.
Drive, camera, brushes/vacuum, headlight, laser, horn, songs and operating modes
remain supported. Raw recordings and the existing checked command pipe stay intact.
Status: policy format 2 now predicts continuous wheel speeds plus accessory events. Held-out data cannot expand the schema. Unknown control variants exclude nearby
Copyable diagnostics and main-card wheel/action displays with a session threshold windows only, with command/count/reason diagnostics and a coverage gate that blocks
slider are implemented. Older model files remain stored but cannot run in this format. promotion. Supported portions still receive evaluation. Current training lineage
Recording, bounded dataset/training, permanent named models, live inference, is resume-v5.pt / schedule-v5.json. Startup removes obsolete format 1-3 models and
checked autonomous command submission, and the verified VIP card are coded. old lineages through v4; recordings and current-format models are preserved.
Human commands do not stop or pause autonomy (explicit latest user requirement).
Focused local checks passed; actual recording/training, live autonomous behavior, Verification for this increment is recorded at the end. Earlier version-specific
concurrent server performance, and browser appearance remain unverified. notes are historical. Manual inputs never pause/cancel autonomy; disabled service
The standalone CPU benchmark was run successfully on the actual server. remains dormant. Actual autonomous performance has not been established.
This document preserves the conversation's decisions across compactions and This document preserves the conversation's decisions across compactions and
usage cutoffs. Keep the checklist and handoff section current during authorized usage cutoffs. Keep the checklist and handoff section current during authorized
@@ -30,9 +33,8 @@ The user declined the manual server checklist and explicitly asked to continue.
Do not block further work or repeatedly ask them to do it. Preserve the unverified Do not block further work or repeatedly ask them to do it. Preserve the unverified
status and have the reader reject unusable recordings automatically. status and have the reader reject unusable recordings automatically.
Latest increment: internal card gating, dock lead-in/tail capture, live prediction Latest increment: see the command-event policy handoff at the end. Earlier
diagnostics, ongoing retained-data sampling, and gated model publication are coded. format 2 wheel-specific design notes below are historical and superseded.
See the final handoff section for settings and the new training lineage.
Next work: verify the integrated feature against actual recordings and the real Next work: verify the integrated feature against actual recordings and the real
camera/rover pipeline when that environment is available; inspect the VIP card in camera/rover pipeline when that environment is available; inspect the VIP card in
@@ -115,7 +117,7 @@ the default development path. Intended layout:
rover-learning/ rover-learning/
recordings/ Rolling video/events/session metadata (implemented) recordings/ Rolling video/events/session metadata (implemented)
models/ Permanent UUID directories: weights.pt and model.json models/ Permanent UUID directories: weights.pt and model.json
training/ resume-v3.pt, schedule-v3.json, and current job.json training/ resume-v4.pt, schedule-v4.json, and current job.json
``` ```
Training/model directories are created only when service and training are enabled. Training/model directories are created only when service and training are enabled.
@@ -786,3 +788,177 @@ no new training reset or configuration migration is required.
- Focused Node syntax and Python worker entry-point checks passed. Targeted card - Focused Node syntax and Python worker entry-point checks passed. Targeted card
lint is run for the UI change. End-to-end training/promotion, snapshot copying, lint is run for the UI change. End-to-end training/promotion, snapshot copying,
active-session pinning and browser behavior remain unverified without live data. active-session pinning and browser behavior remain unverified without live data.
### UI cleanup and obsolete models
User explicitly requested removal of UI narration and deletion of obsolete models.
Removed snapshot/threshold explanatory prose and the old commented manual-control
note; idle status now says Stopped. Kept operational status/errors and diagnostics.
Enabled-service startup now deletes permanent model directories whose validated
manifest explicitly identifies policy format 1, even when training is disabled.
Format 2 snapshots, Latest/Previous, recordings and current training state remain.
This supersedes earlier promises to preserve incompatible format 1 model files.
Deletion occurs on the actual server when the updated service starts; no production
files were accessed from this development machine.
### Recording maintenance race and diagnostic error details
Authorized by "fix it and fix that IPI" after a production report showed two
maintenance failures but omitted their error details. Scoped to those two defects;
evaluation/cache changes remain proposals, not part of this repair.
- listRecordings now tolerates ENOENT for directories/files removed between listing
and stat (dock-buffer pruning or atomic metadata rename). Other filesystem
errors still propagate; no broad error suppression or retries were added.
- Diagnostics reportVersion 2 includes actual redacted error messages, rejection
reasons, controller results, event detail, and worker stderr. Runtime catches
retain error name/code/syscall/path/stack, timestamp and operation. Recording
failures retain the same structured details through session finalization.
- A shared redactor removes URLs, credentials, user keys, email addresses and home
usernames while retaining filesystem operation context and stack frames. Error
categories remain supplementary, not replacements for actual failures.
- Node syntax checks passed for affected modules. No actual production recording
stream is available here; the race is fixed by inspection but cannot yet be
confirmed as the cause of the earlier production events. Previously discarded
error details cannot be reconstructed. No new tests or processes were created.
### Deliberate held-out selection
Latest IPI authorizes the scheduler correction only. Randomly shuffled eligible
sessions are now partitioned with exactly the worker's SHA256 first-eight-hex
modulo-five rule. Jobs select up to three training sessions and one held-out
session when both partitions exist; otherwise up to four training sessions.
No worker starts with held-out sessions alone. Existing split assignments, model
weights, and training progress are unchanged. Diagnostics expose timestamped
available/selected partition counts and the reason when evaluation cannot be
scheduled. Selection guarantees a held-out recording, not usable evaluation
windows: decoding/vocabulary/sample checks still apply in the worker.
Node syntax checks and diff whitespace checks passed; actual job selection on
production recordings remains unverified. No new tests or background processes.
### Active implementation: command-event policy (authorized IPI)
Supersedes format 2 continuous wheel output. The model predicts numeric command
scores and payload values; ordinary code constructs commands for the existing
shared command pipe. It does not generate JSON text. Drive, recorded key-release
stops, and accessories all use the threshold. Remove wheel-speed sensors and
previous drive-command inputs, but preserve raw recordings. Exact zero values
remain template constants so a demonstrated stop cannot regress into creeping.
Keep one continuing learner, Latest/Previous, admin snapshots, recording lifecycle,
permissions, and manual inputs that never cancel autonomy. No prescribed human
driving. Start a new format 3 lineage, reuse recordings, show older snapshots as
incompatible. No model-size increase or recording redesign in this change.
Update the upper card to show every command proposal with decoded parameters,
score, threshold marker and dispatch result, including below-threshold proposals.
Update copied diagnostics and evaluation for command events, parameter error,
missed stops and unwanted commands. Evaluate more than one held-out recording;
event validation is not proof of autonomous driving or physical rollout quality.
Progress: implemented; focused checks passed. User's unrelated webui/package.json
edit was preserved. No changes to runtime recording, disabled startup, control
permissions or manual-input override semantics.
Implementation details for resume:
- workers/policy.py VERSION=3, CNN/GRU unchanged in size; two heads for scores and
command parameters. Shared select_slots chooses one winner per command family
(peripheral id/control are separate families). No unconditional drive head.
- Numeric zeros and address fields stay exact template constants. Stop is the
normal recorded driveDirect {left:0,right:0} command. Unknown templates still
reject explicitly at the 64-shape limit. Arrays remain exact demonstrations.
- workers/dataset.py yields five arrays: images, state, event labels, parameters,
available shapes. It includes accepted human drive events, excludes ambiguous
conflicting events in the same 100ms window, and no longer requires an inferred
held wheel-speed target. Raw recorded sensors/commands are unchanged.
- Training reserves separate bounded samples for stop events, other commands and
no-command windows. Evaluation remains uniformly sampled. Scheduler selects
up to three training and two held-out recordings under the unchanged SHA split.
- Gate: complete evaluation, >=2 usable held-out sessions, >=128 windows, >=32
command events, >=8 stops, >=16 movement events; command precision/recall, stop
recall and movement recall each >=0.5; normalized numeric MSE <=0.1 where
applicable. Diagnostics expose each failing reason and per-command counts.
These are experimental screening thresholds, not a guarantee of useful driving.
- New optimizer/scheduler filenames prevent silently reusing format 2 weights or
old rejected-session bookkeeping. No config values need changing. Existing
permanent format 2 snapshots are preserved, but cannot start; attempting to
snapshot an old candidate reports that a new command-policy checkpoint is needed.
- Live inference keeps existing discrete toggle latching and common command limits.
The threshold applies to drive and stop just like the other command predictions.
No-event means send nothing: it DOES NOT implicitly stop an already moving rover.
It must learn explicit stop events; removing the copy shortcut is not proof that
learning will succeed. Other motion-related sensors still remain as observations.
- Upper card replaces separate wheel meters with generic scored proposals and
decoded parameters; shows applied threshold and post-clamp submitted parameters.
Copied diagnostics retain top eight proposals and up to 16 submissions per
sampled controller-history entry, including their command objects.
Verification: actual train.py --check-dependencies loaded Torch/NumPy/PyAV and
constructed format 3 (803,440 parameters); infer.py --help loaded successfully;
Node --check passed for six changed service modules; targeted card ESLint passed.
No new tests or throwaway programs. Live end-to-end training/inference and browser
visual verification require the actual environment and have not been performed.
Deferred explicitly: chronological commanded-motion replay/closed-loop evaluation,
prepared frame caching and larger architectures. Current evaluation uses held-out
100ms event windows with the live threshold/family selection, but cannot assess
temporal latching or accumulated motion from random reservoir samples. It does
not claim to verify actual physical command execution. After normal deployment,
let the new learner process retained recordings; copied reportVersion 3 diagnostics
provide the next evidence. Do not demand prescribed driving or a manual checklist.
### Obsolete model cleanup after format 3 transition
Latest user IPI explicitly authorizes deleting obsolete snapshots, checkpoints and
training state. This supersedes earlier format 2 preservation notes above.
Enabled-service startup, before trainer/driving/gateway startup, scans both models/
and training/checkpoints/. Identity-validated manifests with integer policy version
>=1 and < current POLICY_VERSION are removed. Current/future/unknown-format models
and recordings are preserved. Candidate/Latest/Previous references to obsolete
checkpoints are atomically removed before directory deletion; leased directories
are retired until release. Cleanup also removes legacy resume/schedule files and
temporary files for the unversioned and v1-v3 lineages. Current v4 state is kept.
Cleanup runs even when training is disabled, but not when the whole service is off.
Production deletion occurs on the next enabled-service startup after deployment.
Verification: focused Node syntax and diff whitespace checks passed. No production
storage was accessed; actual deletion awaits enabled-service startup on the server.
### User-control scope and evaluation coverage (2026-09-29)
Authorized by latest IPI after user clarified that controls include camera, brushes,
lights, laser, horn and songs but NOT generic peripherals. No new command pipe or
separate stop command. sensorStream is neither input/target nor a boundary. Actual
reset/reboot/update remain temporal boundaries because they interrupt observations.
Format 4 seeds drive and motor zero/nonzero field combinations, servo, light/laser
actions, the two four-frequency horn waveforms and horn release, OI start/safe/full/
dock bytes and default song. Seed values describe shapes, not artificial training
examples. Horn frequencies are numeric parameters; integral JSON numbers share
shapes with equivalent floating representations for drive/motors/servo/horn. Songs
remain exact demonstrated note arrays; other song/raw variants can extend the
training schema within its existing 64-slot bound. No promise to pre-enumerate all
possible tunes. Seeded slots receive negative examples even in sessions where that
control was not used, avoiding untrained random seeded outputs.
Evaluation freezes the training schema. Unsupported commands are reported with
payload, reason, occurrence count (up to 32 distinct descriptions), total count and
affected-window count. The affected observation history/target interval is excluded
without labeling the unknown command as inaction or skipping the entire recording.
Any unsupported held-out command blocks promotion; metrics describe only supported
windows. If none survive, dataset diagnostics still report the coverage problem.
Existing permission checks, threshold mechanics, recording and manual input behavior
are unchanged. Generic peripherals are also removed from execution allowlist and
rover compatibility profile. Previous policy models are automatically cleaned up;
no configuration edit is required.
Verification completed: train.py --check-dependencies loaded the production
worker and constructed format 4 with 803,440 parameters and predefined command
shapes. infer.py --help, Node syntax checks for the four changed service modules,
focused VipRoverLearningCard ESLint and diff whitespace checks passed.
No live server data/hardware/browser available locally. Actual training, driving,
and browser appearance remain unverified. No new tests or background processes.
Deploying starts a new learner using existing recordings; enabled-service startup
removes older policy models and optimizer/scheduler lineages through v4. Current
lineage is v5. No settings changes or prescribed human driving are required.
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2
View File
@@ -0,0 +1,2 @@
import{r as g,R as a}from"./index-5EWtcenj.js";const d="64px",u={},k=({src:s,light:t,oEmbedUrl:o,onClickPreview:n,playIcon:b,previewTabIndex:_,previewAriaLabel:p})=>{const[f,r]=g.useState(null);g.useEffect(()=>{!s||!t||!o||x({src:s,light:t,oEmbedUrl:o})},[s,t,o]);const x=async({src:e,light:i,oEmbedUrl:E})=>{if(a.isValidElement(i))return;if(typeof i=="string"){r(i);return}if(u[e]){r(u[e]);return}r(null);const m=await(await fetch(E.replace("{url}",e))).json();if(m.thumbnail_url){const h=m.thumbnail_url.replace("height=100","height=480").replace("-d_295x166","-d_640");r(h),u[e]=h}},I=e=>{(e.key==="Enter"||e.key===" ")&&n?.(e)},v=e=>{n?.(e)},l=a.isValidElement(t),y={display:"flex",alignItems:"center",justifyContent:"center"},c={preview:{width:"100%",height:"100%",backgroundImage:f&&!l?`url(${f})`:void 0,backgroundSize:"cover",backgroundPosition:"center",cursor:"pointer",...y},shadow:{background:"radial-gradient(rgb(0, 0, 0, 0.3), rgba(0, 0, 0, 0) 60%)",borderRadius:d,width:d,height:d,position:l?"absolute":void 0,...y},playIcon:{borderStyle:"solid",borderWidth:"16px 0 16px 26px",borderColor:"transparent transparent transparent white",marginLeft:"7px"}},w=a.createElement("div",{style:c.shadow,className:"react-player__shadow"},a.createElement("div",{style:c.playIcon,className:"react-player__play-icon"}));return a.createElement("div",{style:c.preview,className:"react-player__preview",tabIndex:_,onClick:v,onKeyDown:I,...p?{"aria-label":p}:{}},l?t:null,b||w)};var C=k;export{C as default};
//# sourceMappingURL=Preview-CKTJqKe-.js.map
File diff suppressed because one or more lines are too long
@@ -0,0 +1,2 @@
import{e as F,r as s,j as e,S as P,C as c,L as q}from"./index-5EWtcenj.js";import{o as $,p as D,q as I}from"./api-B8GGCEeo.js";function R(){const l=F(),[a,m]=s.useState(null),[n,b]=s.useState(""),[d,C]=s.useState(""),[p,N]=s.useState(""),[r,S]=s.useState(""),[f,v]=s.useState(""),[o,w]=s.useState(null),[x,h]=s.useState(!1),[g,i]=s.useState("");s.useEffect(()=>{$(l).then(t=>m(t.required)).catch(t=>i(t.message))},[l]);async function j(t){h(!0),i("");try{await t(),m(!1),i("Setup completed. You can now open the administration application and log in.")}catch(u){const E=Array.isArray(u.validationErrors)?` ${u.validationErrors.map(y=>`${y.path}: ${y.message}`).join("; ")}`:"";i(`${u.message}${E}`)}finally{h(!1)}}function k(t){if(t.preventDefault(),r!==f){i("Passwords do not match.");return}j(()=>D(l,{setupCode:n,username:d,discordId:p,password:r}))}function A(t){t.preventDefault(),o&&j(async()=>I(l,{setupCode:n,fileName:o.name,yaml:await o.text()}))}return e.jsxs("div",{className:"min-h-screen bg-neutral-950 p-1 text-slate-100",children:[e.jsx(P,{}),e.jsxs("main",{className:"mx-auto flex min-h-screen w-full max-w-3xl flex-col justify-center gap-0.5",children:[e.jsxs(c,{title:"MultiRover setup",meta:a===null?"checking":a?"required":"complete",bodyClassName:"space-y-0.5 p-1 text-sm",children:[a?e.jsx("p",{children:"Enter the one-time code from setup-code.txt in the server data folder, then create the first lockdown administrator or import an existing configuration."}):null,a===!1?e.jsx(q,{className:"button-dark inline-block",to:"/admin",children:"Open administration"}):null,g?e.jsx("p",{className:"surface p-1 text-sm text-slate-200",children:g}):null]}),a?e.jsxs(e.Fragment,{children:[e.jsxs(c,{title:"Setup authorization",bodyClassName:"p-1",children:[e.jsx("label",{className:"block text-xs font-semibold text-slate-200",children:"One-time setup code"}),e.jsx("input",{className:"field-input mt-0.5 w-full font-mono",value:n,onChange:t=>b(t.target.value)})]}),e.jsx(c,{title:"Create first administrator",bodyClassName:"p-1",children:e.jsxs("form",{className:"grid gap-0.5 md:grid-cols-2",onSubmit:k,children:[e.jsx("input",{className:"field-input",placeholder:"Username",value:d,onChange:t=>C(t.target.value)}),e.jsx("input",{className:"field-input",placeholder:"Discord id (optional)",value:p,onChange:t=>N(t.target.value)}),e.jsx("input",{className:"field-input",type:"password",placeholder:"Password",value:r,onChange:t=>S(t.target.value)}),e.jsx("input",{className:"field-input",type:"password",placeholder:"Confirm password",value:f,onChange:t=>v(t.target.value)}),e.jsx("button",{className:"button-dark md:col-span-2",type:"submit",disabled:x||!n||!d||!r,children:"Create lockdown administrator"})]})}),e.jsxs(c,{title:"Import configuration file",bodyClassName:"space-y-0.5 p-1 text-sm",children:[e.jsx("p",{className:"text-xs text-slate-400",children:"Choose an existing YAML configuration explicitly. The server validates and imports it once, and its secrets are never displayed back in the browser."}),e.jsxs("form",{className:"flex flex-col gap-0.5 md:flex-row",onSubmit:A,children:[e.jsx("input",{className:"field-input flex-1",type:"file",accept:".yaml,.yml,text/yaml",onChange:t=>w(t.target.files?.[0]||null)}),e.jsx("button",{className:"button-dark",type:"submit",disabled:x||!n||!o,children:"Import selected YAML"})]})]})]}):null]})]})}export{R as default};
//# sourceMappingURL=SetupApp-DN7XBkOe.js.map
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+35
View File
@@ -0,0 +1,35 @@
import{R as W}from"./index-5EWtcenj.js";const $=["abort","canplay","canplaythrough","durationchange","emptied","encrypted","ended","error","loadeddata","loadedmetadata","loadstart","pause","play","playing","progress","ratechange","seeked","seeking","stalled","suspend","timeupdate","volumechange","waiting","waitingforkey","resize","enterpictureinpicture","leavepictureinpicture","webkitbeginfullscreen","webkitendfullscreen","webkitpresentationmodechanged"],A=globalThis.document?.createElement("template");A&&(A.innerHTML=`
<style>
:host {
display: inline-block;
line-height: 0;
}
video,
audio {
max-width: 100%;
max-height: 100%;
min-width: 100%;
min-height: 100%;
}
</style>
<slot></slot>
`);const N=(i,{tag:t,is:s})=>{const d=globalThis.document?.createElement(t,{is:s}),f=d?S(d):[];return class g extends i{static Events=$;static template=A;static skipAttributes=[];static#t;static get observedAttributes(){return g.#l(),[...d?.constructor?.observedAttributes??[],"autopictureinpicture","disablepictureinpicture","disableremoteplayback","autoplay","controls","controlslist","crossorigin","loop","muted","playsinline","poster","preload","src"]}static#l(){if(this.#t)return;this.#t=!0;const e=new Set(this.observedAttributes);e.delete("muted");for(let o of f){if(o in this.prototype)continue;if(typeof d[o]=="function")this.prototype[o]=function(...r){this.#e();const a=()=>this.call?this.call(o,...r):this.nativeEl[o].apply(this.nativeEl,r);return this.loadComplete&&!this.isLoaded?this.loadComplete.then(a):a()};else{let r={get(){this.#e();let a=o.toLowerCase();if(e.has(a)){const p=this.getAttribute(a);return p===null?!1:p===""?!0:p}return this.get?.(o)??this.nativeEl?.[o]??this.#s[o]}};o!==o.toUpperCase()&&(r.set=async function(a){this.#e();let p=o.toLowerCase();if(e.has(p)){a===!0||a===!1||a==null?this.toggleAttribute(p,!!a):this.setAttribute(p,a);return}if(this.loadComplete&&!this.isLoaded&&await this.loadComplete,this.set){this.set(o,a);return}this.nativeEl[o]=a}),Object.defineProperty(this.prototype,o,r)}}}#o;#n;#r=!1;#i=!1;#a;#s;constructor(){super(),this.shadowRoot||(this.attachShadow({mode:"open"}),this.shadowRoot.append(this.constructor.template.content.cloneNode(!0))),this.load!==g.prototype.load&&(this.loadComplete=new O)}get loadComplete(){return this.#n}set loadComplete(e){this.#i=!1,this.#n=e,e?.then(()=>{this.#i=!0})}get isLoaded(){return this.#i}get nativeEl(){return this.#a??this.shadowRoot.querySelector(t)??this.querySelector(t)}set nativeEl(e){this.#a=e}get defaultMuted(){return this.hasAttribute("muted")}set defaultMuted(e){this.toggleAttribute("muted",!!e)}get src(){return this.getAttribute("src")}set src(e){this.setAttribute("src",`${e}`)}get preload(){return this.getAttribute("preload")??this.nativeEl?.preload}set preload(e){this.setAttribute("preload",`${e}`)}async#e(){if(this.#o)return;this.#o=!0,this.#u(),this.#c();for(let u of f)this.#d(u);const e=new Map,o=this.shadowRoot.querySelector("slot:not([name])");o?.addEventListener("slotchange",()=>{const u=new Map(e);o.assignedElements().filter(r=>["track","source"].includes(r.localName)).forEach(async r=>{u.delete(r);let a=e.get(r);a||(a=r.cloneNode(),e.set(r,a)),this.loadComplete&&!this.isLoaded&&await this.loadComplete,this.nativeEl.append?.(a)}),u.forEach(r=>r.remove())});for(let u of this.constructor.Events)this.shadowRoot.addEventListener?.(u,r=>{r.target===this.nativeEl&&this.dispatchEvent(new CustomEvent(r.type,{detail:r.detail}))},!0)}#d(e){if(Object.prototype.hasOwnProperty.call(this,e)){const o=this[e];delete this[e],this[e]=o}}#u(){const e=document.createElement(t,{is:s});e.muted=this.hasAttribute("muted");for(let{name:o,value:u}of this.attributes)e.setAttribute(o,u);this.#s={};for(let o of S(e))this.#s[o]=e[o];e.removeAttribute("src"),e.load()}async#c(){if(this.loadComplete&&!this.isLoaded&&await this.loadComplete,!this.nativeEl){const e=document.createElement(t,{is:s});e.part=t,this.shadowRoot.append(e)}this.nativeEl.muted=this.hasAttribute("muted")}attributeChangedCallback(e,o,u){this.#e(),e==="src"&&this.load!==g.prototype.load&&this.#h(),this.#p(e,o,u)}async#h(){this.#r&&(this.loadComplete=new O),this.#r=!0,await Promise.resolve(),await this.load(),this.loadComplete?.resolve(),await this.loadComplete}async#p(e,o,u){this.loadComplete&&!this.isLoaded&&await this.loadComplete,!["id","class",...this.constructor.skipAttributes].includes(e)&&(u===null?this.nativeEl.removeAttribute?.(e):this.nativeEl.setAttribute?.(e,u))}connectedCallback(){this.#e()}}};function S(i){let t=[];for(let s=Object.getPrototypeOf(i);s&&s!==HTMLElement.prototype;s=Object.getPrototypeOf(s))t.push(...Object.getOwnPropertyNames(s));return t}class O extends Promise{constructor(t=()=>{}){let s,d;super((f,c)=>{t(f,c),s=f,d=c}),this.resolve=s,this.reject=d}}const B=globalThis.document?N(HTMLElement,{tag:"video"}):class{};globalThis.document&&N(HTMLElement,{tag:"audio"});var H,R;const L=(H=globalThis.document)==null?void 0:H.createElement("template");L&&(L.innerHTML=`
<div class="wistia_embed"></div>
`);const P=(R=globalThis.document)==null?void 0:R.createElement("template");P&&(P.innerHTML=`
<style>
:host {
display: inline-block;
min-width: 300px;
min-height: 150px;
position: relative;
}
::slotted(.wistia_embed) {
position: absolute;
width: 100%;
height: 100%;
}
</style>
<slot></slot>
`);function D(i,t){if(Object.hasOwn(i,t)){const s=i[t];delete i[t],i[t]=s}}class q extends B{static template=P;static skipAttributes=["src"];#t=null;constructor(){super(),D(this,"config")}get config(){return this.#t}set config(t){this.#t=t}get nativeEl(){var t;return((t=this.api)==null?void 0:t.elem())??this.querySelector("video")}async load(){var t;if((t=this.querySelector(".wistia_embed"))==null||t.remove(),!this.src)return;await new Promise(o=>setTimeout(o,50));const s=/(?:wistia\.com|wi\.st)\/(?:medias|embed)\/(.*)$/i,d=this.src.match(s)[1],f={autoPlay:this.autoplay,silentAutoPlay:this.autoplay?"allow":void 0,preload:this.preload??"metadata",playsinline:this.playsInline,endVideoBehavior:this.loop?"loop":void 0,chromeless:!this.controls,playButton:this.controls,muted:this.defaultMuted,...this.#t??{}};this.append(L.content.cloneNode(!0));const c=this.querySelector(".wistia_embed");c.id||(c.id=z(d)),c.classList.add(`wistia_async_${d}`),await F("https://fast.wistia.com/assets/external/E-v1.js","Wistia"),this.api=await new Promise(o=>{globalThis._wq.push({id:c.id,onReady:o,options:f})})}async attributeChangedCallback(t,s,d){if(t==="controls"){switch(await this.loadComplete,t){case"controls":this.api.bigPlayButtonEnabled(this.controls),this.controls?this.api.releaseChromeless():this.api.requestChromeless();break}return}super.attributeChangedCallback(t,s,d)}get duration(){var t;return(t=this.api)==null?void 0:t.duration()}async play(){return await this.loadComplete,this.api.play(),new Promise(t=>this.addEventListener("playing",t))}}const T={};async function F(i,t){if(T[i])return T[i];if(self[t])return self[t];const s=new Promise((d,f)=>{const c=document.createElement("script");c.defer=!0,c.src=i,c.onload=()=>d(self[t]),c.onerror=f,document.head.append(c)});return T[i]=s,s}let U=0;function z(i){const t=++U;return`${i}${t}`}globalThis.customElements&&!globalThis.customElements.get("wistia-video")&&globalThis.customElements.define("wistia-video",q);var V=q,G=new Set(["style","children","ref","key","suppressContentEditableWarning","suppressHydrationWarning","dangerouslySetInnerHTML"]),J={className:"class",htmlFor:"for"};function I(i){return i.toLowerCase()}function j(i){if(typeof i=="boolean")return i?"":void 0;if(typeof i!="function"&&!(typeof i=="object"&&i!==null))return i}function K({react:i,tagName:t,elementClass:s,events:d,displayName:f,defaultProps:c,toAttributeName:e=I,toAttributeValue:o=j}){const u=Number.parseInt(i.version)>=19,r=i.forwardRef((a,p)=>{var M,k;const m=i.useRef(null),E=i.useRef(new Map),C={},_={},v={},w={};for(const[n,l]of Object.entries(a)){if(G.has(n)){v[n]=l;continue}const h=e(J[n]??n);if(s.prototype&&n in s.prototype&&!(n in(((M=globalThis.HTMLElement)==null?void 0:M.prototype)??{}))&&!((k=s.observedAttributes)!=null&&k.some(b=>b===h))){w[n]=l;continue}if(n.startsWith("on")){C[n]=l;continue}const y=o(l);if(h&&y!=null&&(_[h]=String(y),u||(v[h]=y)),h&&u){const b=j(l);y!==b?v[h]=y:v[h]=l}}if(typeof window<"u"){for(const n in C){const l=C[n],h=n.endsWith("Capture"),y=(d?.[n]??n.slice(2).toLowerCase()).slice(0,h?-7:void 0);i.useLayoutEffect(()=>{const b=m?.current;if(!(!b||typeof l!="function"))return b.addEventListener(y,l,h),()=>{b.removeEventListener(y,l,h)}},[m?.current,l])}i.useLayoutEffect(()=>{if(m.current===null)return;const n=new Map;for(const l in w)x(m.current,l,w[l]),E.current.delete(l),n.set(l,w[l]);for(const[l,h]of E.current)x(m.current,l,void 0);E.current=n})}if(typeof window>"u"&&s?.getTemplateHTML&&s?.shadowRootOptions){const{mode:n,delegatesFocus:l}=s.shadowRootOptions,h=i.createElement("template",{shadowrootmode:n,shadowrootdelegatesfocus:l,dangerouslySetInnerHTML:{__html:s.getTemplateHTML(_,a)},key:"ce-la-react-ssr-template-shadow-root"});v.children=[h,v.children]}return i.createElement(t,{...c,...v,ref:i.useCallback(n=>{m.current=n,typeof p=="function"?p(n):p!==null&&(p.current=n)},[p])},v.children)});return r.displayName=f??s.name,r}function x(i,t,s){var d;i[t]=s,s==null&&t in(((d=globalThis.HTMLElement)==null?void 0:d.prototype)??{})&&i.removeAttribute(t)}var X=K({react:W,tagName:"wistia-video",elementClass:V,toAttributeName(i){return i==="muted"?"":i==="defaultMuted"?"muted":I(i)}});export{X as default};
//# sourceMappingURL=react-DxdYv-AY.js.map
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+7
View File
@@ -0,0 +1,7 @@
import{_ as N,R as A}from"./index-5EWtcenj.js";import{M as D,C as S}from"./mixin-BY5xW4uI.js";class P extends D(S){static shadowRootOptions={...S.shadowRootOptions};static getTemplateHTML=i=>{const{src:n,...o}=i;return S.getTemplateHTML(o)};#t;attributeChangedCallback(i,n,o){i!=="src"&&super.attributeChangedCallback(i,n,o),i==="src"&&n!=o&&this.load()}async _initThumbnails(i){const n=async(d,m)=>{const v=[],g=i.timescale||1,h=i.startNumber||1,E=i.presentationTimeOffset?i.presentationTimeOffset/g:0,_=i.segmentDuration;for(let l=0;l<d;l++){const T=C({thIndex:l,thduration:m,ttiles:d,tduration:_,startNumber:h,pto:E}),w=T+m,M=new Promise((f,y)=>{this.api.provideThumbnail(T,({url:a,width:s,height:c,x:p,y:b})=>{try{const R=new VTTCue(T,w,`${a}#xywh=${p},${b},${s},${c}`);f(R)}catch(R){y(R)}})});v.push(M)}return await Promise.all(v).catch(l=>console.error("Error processing thumbnails",l))},{totalThumbnails:o,thumbnailDuration:e}=$(i),u=await n(o,e);let r=this.nativeEl.querySelector('track[label="thumbnails"]');if(!r){r=O(),this.nativeEl.appendChild(r);const d=x(u);r.src=d,r.dispatchEvent(new Event("change"))}}async load(){if(this.#t){this.api.attachSource(this.src);return}this.#t=!0;const i=await N(()=>import("./dash.all.min-C9gxBrL1.js"),[]);this.api=i.MediaPlayer().create(),this.api.initialize(this.nativeEl,this.src,this.autoplay),this.api.on(i.MediaPlayer.events.STREAM_INITIALIZED,()=>{const n=this.api.getRepresentationsByType("video");let o=this.videoTracks.getTrackById("main");o||(o=this.addVideoTrack("main"),o.id="main",o.selected=!0),n.forEach(e=>{const u=e.bandwidth??e.bitrate??(Number.isFinite(e.bitrateInKbit)?e.bitrateInKbit*1e3:void 0),r=o.addRendition(e.id,e.width,e.height,e.mimeType??e.codec,u);r.id=e.id}),this.videoRenditions.addEventListener("change",()=>{const e=this.videoRenditions[this.videoRenditions.selectedIndex];e?.id?(this.api.updateSettings({streaming:{abr:{autoSwitchBitrate:{video:!1}}}}),this.api.setRepresentationForTypeById("video",e.id,!0)):this.api.updateSettings({streaming:{abr:{autoSwitchBitrate:{video:!0}}}})}),this.api.isDynamic()||this.api.getRepresentationsByType("image").forEach(async(u,r)=>{r>0||this._initThumbnails(u)})})}}function $(t){var i,n;const o=t.essentialProperties[0],[e,u]=o.value.split("x").map(Number),r=e*u,d=((n=(i=t.adaptation)==null?void 0:i.period)==null?void 0:n.duration)||null,m=t.segmentDuration,v=t.timescale||1,h=m/v/r;return{totalThumbnails:d!=null?Math.ceil(d/h):Math.ceil(m/h),thumbnailDuration:h}}function C({thIndex:t,tduration:i,thduration:n,ttiles:o,startNumber:e,pto:u}){const r=Math.floor(t/o)+e,d=t%o+1,m=(r-1)*i-u,v=(d-1)*n;return m+v}function O(){const t=document.createElement("track");return t.kind="metadata",t.label="thumbnails",t.srclang="en",t.mode="hidden",t.default=!0,t}function x(t){let i=`WEBVTT
`;for(const e of t)i+=`${o(e.startTime)} --> ${o(e.endTime)}
`,i+=`${e.text}
`;const n=new Blob([i],{type:"text/vtt"});return URL.createObjectURL(n);function o(e){const u=String(Math.floor(e/3600)).padStart(2,"0"),r=String(Math.floor(e%3600/60)).padStart(2,"0"),d=(e%60).toFixed(3).padStart(6,"0");return`${u}:${r}:${d}`}}globalThis.customElements&&!globalThis.customElements.get("dash-video")&&globalThis.customElements.define("dash-video",P);var B=P,H=new Set(["style","children","ref","key","suppressContentEditableWarning","suppressHydrationWarning","dangerouslySetInnerHTML"]),F={className:"class",htmlFor:"for"};function I(t){return t.toLowerCase()}function k(t){if(typeof t=="boolean")return t?"":void 0;if(typeof t!="function"&&!(typeof t=="object"&&t!==null))return t}function W({react:t,tagName:i,elementClass:n,events:o,displayName:e,defaultProps:u,toAttributeName:r=I,toAttributeValue:d=k}){const m=Number.parseInt(t.version)>=19,v=t.forwardRef((g,h)=>{var E,_;const l=t.useRef(null),T=t.useRef(new Map),w={},M={},f={},y={};for(const[a,s]of Object.entries(g)){if(H.has(a)){f[a]=s;continue}const c=r(F[a]??a);if(n.prototype&&a in n.prototype&&!(a in(((E=globalThis.HTMLElement)==null?void 0:E.prototype)??{}))&&!((_=n.observedAttributes)!=null&&_.some(b=>b===c))){y[a]=s;continue}if(a.startsWith("on")){w[a]=s;continue}const p=d(s);if(c&&p!=null&&(M[c]=String(p),m||(f[c]=p)),c&&m){const b=k(s);p!==b?f[c]=p:f[c]=s}}if(typeof window<"u"){for(const a in w){const s=w[a],c=a.endsWith("Capture"),p=(o?.[a]??a.slice(2).toLowerCase()).slice(0,c?-7:void 0);t.useLayoutEffect(()=>{const b=l?.current;if(!(!b||typeof s!="function"))return b.addEventListener(p,s,c),()=>{b.removeEventListener(p,s,c)}},[l?.current,s])}t.useLayoutEffect(()=>{if(l.current===null)return;const a=new Map;for(const s in y)L(l.current,s,y[s]),T.current.delete(s),a.set(s,y[s]);for(const[s,c]of T.current)L(l.current,s,void 0);T.current=a})}if(typeof window>"u"&&n?.getTemplateHTML&&n?.shadowRootOptions){const{mode:a,delegatesFocus:s}=n.shadowRootOptions,c=t.createElement("template",{shadowrootmode:a,shadowrootdelegatesfocus:s,dangerouslySetInnerHTML:{__html:n.getTemplateHTML(M,g)},key:"ce-la-react-ssr-template-shadow-root"});f.children=[c,f.children]}return t.createElement(i,{...u,...f,ref:t.useCallback(a=>{l.current=a,typeof h=="function"?h(a):h!==null&&(h.current=a)},[h])},f.children)});return v.displayName=e??n.name,v}function L(t,i,n){var o;t[i]=n,n==null&&i in(((o=globalThis.HTMLElement)==null?void 0:o.prototype)??{})&&t.removeAttribute(i)}var V=W({react:A,tagName:"dash-video",elementClass:B,toAttributeName(t){return t==="muted"?"":t==="defaultMuted"?"muted":I(t)}});export{V as default};
//# sourceMappingURL=react-gdx3v4jU.js.map
File diff suppressed because one or more lines are too long
+12 -2
View File
@@ -7,6 +7,7 @@ const logger = require('../../globals/logger').child('assignment');
const { MODES, getMode, modeEvents } = require('../modeManager'); const { MODES, getMode, modeEvents } = require('../modeManager');
const { roleEvents, getRole, isAdmin, isLockdownAdmin } = require('../roleService'); const { roleEvents, getRole, isAdmin, isLockdownAdmin } = require('../roleService');
const roverManager = require('../roverManager'); const roverManager = require('../roverManager');
const { getOperatingMode } = require('../operatingModeService');
const { compareRoversForAssignment } = require('./roverRanking'); const { compareRoversForAssignment } = require('./roverRanking');
const socketRefs = new Map(); // socketId -> socket const socketRefs = new Map(); // socketId -> socket
@@ -131,7 +132,7 @@ roverManager.managerEvents.on('switch', ({ socketId, roverId }) => {
}); });
function assignSocket(socket, options = {}) { function assignSocket(socket, options = {}) {
if (!socket || isAdmin(socket) || getRole(socket) !== 'user') { if (!socket || getOperatingMode(socket) !== 'rover' || isAdmin(socket) || getRole(socket) !== 'user') {
return; return;
} }
// avoid double assignment // avoid double assignment
@@ -311,13 +312,22 @@ function describeAssignment(socketId) {
module.exports = { module.exports = {
assignmentEvents, assignmentEvents,
resumeAssignment: assignSocket,
releaseForOperatingMode: (socket) => {
// Include direct admin control as well as automatic assignment ownership.
const roverIds = new Set(roverManager.getRoversForSocket(socket.id));
const assigned = assignments.get(socket.id);
if (assigned) roverIds.add(assigned);
waiting.delete(socket.id);
roverIds.forEach((roverId) => forceRelease(roverId, socket.id));
},
describeAssignment, describeAssignment,
forceRelease, forceRelease,
forceReleaseWithNotice, forceReleaseWithNotice,
rerollAssignments, rerollAssignments,
getAssignedRover: (socketId) => assignments.get(socketId) || null, getAssignedRover: (socketId) => assignments.get(socketId) || null,
moveAssignment: (socket, roverId, { releasePrevious = true } = {}) => { moveAssignment: (socket, roverId, { releasePrevious = true } = {}) => {
if (!socket || !roverId) return; if (!socket || !roverId || getOperatingMode(socket) !== 'rover') return;
const previous = assignments.get(socket.id); const previous = assignments.get(socket.id);
if (previous && previous !== roverId && releasePrevious) { if (previous && previous !== roverId && releasePrevious) {
roverManager.releaseControl(previous, socket); roverManager.releaseControl(previous, socket);
@@ -2,20 +2,14 @@
// Purpose: Builds normalized chat message/typing payloads and rover context snapshots. // Purpose: Builds normalized chat message/typing payloads and rover context snapshots.
// Scope: Encapsulates chat DTO construction and rover/driver metadata extraction. // Scope: Encapsulates chat DTO construction and rover/driver metadata extraction.
const { v4: uuidv4 } = require('uuid'); const { v4: uuidv4 } = require('uuid');
const io = require('../../globals/io');
const roverManager = require('../roverManager'); const roverManager = require('../roverManager');
const { getRole } = require('../roleService'); const { getRole } = require('../roleService');
const { getOperatingMode } = require('../operatingModeService');
const { describeAssignment } = require('../assignmentService'); const { describeAssignment } = require('../assignmentService');
const { getNickname } = require('../nicknameService'); const { getNickname } = require('../nicknameService');
const ptzCameraService = require('../ptzCameraService'); const ptzCameraService = require('../ptzCameraService');
function resolvePtzChatTarget(socketId) {
return ptzCameraService.getChatTargetForSocket(socketId) || null;
}
function resolveRoverId(socketId) { function resolveRoverId(socketId) {
const ptzTarget = resolvePtzChatTarget(socketId);
if (ptzTarget?.roverId) return ptzTarget.roverId;
const primary = roverManager.getPrimaryRoverForSocket(socketId); const primary = roverManager.getPrimaryRoverForSocket(socketId);
if (primary) return primary; if (primary) return primary;
const assignment = describeAssignment(socketId); const assignment = describeAssignment(socketId);
@@ -24,54 +18,22 @@ function resolveRoverId(socketId) {
function resolveRoverColor(roverId) { function resolveRoverColor(roverId) {
if (!roverId) return null; if (!roverId) return null;
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
return ptzCameraService.getPublicState()?.color || null;
}
const record = roverManager.rovers.get(String(roverId)); const record = roverManager.rovers.get(String(roverId));
return record?.meta?.color || null; return record?.meta?.color || null;
} }
function resolveRoverName(roverId) { function resolveRoverName(roverId) {
if (!roverId) return null; if (!roverId) return null;
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
return ptzCameraService.getPublicState()?.name || null;
}
const record = roverManager.rovers.get(String(roverId)); const record = roverManager.rovers.get(String(roverId));
return record?.meta?.name || null; return record?.meta?.name || null;
} }
function isPtzChatTargetId(roverId) {
/*
PTZ is intentionally treated as a virtual rover for chat identity only. It
does not live in roverManager.rovers because movement, video authorization,
and queue ownership are PTZ-service concerns, but chat needs one stable
"rover-like" id so the existing web UI, Discord bridge, and AI transcript
code can all render the same badge without learning PTZ internals.
*/
return Boolean(roverId) && String(roverId) === ptzCameraService.PTZ_CAMERA_ID;
}
function isPublicChatTargetId(roverId, socket = null) { function isPublicChatTargetId(roverId, socket = null) {
if (!roverId) return false; return Boolean(roverId) && roverManager.canReplayRoverId(roverId, socket) === true;
/*
Normal rovers remain governed by the existing replay visibility rule, which
is also the rule chat historically used to avoid exposing closed private
rover activity. PTZ gets an explicit allow-list entry here because it is a
public chat target that deliberately pretends to be a rover, even though it
is not a roverManager record.
*/
if (isPtzChatTargetId(roverId)) return true;
return roverManager.canReplayRoverId(roverId, socket) === true;
} }
function isPrivateClosedRoverId(roverId) { function isPrivateClosedRoverId(roverId) {
if (!roverId) return false; return Boolean(roverId) && !isPublicChatTargetId(roverId);
/*
PTZ uses the existing rover badge fields so chat rows can reuse RoverLabel,
but it is not a private rover. Let PTZ-badged messages broadcast normally
instead of falling into the closed-private rover path for unknown ids.
*/
return !isPublicChatTargetId(roverId);
} }
function normalizeProfileImageUrl(value) { function normalizeProfileImageUrl(value) {
@@ -170,6 +132,8 @@ function buildMessage(socket, text, meta = {}) {
socketId: socket?.id || null, socketId: socket?.id || null,
nickname: meta.nickname || getNickname(socket) || null, nickname: meta.nickname || getNickname(socket) || null,
role: meta.role || getRole(socket), role: meta.role || getRole(socket),
operatingMode: socket ? getOperatingMode(socket) : null,
operatingModeDisplay: socket ? ptzCameraService.getOperatingModeDisplay(socket.id) : null,
roverId, roverId,
roverName, roverName,
roverColor, roverColor,
@@ -215,6 +179,8 @@ function buildTypingPayload(socket, meta = {}) {
socketId, socketId,
nickname: meta.nickname || getNickname(socket) || null, nickname: meta.nickname || getNickname(socket) || null,
role: meta.role || getRole(socket), role: meta.role || getRole(socket),
operatingMode: socket ? getOperatingMode(socket) : null,
operatingModeDisplay: socket ? ptzCameraService.getOperatingModeDisplay(socket.id) : null,
roverId, roverId,
roverName, roverName,
roverColor, roverColor,
@@ -231,7 +197,6 @@ function buildTypingPayload(socket, meta = {}) {
module.exports = { module.exports = {
resolveRoverId, resolveRoverId,
isPtzChatTargetId,
isPublicChatTargetId, isPublicChatTargetId,
isPrivateClosedRoverId, isPrivateClosedRoverId,
buildRoverCtxSnapshot, buildRoverCtxSnapshot,
@@ -19,13 +19,6 @@ const { getLastAccessNoticeAt, setLastAccessNoticeAt } = require('./state');
function playTypingNote(roverId, note, socketId) { function playTypingNote(roverId, note, socketId) {
if (!roverId) return; if (!roverId) return;
/*
PTZ borrows the roverId field for chat badges, but it has no rover command
channel. Skipping the song command here keeps PTZ chat from producing noisy
"unknown rover" command attempts while still allowing the message itself to
behave like rover chat everywhere else.
*/
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) return;
try { try {
issueCommand(roverId, { issueCommand(roverId, {
type: 'song', type: 'song',
@@ -90,8 +83,8 @@ function maybeSendAccessNotice(message, sendSystemMessage) {
} }
function maybeSpeak(socket, message, ttsOptions) { function maybeSpeak(socket, message, ttsOptions) {
if (!ttsOptions || !message?.roverId) return; if (!ttsOptions || !message) return;
if (String(message.roverId) === ptzCameraService.PTZ_CAMERA_ID) { if (message.operatingMode === 'ptz') {
/* /*
PTZ has no rover websocket, but it does have a real speaker behind the PTZ has no rover websocket, but it does have a real speaker behind the
Reolink/neolink path. Keep PTZ routing here so chat remains the single Reolink/neolink path. Keep PTZ routing here so chat remains the single
@@ -107,6 +100,7 @@ function maybeSpeak(socket, message, ttsOptions) {
}); });
return; return;
} }
if (!message.roverId) return;
const record = roverManager.rovers.get(message.roverId); const record = roverManager.rovers.get(message.roverId);
const ttsEnabled = Boolean(record?.meta?.audio?.ttsEnabled); const ttsEnabled = Boolean(record?.meta?.audio?.ttsEnabled);
if (!ttsEnabled) return; if (!ttsEnabled) return;
@@ -59,12 +59,6 @@ function createChatBridgeHandlers(deps) {
function handleChatBridgeOutbound(event) { function handleChatBridgeOutbound(event) {
const payload = event?.payload; const payload = event?.payload;
if (!payload) return; if (!payload) return;
/*
Outbound bridge filtering must use chat visibility, not rover replay
visibility. PTZ deliberately uses roverId: "ptz-camera" so the existing
chat badge path can be reused, but that id is not a roverManager rover and
would be dropped by canReplayRoverId().
*/
if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return; if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return;
const guildConfigs = listGuildConfigs(); const guildConfigs = listGuildConfigs();
if (!guildConfigs.length) return; if (!guildConfigs.length) return;
@@ -24,14 +24,8 @@ function formatWebhookUsername(payload) {
const origin = payload.discordGuildName ? ` (From: ${payload.discordGuildName})` : ''; const origin = payload.discordGuildName ? ` (From: ${payload.discordGuildName})` : '';
return `${name}${origin}${botTag}${spectatorTag}${adminTag}`; return `${name}${origin}${botTag}${spectatorTag}${adminTag}`;
} }
/* const tagLabel = payload.operatingModeDisplay?.name || payload.roverName || payload.roverId;
The chat payload already carries the resolved display name for rover-like const roverTag = tagLabel ? ` [${tagLabel}]` : '';
targets. Prefer that name so PTZ, which is intentionally pretending to be a
rover in chat, shows up as "PTZ Camera" instead of the internal id
"ptz-camera"; fall back to the id for older payloads or missing metadata.
*/
const roverTagLabel = payload.roverName || payload.roverId;
const roverTag = payload.roverId ? ` [${roverTagLabel}]` : '';
return `${name}${botTag}${spectatorTag}${adminTag}${roverTag}`; return `${name}${botTag}${spectatorTag}${adminTag}${roverTag}`;
} }
@@ -10,7 +10,8 @@ const { loadConfig, getFeatureFlags, registerConfigurationHandler } = require('.
const { getConfiguredSocials } = require('../sessionService/configuration'); const { getConfiguredSocials } = require('../sessionService/configuration');
const { getMode, MODES } = require('../modeManager'); const { getMode, MODES } = require('../modeManager');
const roverManager = require('../roverManager'); const roverManager = require('../roverManager');
const { getTurnQueues } = require('../turnService'); const { getTurnQueues, getActiveDrivers } = require('../turnService');
const { buildRoverTurn } = require('../turnService/display');
const { getRoomCameras, getRoomCameraState } = require('../roomCameraService'); const { getRoomCameras, getRoomCameraState } = require('../roomCameraService');
const { getRoverSnapshotState } = require('../roverSnapshotService'); const { getRoverSnapshotState } = require('../roverSnapshotService');
const { getRole } = require('../roleService'); const { getRole } = require('../roleService');
@@ -157,22 +158,6 @@ function publicRoverIdSet(roster = []) {
return new Set(roster.map((rover) => String(rover?.id || '')).filter(Boolean)); return new Set(roster.map((rover) => String(rover?.id || '')).filter(Boolean));
} }
function filterPublicTurnQueues(turnQueues = {}, publicIds) {
const visible = publicIds instanceof Set ? publicIds : new Set();
const next = {};
/*
RoverQueuesPanel creates fallback rows for queue ids that are not present in
the roster, so the public payload must filter queues with the exact same
privacy boundary as the roster. Otherwise a closed-private rover can leak as
an orphan queue row after a private access grant assigns someone to it.
*/
Object.entries(turnQueues || {}).forEach(([roverId, info]) => {
if (!visible.has(String(roverId))) return;
next[roverId] = info;
});
return next;
}
function filterPublicUsers(users = [], publicIds) { function filterPublicUsers(users = [], publicIds) {
const visible = publicIds instanceof Set ? publicIds : new Set(); const visible = publicIds instanceof Set ? publicIds : new Set();
/* /*
@@ -203,6 +188,8 @@ function buildLocalInfo() {
const publicIds = publicRoverIdSet(publicRoster); const publicIds = publicRoverIdSet(publicRoster);
const roster = publicRoster.map((rover) => (lockdown ? rover : addRoverSnapshotLinks(rover))); const roster = publicRoster.map((rover) => (lockdown ? rover : addRoverSnapshotLinks(rover)));
const users = filterPublicUsers(Array.from(io.sockets.sockets.values()).map(buildUserEntry), publicIds); const users = filterPublicUsers(Array.from(io.sockets.sockets.values()).map(buildUserEntry), publicIds);
const turnQueues = getTurnQueues();
const activeDrivers = getActiveDrivers();
const roomCameras = lockdown || !features.roomCameras ? [] : getRoomCameras().map(buildRoomCameraInfo); const roomCameras = lockdown || !features.roomCameras ? [] : getRoomCameras().map(buildRoomCameraInfo);
return { return {
instance: { instance: {
@@ -212,8 +199,12 @@ function buildLocalInfo() {
features, features,
updatedAt: Date.now(), updatedAt: Date.now(),
}, },
roster, // Only public roster entries reach the builder; queue metadata follows the
turnQueues: filterPublicTurnQueues(getTurnQueues(), publicIds), // same visibility boundary instead of creating separate orphan queue rows.
roster: roster.map((rover) => ({
...rover,
turn: buildRoverTurn({ rover, mode, socketId: null, turnInfo: turnQueues[rover.id], activeDriverId: activeDrivers[rover.id], users }),
})),
users, users,
roomCameras, roomCameras,
socials: features.socials ? getConfiguredSocials(config) : [], socials: features.socials ? getConfiguredSocials(config) : [],
@@ -349,7 +340,6 @@ function normalizeRemotePayload(entry, payload) {
features: instance.features && typeof instance.features === 'object' ? instance.features : {}, features: instance.features && typeof instance.features === 'object' ? instance.features : {},
}, },
roster: Array.isArray(payload?.roster) ? payload.roster : [], roster: Array.isArray(payload?.roster) ? payload.roster : [],
turnQueues: payload?.turnQueues && typeof payload.turnQueues === 'object' ? payload.turnQueues : {},
users: Array.isArray(payload?.users) ? payload.users : [], users: Array.isArray(payload?.users) ? payload.users : [],
roomCameras: Array.isArray(payload?.roomCameras) ? payload.roomCameras : [], roomCameras: Array.isArray(payload?.roomCameras) ? payload.roomCameras : [],
socials: Array.isArray(payload?.socials) ? payload.socials : [], socials: Array.isArray(payload?.socials) ? payload.socials : [],
@@ -419,7 +409,6 @@ function markOffline(entry, error) {
publicUrl: previous.instance?.publicUrl || entry.url, publicUrl: previous.instance?.publicUrl || entry.url,
}, },
roster: previous.roster || [], roster: previous.roster || [],
turnQueues: previous.turnQueues || {},
users: previous.users || [], users: previous.users || [],
roomCameras: previous.roomCameras || [], roomCameras: previous.roomCameras || [],
socials: previous.socials || [], socials: previous.socials || [],
@@ -370,11 +370,6 @@ function createSnapshotEngine(deps) {
.filter((entry) => { .filter((entry) => {
const roverId = entry?.roverId ? String(entry.roverId) : null; const roverId = entry?.roverId ? String(entry.roverId) : null;
if (!roverId) return true; if (!roverId) return true;
/*
This is a chat transcript filter, not a physical-rover filter. PTZ
chat intentionally carries a rover-like id so transcript consumers can
render it consistently, even though roverManager cannot replay that id.
*/
return isPublicChatTargetId(roverId); return isPublicChatTargetId(roverId);
}); });
const chatRecent = allRecentMessages const chatRecent = allRecentMessages
@@ -430,6 +425,8 @@ function createSnapshotEngine(deps) {
type: 'chat', type: 'chat',
nickname: entry.nickname || entry.socketId?.slice(0, 6) || 'unknown', nickname: entry.nickname || entry.socketId?.slice(0, 6) || 'unknown',
text: entry.text || '', text: entry.text || '',
operating_mode: entry.operatingMode || null,
operating_mode_name: entry.operatingModeDisplay?.name || null,
rover_id: roverId, rover_id: roverId,
rover_ctx: { ...baseCtx, ...storedCtx }, rover_ctx: { ...baseCtx, ...storedCtx },
}; };
@@ -0,0 +1,74 @@
const EventEmitter = require('events');
const operatingModeEvents = new EventEmitter();
const transitions = new WeakMap();
function getOperatingMode(socket) {
// Handshake intent reserves the mode before role-driven assignment runs.
// It grants no camera permissions; entry still checks access and readiness.
return socket?.data?.operatingMode
|| (socket?.handshake?.auth?.operatingMode === 'ptz' ? 'ptz' : 'rover');
}
async function transition(socket, operatingMode) {
// Resolve services at the action boundary to avoid a dependency cycle with
// assignment and camera policy, which both read operating mode.
const assignment = require('../assignmentService');
const roverManager = require('../roverManager');
const ptz = require('../ptzCameraService');
if (operatingMode === 'rover' && getOperatingMode(socket) === 'rover') {
return { operatingMode, state: ptz.getPublicState(socket) };
}
if (operatingMode === 'ptz') {
await ptz.prepareParticipation(socket);
if (socket.disconnected) throw new Error('Session disconnected');
ptz.checkParticipationAccess(socket);
const leave = roverManager.canLeaveCurrentRover(socket);
if (!leave.ok) throw new Error(leave.message);
// Block new rover ownership before release events notify other services.
socket.data.operatingMode = 'ptz';
assignment.releaseForOperatingMode(socket);
socket.data.ptzEntered = true;
} else {
ptz.releaseTurn(socket);
socket.data.ptzEntered = false;
socket.data.operatingMode = 'rover';
assignment.resumeAssignment(socket);
}
operatingModeEvents.emit('change', { socket, operatingMode });
return { operatingMode, state: ptz.getPublicState(socket) };
}
function enqueueTransition(socket, action) {
// Camera initialization can yield. Serialize switches and turn release per
// socket so a late entry cannot undo a subsequent exit or release request.
const previous = transitions.get(socket) || Promise.resolve();
const next = previous.catch(() => {}).then(() => {
if (socket.disconnected) throw new Error('Session disconnected');
return action();
});
transitions.set(socket, next);
return next;
}
function setOperatingMode(socket, operatingMode) {
if (!['rover', 'ptz'].includes(operatingMode)) {
return Promise.reject(new Error('Invalid operating mode'));
}
return enqueueTransition(socket, () => transition(socket, operatingMode));
}
function runPtzTurnAction(socket, action) {
return enqueueTransition(socket, () => {
const ptz = require('../ptzCameraService');
if (action === 'release') return ptz.releaseTurn(socket);
if (action !== 'request') throw new Error('Invalid PTZ turn action');
if (getOperatingMode(socket) !== 'ptz' || !socket.data.ptzEntered) {
throw new Error('Enter PTZ before requesting a turn');
}
return ptz.claimTurn(socket);
});
}
module.exports = { getOperatingMode, setOperatingMode, runPtzTurnAction, operatingModeEvents };
@@ -396,11 +396,6 @@ async function runDecision(triggerReason) {
.filter((entry) => Number(entry?.ts || 0) >= runtime.contextResetAt) .filter((entry) => Number(entry?.ts || 0) >= runtime.contextResetAt)
.filter((entry) => { .filter((entry) => {
if (!entry?.roverId) return true; if (!entry?.roverId) return true;
/*
Chat context should preserve every public chat target, including the
PTZ virtual rover. Rover replay visibility alone would drop PTZ because
it is owned by ptzCameraService instead of roverManager.
*/
return isPublicChatTargetId(entry.roverId); return isPublicChatTargetId(entry.roverId);
}) })
.slice(-(MAX_CHAT_CONTEXT + MAX_BOT_CONTEXT)); .slice(-(MAX_CHAT_CONTEXT + MAX_BOT_CONTEXT));
@@ -2,7 +2,6 @@
// Purpose: Generates server-side TTS files and sends them to the Reolink TrackMix speaker through neolink. // Purpose: Generates server-side TTS files and sends them to the Reolink TrackMix speaker through neolink.
// Scope: Owns file/cache/process details for PTZ speech only; PTZ ownership, chat identity, and camera motion stay in index.js. // Scope: Owns file/cache/process details for PTZ speech only; PTZ ownership, chat identity, and camera motion stay in index.js.
const crypto = require('crypto'); const crypto = require('crypto');
const fs = require('fs');
const fsp = require('fs/promises'); const fsp = require('fs/promises');
const path = require('path'); const path = require('path');
const { spawn } = require('child_process'); const { spawn } = require('child_process');
@@ -78,6 +77,7 @@ function createPtzAudioPlayback(deps) {
const volume = clampNumber(audioConfig.volume, DEFAULT_VOLUME, 0, 4); const volume = clampNumber(audioConfig.volume, DEFAULT_VOLUME, 0, 4);
const fliteDefaultVoice = String(audioConfig.fliteDefaultVoice || 'kal').trim(); const fliteDefaultVoice = String(audioConfig.fliteDefaultVoice || 'kal').trim();
let disposed = false;
let playbackProc = null; let playbackProc = null;
let playbackSeq = 0; let playbackSeq = 0;
@@ -244,12 +244,16 @@ function createPtzAudioPlayback(deps) {
}, 1200); }, 1200);
} }
async function playFile(filePath, context = {}) { async function playFile(filePath, context = {}, authorize = () => {}) {
if (!audioEnabled) throw new Error('PTZ audio disabled'); if (disposed || !audioEnabled) throw new Error('PTZ audio unavailable');
authorize();
const neolinkConfigPath = await ensureNeolinkConfig(); const neolinkConfigPath = await ensureNeolinkConfig();
const stat = await fsp.stat(filePath); const stat = await fsp.stat(filePath);
if (!stat.isFile()) throw new Error(`PTZ TTS file is not a regular file: ${filePath}`); if (!stat.isFile()) throw new Error(`PTZ TTS file is not a regular file: ${filePath}`);
// Rendering and config IO may outlive a turn or camera configuration.
if (disposed) throw new Error('PTZ audio configuration changed');
authorize();
stopActivePlayback('new-playback'); stopActivePlayback('new-playback');
const seq = ++playbackSeq; const seq = ++playbackSeq;
const args = [ const args = [
@@ -302,13 +306,13 @@ function createPtzAudioPlayback(deps) {
return { pid: proc.pid || null, seq }; return { pid: proc.pid || null, seq };
} }
async function speakText(text, ttsOptions = {}, context = {}) { async function speakText(text, ttsOptions = {}, context = {}, authorize = () => {}) {
const rendered = await ensureTtsFile(text, ttsOptions); const rendered = await ensureTtsFile(text, ttsOptions);
await playFile(rendered.filePath, { await playFile(rendered.filePath, {
...context, ...context,
engine: rendered.engine, engine: rendered.engine,
cached: rendered.cached, cached: rendered.cached,
}); }, authorize);
return rendered; return rendered;
} }
@@ -316,6 +320,10 @@ function createPtzAudioPlayback(deps) {
getState, getState,
speakText, speakText,
stopActivePlayback, stopActivePlayback,
dispose: () => {
disposed = true;
stopActivePlayback('configuration-change');
},
}; };
} }
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,98 @@
// Owns PTZ turns only. Camera IO, permissions and operating-mode entry are supplied
// by the service, so queue transitions cannot create physical rover ownership.
function createParticipation({ io, events, emitChange, getTurnDurationMs, canParticipate, stopMotion, revokeVideo }) {
const state = { operatorSocketId: null, queue: [], deadline: null, generation: 0 };
let timer = null;
function remove(socketId) {
state.queue = state.queue.filter((id) => id !== socketId);
}
function stopOperator(reason) {
clearTimeout(timer);
timer = null;
const previous = state.operatorSocketId;
state.operatorSocketId = null;
state.deadline = null;
state.generation += 1;
if (!previous) return;
revokeVideo(previous);
stopMotion(reason).catch(() => {});
events.emit('operator', { socketId: previous, action: 'release', reason });
}
function scheduleTurn() {
// Uncontested control has no deadline. Starting a new wait gets a full turn,
// while later queue changes must not reset an already running countdown.
if (!state.operatorSocketId || !state.queue.length) {
clearTimeout(timer);
timer = null;
state.deadline = null;
return;
}
if (timer) return;
const duration = getTurnDurationMs();
state.deadline = Date.now() + duration;
timer = setTimeout(() => {
timer = null;
state.queue = state.queue.filter(eligible);
if (state.queue.length) {
// Preserve PTZ's explicit request-to-rejoin rule after a handoff.
stopOperator('turn-expired');
advance('turn-expired');
} else {
scheduleTurn();
emitChange('turn-uncontested');
}
}, duration);
}
function eligible(socketId) {
const socket = io.sockets.sockets.get(socketId);
return socket && !socket.disconnected && canParticipate(socket);
}
function advance(reason) {
state.queue = state.queue.filter(eligible);
while (!state.operatorSocketId && state.queue.length) {
const next = state.queue.shift();
if (!eligible(next)) continue;
state.operatorSocketId = next;
state.generation += 1;
events.emit('operator', { socketId: next, action: 'active' });
}
scheduleTurn();
emitChange(reason);
}
function claim(socket) {
if (!eligible(socket.id)) throw new Error('PTZ participation required');
if (state.operatorSocketId === socket.id || state.queue.includes(socket.id)) return;
state.queue.push(socket.id);
advance('queue-join');
}
function release(socketId, reason = 'turn-release') {
remove(socketId);
if (state.operatorSocketId === socketId) stopOperator(reason);
advance(reason);
}
function reconcile() {
state.queue = state.queue.filter(eligible);
if (state.operatorSocketId && !eligible(state.operatorSocketId)) stopOperator('access-change');
advance('access-change');
}
function clear(reason) {
state.queue = [];
stopOperator(reason);
}
return {
state, claim, release, reconcile, clear,
revoke: (reason) => { stopOperator(reason); advance(reason); },
};
}
module.exports = { createParticipation };
@@ -0,0 +1,123 @@
// Socket transport delegates policy and lifecycle decisions to the PTZ service.
function registerSocketGateway(deps) {
const { io, isAdmin, participation, emitChange, normalizeSocketArgs,
runPtzTurnAction, acceptMotionIntent, getStatus, setSpotlight, setIr,
listPresets, gotoPreset, createPreset, removePreset, passesMode,
normalizeSnapshotIds, addSnapshotSubscription, sendSnapshotFrame,
getSnapshot, removeSnapshotSubscriptions } = deps;
io.on('connection', (socket) => {
socket.on('ptzCamera:revokeOperator', (firstArg, secondArg) => {
const { cb } = normalizeSocketArgs(firstArg, secondArg);
if (!isAdmin(socket)) return cb({ error: 'Admin required' });
participation.revoke('admin-revoke');
cb({ ok: true });
});
socket.on('ptzCamera:requestTurn', async (firstArg, secondArg) => {
const { cb } = normalizeSocketArgs(firstArg, secondArg);
try { cb({ ok: true, state: await runPtzTurnAction(socket, 'request') }); }
catch (err) { cb({ error: err.message }); }
});
socket.on('ptzCamera:release', async (firstArg, secondArg) => {
const { cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb({ ok: true, state: await runPtzTurnAction(socket, 'release') });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:motion', (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
/*
Acknowledge acceptance of the newest desired state immediately. The
serialized ONVIF pump deliberately runs independently of Socket.IO
request latency so browser heartbeats cannot accumulate while waiting
for a camera SOAP response.
*/
cb(acceptMotionIntent(socket, payload));
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:status', async (firstArg, secondArg) => {
const { cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb({ ok: true, status: await getStatus(socket) });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:spotlight', async (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb({ ok: true, light: await setSpotlight(socket, payload) });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:ir', async (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb({ ok: true, ir: await setIr(socket, payload) });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:presets:list', async (firstArg, secondArg) => {
const { cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb({ ok: true, presets: await listPresets(socket) });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:preset:goto', async (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb(await gotoPreset(socket, payload));
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:preset:create', async (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb(await createPreset(socket, payload));
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:preset:remove', async (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
cb(await removePreset(socket, payload));
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:snapshotSubscribe', (firstArg, secondArg) => {
const { payload, cb } = normalizeSocketArgs(firstArg, secondArg);
try {
if (!passesMode(socket)) throw new Error('Not authorized for PTZ snapshots');
const ids = normalizeSnapshotIds(payload);
addSnapshotSubscription(socket, ids);
const lastSnapshotState = getSnapshot();
if (lastSnapshotState?.frame) sendSnapshotFrame(socket, lastSnapshotState.frame, lastSnapshotState.ts);
cb({ ok: true, subscribed: ids });
} catch (err) {
cb({ error: err.message });
}
});
socket.on('ptzCamera:snapshotUnsubscribe', (firstArg, secondArg) => {
const { payload } = normalizeSocketArgs(firstArg, secondArg);
removeSnapshotSubscriptions(socket.id, normalizeSnapshotIds(payload));
});
socket.on('disconnect', () => {
participation.release(socket.id, 'disconnect');
removeSnapshotSubscriptions(socket.id);
emitChange('disconnect');
});
});
}
module.exports = { registerSocketGateway };
@@ -0,0 +1,9 @@
const labels = {
...require('../turnService/labels'),
waiting: 'Someone else has control',
active: 'You control the camera',
showName: 'Show camera name',
hideName: 'Hide camera name',
};
module.exports = labels;
@@ -68,13 +68,13 @@ function getDefaultWebSources(assignment = {}, socket = null) {
replay`, follows the camera currently controlled by that socket just as it replay`, follows the camera currently controlled by that socket just as it
follows an assigned rover below. follows an assigned rover below.
isOperator is deliberately stricter than PTZ access or queue membership: turn.ownsControl is deliberately stricter than PTZ access or queue membership:
spectators and users waiting for a camera turn must not silently replay a spectators and users waiting for a camera turn must not silently replay a
camera they are not currently operating. Keeping this rule here also makes camera they are not currently operating. Keeping this rule here also makes
every web replay entry point share the same default instead of teaching the every web replay entry point share the same default instead of teaching the
chat-command adapter about PTZ-specific state. chat-command adapter about PTZ-specific state.
*/ */
if (ptzCameraService.getPublicState(socket).isOperator) { if (ptzCameraService.getPublicState(socket).turn.ownsControl) {
const source = ptzCameraService.getReplaySource(); const source = ptzCameraService.getReplaySource();
if (!source) return []; if (!source) return [];
return [{ type: source.type, id: String(source.id), label: source.label || source.id }]; return [{ type: source.type, id: String(source.id), label: source.label || source.id }];
@@ -1,7 +1,7 @@
// Room-Camera Configuration // Room-Camera Configuration
// Purpose: Defines the optional named snapshot and stream camera catalog. // Purpose: Defines the optional named snapshot and stream camera catalog.
// Scope: Contains configuration metadata only and never contacts a camera. // Scope: Contains configuration metadata only and never contacts a camera.
const { strictObject, string, boolean } = require('../../configuration/schemaHelpers'); const { strictObject, string, boolean, integer } = require('../../configuration/schemaHelpers');
module.exports = { module.exports = {
key: 'roomCameras', key: 'roomCameras',
@@ -10,6 +10,7 @@ module.exports = {
// initial configuration while the feature switch prevents network requests. // initial configuration while the feature switch prevents network requests.
defaultValue: { defaultValue: {
enabled: false, enabled: false,
streamIntervalMs: 2000,
cameras: [ cameras: [
{ {
id: 'lobby', id: 'lobby',
@@ -29,6 +30,7 @@ module.exports = {
}, },
schema: strictObject({ schema: strictObject({
enabled: boolean({ description: 'Immediately publishes the configured room-camera catalog and enables camera snapshots and streams.' }), enabled: boolean({ description: 'Immediately publishes the configured room-camera catalog and enables camera snapshots and streams.' }),
streamIntervalMs: integer({ title: 'Stream interval (ms)', description: 'Interval for sending room camera images to clients. Lowering this increases bandwidth usage FAST because it sends a full image each time.', minimum: 1 }),
cameras: { cameras: {
type: 'array', type: 'array',
description: 'Room cameras available to the web UI and replay system.', description: 'Room cameras available to the web UI and replay system.',
@@ -8,10 +8,12 @@ const replay = require('../replayEngineV2/roomCameraReplayBuilder');
const { loadConfig, registerConfigurationHandler } = require('../../configuration'); const { loadConfig, registerConfigurationHandler } = require('../../configuration');
let enabled = false; let enabled = false;
let streamIntervalMs = 2000;
const snapshotEngine = createSnapshotEngine({ getRoomCameras, roomCameraEvents }); const snapshotEngine = createSnapshotEngine({ getRoomCameras, roomCameraEvents });
function applyRoomCameraConfig(roomCameraConfig = {}) { function applyRoomCameraConfig(roomCameraConfig = {}) {
enabled = Boolean(roomCameraConfig.enabled); enabled = Boolean(roomCameraConfig.enabled);
streamIntervalMs = roomCameraConfig.streamIntervalMs ?? 2000;
// Loading an empty catalog on disable causes the snapshot engine's existing // Loading an empty catalog on disable causes the snapshot engine's existing
// update listener to close every stream and timer without unregistering the // update listener to close every stream and timer without unregistering the
// stable browser gateway. // stable browser gateway.
@@ -19,6 +21,7 @@ function applyRoomCameraConfig(roomCameraConfig = {}) {
} }
registerRoomCameraSocketGateway({ registerRoomCameraSocketGateway({
getStreamIntervalMs: () => streamIntervalMs,
getRoomCamera, getRoomCamera,
getRoomCameras, getRoomCameras,
getRoomCameraState: snapshotEngine.getRoomCameraState, getRoomCameraState: snapshotEngine.getRoomCameraState,
@@ -8,7 +8,6 @@ const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
const SUBSCRIBE_LIMIT = 50; const SUBSCRIBE_LIMIT = 50;
const SUBSCRIBE_WINDOW_MS = 10000; const SUBSCRIBE_WINDOW_MS = 10000;
const STREAM_INTERVAL_MS = 2000;
function passesMode(socket) { function passesMode(socket) {
const mode = getMode(); const mode = getMode();
@@ -20,7 +19,7 @@ function passesMode(socket) {
return true; return true;
} }
function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoomCameraState, roomCameraStreamEvents }) { function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoomCameraState, roomCameraStreamEvents, getStreamIntervalMs }) {
const cameraSubscribers = new Map(); const cameraSubscribers = new Map();
const socketSubscriptions = new Map(); const socketSubscriptions = new Map();
const subscribeBuckets = new Map(); const subscribeBuckets = new Map();
@@ -82,7 +81,7 @@ function registerRoomCameraSocketGateway({ getRoomCamera, getRoomCameras, getRoo
} }
const lastSent = lastMap.get(id) || 0; const lastSent = lastMap.get(id) || 0;
const now = ts || Date.now(); const now = ts || Date.now();
if (now - lastSent < STREAM_INTERVAL_MS) return; if (now - lastSent < getStreamIntervalMs()) return;
lastMap.set(id, now); lastMap.set(id, now);
sendFrame(socket, id, { ts }, buffer); sendFrame(socket, id, { ts }, buffer);
}); });
@@ -1,15 +1,16 @@
// Match actuator configuration, not a rover's display name or changing telemetry. // Match actuator configuration, not a rover's display name or changing telemetry.
const { isDeepStrictEqual } = require('node:util'); const { isDeepStrictEqual } = require('node:util');
const POLICY_VERSION = 4;
function controlProfile(meta = {}) { function controlProfile(meta = {}) {
return Object.fromEntries(['cameraServo', 'peripherals', 'headlight', 'laser', 'roomba', 'platform'] return Object.fromEntries(['cameraServo', 'headlight', 'laser', 'roomba', 'platform']
.map((key) => [key, meta?.[key] ?? null])); .map((key) => [key, meta?.[key] ?? null]));
} }
function compatible(model, record) { function compatible(model, record) {
if (model?.specification?.version !== 2 || !record) return false; if (model?.specification?.version !== POLICY_VERSION || !record) return false;
const profile = controlProfile(record.meta); const profile = controlProfile(record.meta);
return Array.isArray(model.controlProfiles) && model.controlProfiles.some((saved) => isDeepStrictEqual(saved, profile)); return Array.isArray(model.controlProfiles) && model.controlProfiles.some((saved) => isDeepStrictEqual(saved, profile));
} }
module.exports = { controlProfile, compatible }; module.exports = { POLICY_VERSION, controlProfile, compatible };
@@ -20,7 +20,7 @@ module.exports = {
enabled: boolean({ description: 'Enable human-session recording, background learning, and verified-user model control. Disabled starts no workers or storage.' }), enabled: boolean({ description: 'Enable human-session recording, background learning, and verified-user model control. Disabled starts no workers or storage.' }),
driving: strictObject({ driving: strictObject({
threads: integer({ minimum: 1, maximum: 16, description: 'CPU threads per active model controller.' }), threads: integer({ minimum: 1, maximum: 16, description: 'CPU threads per active model controller.' }),
actionThreshold: number({ minimum: 0.5, maximum: 0.99, description: 'Minimum predicted action probability for issuing a new control event.' }), actionThreshold: number({ minimum: 0.5, maximum: 0.99, description: 'Minimum model score for issuing a command, including drive and stop commands.' }),
staleMs: integer({ minimum: 1000, maximum: 5000, description: 'Stop when video, sensors, or model predictions are older than this many milliseconds.' }), staleMs: integer({ minimum: 1000, maximum: 5000, description: 'Stop when video, sensors, or model predictions are older than this many milliseconds.' }),
}, { description: 'Live model execution. Manual inputs do not stop or pause the model.', required: ['threads', 'actionThreshold', 'staleMs'] }), }, { description: 'Live model execution. Manual inputs do not stop or pause the model.', required: ['threads', 'actionThreshold', 'staleMs'] }),
recording: strictObject({ recording: strictObject({
@@ -1,4 +1,6 @@
// Export an explicit allowlist, never the raw socket/session/configuration object. // Export an explicit allowlist, never the raw socket/session/configuration object.
const { redact } = require('./errors');
const { POLICY_VERSION } = require('./capabilities');
const os = require('node:os'); const os = require('node:os');
const fs = require('node:fs'); const fs = require('node:fs');
const path = require('node:path'); const path = require('node:path');
@@ -6,7 +8,7 @@ const { createHash } = require('node:crypto');
const applicationVersion = require('../../../package.json').version; const applicationVersion = require('../../../package.json').version;
const hash = createHash('sha256'); const hash = createHash('sha256');
for (const file of ['workers/policy.py', 'workers/dataset.py', 'workers/train.py', 'workers/infer.py', for (const file of ['workers/policy.py', 'workers/dataset.py', 'workers/train.py', 'workers/infer.py',
'models.js', 'driving.js', 'recording.js', 'training.js', 'socketGateway.js', 'diagnostics.js']) { 'capabilities.js', 'errors.js', 'storage.js', 'runtime.js', 'models.js', 'driving.js', 'recording.js', 'training.js', 'socketGateway.js', 'diagnostics.js']) {
hash.update(fs.readFileSync(path.join(__dirname, file))); hash.update(fs.readFileSync(path.join(__dirname, file)));
} }
const implementationSha256 = hash.digest('hex'); const implementationSha256 = hash.digest('hex');
@@ -28,8 +30,8 @@ function report(state, controller) {
const metrics = training.diagnostics || {}; const metrics = training.diagnostics || {};
const session = state.session; const session = state.session;
return { return {
reportVersion: 1, generatedAtUnixMs: state.updatedAt, reportVersion: 3, generatedAtUnixMs: state.updatedAt,
software: { applicationVersion, implementationSha256, policyVersion: 2, node: process.version, software: { applicationVersion, implementationSha256, policyVersion: POLICY_VERSION, node: process.version,
dependencies: metrics.dependencies || null }, dependencies: metrics.dependencies || null },
host: { logicalCpus: os.cpus().length, totalMemoryBytes: os.totalmem(), freeMemoryBytes: os.freemem(), loadAverage: os.loadavg() }, host: { logicalCpus: os.cpus().length, totalMemoryBytes: os.totalmem(), freeMemoryBytes: os.freemem(), loadAverage: os.loadavg() },
settings: state.limits, settings: state.limits,
@@ -37,23 +39,24 @@ function report(state, controller) {
freeBytes: state.recording.freeBytes, pendingBytes: state.recording.pendingBytes, freeBytes: state.recording.freeBytes, pendingBytes: state.recording.pendingBytes,
droppedPending: state.recording.droppedPending, droppedPending: state.recording.droppedPending,
sessions: state.recording.sessions.map((item) => ({ startedAtUnixMs: item.startedAt, sessions: state.recording.sessions.map((item) => ({ startedAtUnixMs: item.startedAt,
status: item.video, droppedEvents: item.droppedEvents, errorCategory: errorCategory(item.error) })) }, status: item.video, droppedEvents: item.droppedEvents, error: redact(item.error), failureDetails: item.failureDetails || null, stderr: redact(item.stderr), errorCategory: errorCategory(item.error) })) },
training: { status: training.status, steps: training.steps, examples: training.examples, training: { status: training.status, steps: training.steps, examples: training.examples,
selection: training.selection || null,
loss: training.loss, losses: training.losses || null, publication: training.publication || null, loss: training.loss, losses: training.losses || null, publication: training.publication || null,
evaluation: training.evaluation || null, dataset: metrics.dataset || training.dataset || null, evaluation: training.evaluation || null, dataset: metrics.dataset || training.dataset || null,
resources: { decodeSeconds: metrics.decodeSeconds ?? null, trainingSeconds: metrics.trainingSeconds ?? null, resources: { decodeSeconds: metrics.decodeSeconds ?? null, trainingSeconds: metrics.trainingSeconds ?? null,
evaluationSeconds: metrics.evaluationSeconds ?? null, checkpointSeconds: metrics.checkpointSeconds ?? null, cpuSeconds: metrics.cpuSeconds ?? null, evaluationSeconds: metrics.evaluationSeconds ?? null, checkpointSeconds: metrics.checkpointSeconds ?? null, cpuSeconds: metrics.cpuSeconds ?? null,
peakRssMiB: metrics.peakRssMiB ?? null }, peakRssMiB: metrics.peakRssMiB ?? null },
history: training.history || [], rejectionCategories: (training.rejectionReasons || []).map(errorCategory), history: training.history || [], rejectionReasons: (training.rejectionReasons || []).map(redact), rejectionCategories: (training.rejectionReasons || []).map(errorCategory),
errorCategory: errorCategory(training.error), nextAttemptAtUnixMs: training.nextAttemptAt }, error: redact(training.error), errorCategory: errorCategory(training.error), nextAttemptAtUnixMs: training.nextAttemptAt },
model: session ? { id: session.modelId, checkpointId: session.checkpointId, threshold: session.threshold, appliedThreshold: session.appliedThreshold } : null, model: session ? { id: session.modelId, checkpointId: session.checkpointId, threshold: session.threshold, appliedThreshold: session.appliedThreshold } : null,
controller: controller ? { modelId: controller.modelId, sensorPresent: controller.sensorPresent || null, controller: controller ? { modelId: controller.modelId, sensorPresent: controller.sensorPresent || null,
stopCategory: errorCategory(controller.stopReason), history: (controller.history || []).map((item) => ({ stopReason: redact(controller.stopReason), stopCategory: errorCategory(controller.stopReason), history: (controller.history || []).map((item) => ({
...item, submissions: item.submissions.map((submission) => ({ type: submission.type, ...item, submissions: item.submissions.map((submission) => ({ type: submission.type,
wheels: submission.wheels, result: submission.result === 'issued' ? 'issued' : errorCategory(submission.result) })) })) } : null, command: submission.command, result: redact(submission.result) })) })) } : null,
sensorFields: metrics.policy?.sensorFields || null, sensorFields: metrics.policy?.sensorFields || null,
timing: { alignment: 'server_receipt', browserVideoLatencyMs: null, wheelCommandExpiryMs: null }, timing: { alignment: 'server_receipt', browserVideoLatencyMs: null, wheelCommandExpiryMs: null },
events: state.activity.map(({ at, message }) => ({ atUnixMs: at, message })), events: state.activity.map(({ at, message, detail, error, stderr }) => ({ atUnixMs: at, message: redact(message), detail: redact(detail), error: error || null, stderr: redact(stderr) })),
unavailable: ['browser_video_latency', 'actual_physical_command_execution', 'live_training_cpu_utilization'], unavailable: ['browser_video_latency', 'actual_physical_command_execution', 'live_training_cpu_utilization'],
}; };
} }
@@ -9,7 +9,7 @@ const { recordCommandRequest } = require('../commandService/recording');
const { acquireModel } = require('./models'); const { acquireModel } = require('./models');
const { compatible } = require('./capabilities'); const { compatible } = require('./capabilities');
const OPERATING = new Set(['drive', 'motors', 'servo', 'headlight', 'laser', 'horn', 'peripheral', 'song', 'raw']); const OPERATING = new Set(['drive', 'motors', 'servo', 'headlight', 'laser', 'horn', 'song', 'raw']);
const clamp = (value, min, max) => Math.max(min, Math.min(max, Number(value) || 0)); const clamp = (value, min, max) => Math.max(min, Math.min(max, Number(value) || 0));
function canControl(socket, roverId) { function canControl(socket, roverId) {
@@ -127,21 +127,19 @@ function createDriving({ root, config, onChange, activity }) {
session.latencyMs = message.latencyMs; session.latencyMs = message.latencyMs;
session.proposals = Array.isArray(message.proposals) ? message.proposals.slice(0, 64) : []; session.proposals = Array.isArray(message.proposals) ? message.proposals.slice(0, 64) : [];
session.appliedThreshold = message.threshold; session.appliedThreshold = message.threshold;
session.wheelSpeeds = message.wheelSpeeds;
session.sensorPresent = message.sensorPresent; session.sensorPresent = message.sensorPresent;
session.submissions = []; session.submissions = [];
if (!Array.isArray(message.commands) || message.commands.length > 65) throw new Error('Invalid action batch'); if (!Array.isArray(message.commands) || message.commands.length > 65) throw new Error('Invalid action batch');
session.predictionStatus = message.wheelSpeeds?.some((value) => value !== 0) ? 'Sending wheel controls' : 'Predicting stopped wheels'; session.predictionStatus = message.commands.length ? `${message.commands.length} command(s) proposed` : 'No command proposed';
// Manual commands intentionally do not stop or pause this controller. // Manual commands intentionally do not stop or pause this controller.
for (const command of message.commands) transmit(session, command); for (const command of message.commands) transmit(session, command);
// Bound report history independently of camera and UI frame rates. // Bound report history independently of camera and UI frame rates.
if (!session.history.length || Date.now() - session.history.at(-1).at >= 500) { if (!session.history.length || Date.now() - session.history.at(-1).at >= 500) {
session.history.push({ at: Date.now(), wheelSpeeds: session.wheelSpeeds, session.history.push({ at: Date.now(),
latencyMs: session.latencyMs, frameAgeMs: Date.now() - session.frameAt, latencyMs: session.latencyMs, frameAgeMs: Date.now() - session.frameAt,
sensorAgeMs: Date.now() - session.sensorAt, threshold: session.appliedThreshold, sensorAgeMs: Date.now() - session.sensorAt, threshold: session.appliedThreshold,
proposals: [...session.proposals].sort((a, b) => b.score - a.score).slice(0, 8).map(({ slot, score, reason }) => ({ slot, score, reason })), proposals: [...session.proposals].sort((a, b) => b.score - a.score).slice(0, 8).map(({ slot, score, reason, command }) => ({ slot, score, reason, command })),
submissions: session.submissions.slice(0, 16).map(({ command, result }) => ({ type: command.type, result, submissions: session.submissions.slice(0, 16).map(({ command, result }) => ({ type: command.type, result, command })) });
wheels: command.type === 'drive' ? command.driveDirect : undefined })) });
if (session.history.length > 60) session.history.shift(); if (session.history.length > 60) session.history.shift();
} }
} catch (error) { stop(roverId, error.message); break; } } catch (error) { stop(roverId, error.message); break; }
@@ -0,0 +1,18 @@
// Keep actionable failures while removing credentials and user identifiers.
function redact(value) {
if (value == null) return null;
return String(value).slice(0, 12000)
.replace(/\b(?:https?|rtsp|rtsps):\/\/[^\s'"<>]+/gi, '[redacted URL]')
.replace(/\b(Bearer\s+)\S+/gi, '$1[redacted]')
.replace(/((?:password|token|secret|authorization|cookie|api[_-]?key)\s*[=:]\s*)[^\s,;]+/gi, '$1[redacted]')
.replace(/\bcu_[a-z0-9]+/gi, '[redacted user]')
.replace(/\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi, '[redacted email]')
.replace(/\/(?:home|Users)\/[^/\s]+/g, '/[user]');
}
function errorDetails(error, operation) {
return { atUnixMs: Date.now(), operation: redact(operation),
name: redact(error?.name), message: redact(error?.message ?? error),
code: redact(error?.code), syscall: redact(error?.syscall),
path: redact(error?.path), stack: redact(error?.stack) };
}
module.exports = { redact, errorDetails };
@@ -1,3 +1,4 @@
const { POLICY_VERSION } = require('./capabilities');
// Mutable aliases reference immutable checkpoints. All readers and retention share // Mutable aliases reference immutable checkpoints. All readers and retention share
// a queue so a model cannot disappear between resolving its alias and loading it. // a queue so a model cannot disappear between resolving its alias and loading it.
const fs = require('node:fs/promises'); const fs = require('node:fs/promises');
@@ -19,7 +20,7 @@ async function index(root) {
} }
async function metadata(directory) { async function metadata(directory) {
const model = JSON.parse(await fs.readFile(path.join(directory, 'model.json'), 'utf8')); const model = JSON.parse(await fs.readFile(path.join(directory, 'model.json'), 'utf8'));
if (model.id !== path.basename(directory)) throw new Error('Model identity mismatch'); if (model?.id !== path.basename(directory)) throw new Error('Model identity mismatch');
return model; return model;
} }
async function prune(root, state) { async function prune(root, state) {
@@ -82,6 +83,7 @@ function saveSnapshot(root) {
if (!UUID.test(state.candidate || '')) throw new Error('No completed learner checkpoint yet'); if (!UUID.test(state.candidate || '')) throw new Error('No completed learner checkpoint yet');
const source = path.join(checkpointRoot(root), state.candidate); const source = path.join(checkpointRoot(root), state.candidate);
const original = await metadata(source); const original = await metadata(source);
if (original.specification?.version !== POLICY_VERSION) throw new Error('Waiting for a completed command-policy checkpoint');
const id = randomUUID(); const id = randomUUID();
const words = [['amber', 'silver', 'quiet', 'curious'], ['otter', 'finch', 'fox', 'wren']]; const words = [['amber', 'silver', 'quiet', 'curious'], ['otter', 'finch', 'fox', 'wren']];
const name = `${words[0][randomInt(4)]}-${words[1][randomInt(4)]}-${id.slice(0, 8)}`; const name = `${words[0][randomInt(4)]}-${words[1][randomInt(4)]}-${id.slice(0, 8)}`;
@@ -98,6 +100,58 @@ function saveSnapshot(root) {
return model; return model;
}); });
} }
// Retire recognized older policies before workers start. Unknown/malformed or
// future manifests are not evidence that a directory is disposable.
function removeObsoleteModels(root) {
return serial(async () => {
const obsolete = [];
for (const directory of [path.join(root, 'models'), checkpointRoot(root)]) {
let entries;
try { entries = await fs.readdir(directory, { withFileTypes: true }); }
catch (error) { if (error.code === 'ENOENT') continue; throw error; }
for (const entry of entries) {
if (!entry.isDirectory() || !UUID.test(entry.name)) continue;
const target = path.join(directory, entry.name);
let model;
try { model = await metadata(target); }
catch (error) {
if (error.code === 'ENOENT' || error instanceof SyntaxError || error.message === 'Model identity mismatch') continue;
throw error;
}
const version = model.specification?.version;
if (Number.isInteger(version) && version >= 1 && version < POLICY_VERSION) {
obsolete.push({ target, checkpoint: directory === checkpointRoot(root), id: entry.name });
}
}
}
const state = await index(root);
const oldCheckpoints = new Set(obsolete.filter((entry) => entry.checkpoint).map((entry) => entry.id));
let changed = false;
for (const alias of ['candidate', 'latest', 'previous']) {
if (oldCheckpoints.has(state[alias])) { delete state[alias]; changed = true; }
}
// Remove aliases atomically first, so interruption cannot leave the catalog
// referring to a checkpoint that cleanup already deleted.
if (changed) {
const filename = path.join(checkpointRoot(root), 'index.json');
await fs.writeFile(`${filename}.tmp`, JSON.stringify(state));
await fs.rename(`${filename}.tmp`, filename);
}
for (const { target } of obsolete) {
if (leases.has(target)) retired.add(target);
else { await fs.rm(target, { recursive: true, force: true }); retired.delete(target); }
}
// These filenames belong to superseded optimizer/scheduler lineages, not
// policy versions. The current command policy uses lineage v5.
for (const suffix of ['', '-v1', '-v2', '-v3', '-v4']) {
for (const filename of [`resume${suffix}.pt`, `resume${suffix}.tmp`,
`schedule${suffix}.json`, `schedule${suffix}.json.tmp`]) {
await fs.rm(path.join(root, 'training', filename), { force: true });
}
}
});
}
function listModels(root) { function listModels(root) {
return serial(async () => { return serial(async () => {
let entries; let entries;
@@ -116,4 +170,4 @@ function listModels(root) {
return models; return models;
}); });
} }
module.exports = { listModels, acquireModel, adoptCheckpoint, saveSnapshot }; module.exports = { removeObsoleteModels, listModels, acquireModel, adoptCheckpoint, saveSnapshot };
@@ -1,5 +1,6 @@
// One bounded recording session. Video packets retain receiver wall-clock PTS; // One bounded recording session. Video packets retain receiver wall-clock PTS;
// telemetry uses the same server clock, not an assumed browser playback clock. // telemetry uses the same server clock, not an assumed browser playback clock.
const { errorDetails, redact } = require('./errors');
const fs = require('node:fs'); const fs = require('node:fs');
const fsp = require('node:fs/promises'); const fsp = require('node:fs/promises');
const path = require('node:path'); const path = require('node:path');
@@ -34,6 +35,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
let dockedAt = null; let dockedAt = null;
let pruning = Promise.resolve(); let pruning = Promise.resolve();
let failure = null; let failure = null;
let failureDetails = null;
let stopping = false; let stopping = false;
let stopPromise; let stopPromise;
let child; let child;
@@ -43,7 +45,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
let resolveVideo; let resolveVideo;
const videoClosed = new Promise((resolve) => { resolveVideo = resolve; }); const videoClosed = new Promise((resolve) => { resolveVideo = resolve; });
output.on('error', (error) => { output.on('error', (error) => {
failure = error.message; failure = error.message; failureDetails = errorDetails(error, 'recording session');
// Stop file growth immediately when telemetry is no longer writable. // Stop file growth immediately when telemetry is no longer writable.
child?.kill('SIGTERM'); child?.kill('SIGTERM');
}); });
@@ -76,7 +78,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
output.write(line); output.write(line);
} catch (error) { } catch (error) {
metadata.droppedEvents += 1; metadata.droppedEvents += 1;
failure = error.message; failure = error.message; failureDetails = errorDetails(error, 'recording session');
child?.kill('SIGTERM'); child?.kill('SIGTERM');
} }
return true; return true;
@@ -97,7 +99,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
'-reset_timestamps', '0', path.join(directory, 'video-%09d.mkv'), '-reset_timestamps', '0', path.join(directory, 'video-%09d.mkv'),
], { stdio: ['ignore', 'ignore', 'pipe'] }); ], { stdio: ['ignore', 'ignore', 'pipe'] });
child.stderr.on('data', (chunk) => { stderr = (stderr + chunk.toString()).slice(-4096); }); child.stderr.on('data', (chunk) => { stderr = (stderr + chunk.toString()).slice(-4096); });
child.on('error', (error) => { failure = error.message; }); child.on('error', (error) => { failure = error.message; failureDetails = errorDetails(error, 'recording session'); });
child.once('close', (code, signal) => { child.once('close', (code, signal) => {
childClosed = true; childClosed = true;
videoResult = { code, signal, stderr }; videoResult = { code, signal, stderr };
@@ -131,7 +133,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
if (!metadata.bufferOnly || stopping) return; if (!metadata.bufferOnly || stopping) return;
if (stat.mtimeMs < Date.now() - 5000) await fsp.unlink(filename); if (stat.mtimeMs < Date.now() - 5000) await fsp.unlink(filename);
} }
}).catch((error) => { failure = error.message; child.kill('SIGTERM'); }); }).catch((error) => { failure = error.message; failureDetails = errorDetails(error, 'recording session'); child.kill('SIGTERM'); });
} }
async function stop(reason) { async function stop(reason) {
@@ -158,9 +160,10 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
metadata.endedAt = Date.now(); metadata.endedAt = Date.now();
metadata.reason = reason; metadata.reason = reason;
metadata.error = failure; metadata.error = failure;
metadata.failureDetails = failureDetails;
metadata.video.result = videoResult; metadata.video.result = videoResult;
await saveMetadata(directory, metadata); await saveMetadata(directory, metadata);
if (failure) logger.warn('Recording session incomplete', { roverId, id, error: failure, stderr }); if (failure) logger.warn('Recording session incomplete', { roverId, id, error: failure, failure: failureDetails, stderr });
})(); })();
return stopPromise; return stopPromise;
} }
@@ -173,7 +176,7 @@ async function createRecording({ root, roverId, snapshot, logger, docked = false
get failed() { return Boolean(failure); }, get failed() { return Boolean(failure); },
get stopping() { return stopping; }, get stopping() { return stopping; },
getStatus: () => ({ id, roverId, startedAt: metadata.startedAt, droppedEvents: metadata.droppedEvents, getStatus: () => ({ id, roverId, startedAt: metadata.startedAt, droppedEvents: metadata.droppedEvents,
video: childClosed ? 'stopped' : metadata.bufferOnly ? 'Dock lead-in buffer' : 'Recording human driving', error: failure, stopping }), video: childClosed ? 'stopped' : metadata.bufferOnly ? 'Dock lead-in buffer' : 'Recording human driving', error: redact(failure), failureDetails, stderr: redact(stderr), stopping }),
// Process exit is synchronous; normal disable uses stop() and awaits close. // Process exit is synchronous; normal disable uses stop() and awaits close.
kill: () => { if (!childClosed) child.kill('SIGKILL'); }, kill: () => { if (!childClosed) child.kill('SIGKILL'); },
}; };
@@ -1,4 +1,5 @@
// Enabled-only recording, training, controller, and card lifecycle. // Enabled-only recording, training, controller, and card lifecycle.
const { redact, errorDetails } = require('./errors');
const fs = require('node:fs/promises'); const fs = require('node:fs/promises');
const path = require('node:path'); const path = require('node:path');
const { resolveDataPath } = require('../../helpers/dataPaths'); const { resolveDataPath } = require('../../helpers/dataPaths');
@@ -13,6 +14,7 @@ const { createRecording } = require('./recording');
const { recoverRecordings, enforceRetention } = require('./storage'); const { recoverRecordings, enforceRetention } = require('./storage');
const { createTrainer } = require('./training'); const { createTrainer } = require('./training');
const { createDriving } = require('./driving'); const { createDriving } = require('./driving');
const { removeObsoleteModels } = require('./models');
const { createGateway } = require('./socketGateway'); const { createGateway } = require('./socketGateway');
function createRuntime({ config, logger }) { function createRuntime({ config, logger }) {
@@ -21,7 +23,9 @@ function createRuntime({ config, logger }) {
const recentActivity = []; const recentActivity = [];
function activity(message, details = {}) { function activity(message, details = {}) {
recentActivity.push({ at: Date.now(), message, roverId: details.roverId || null, recentActivity.push({ at: Date.now(), message, roverId: details.roverId || null,
detail: details.reason || details.error || details.model || null }); detail: redact(details.reason || details.error || details.model),
error: details.failure || (details.error ? errorDetails(details.error, message) : null),
stderr: redact(details.stderr) });
if (recentActivity.length > 40) recentActivity.shift(); if (recentActivity.length > 40) recentActivity.shift();
} }
const baseLogger = logger; const baseLogger = logger;
@@ -76,7 +80,7 @@ function createRuntime({ config, logger }) {
// The serialized maintenance pass still owns closing/removing the session. // The serialized maintenance pass still owns closing/removing the session.
for (const [id, session] of sessions) { for (const [id, session] of sessions) {
if (!next.has(id)) session.stop('no-human-controller').catch((error) => { if (!next.has(id)) session.stop('no-human-controller').catch((error) => {
logger.warn('Unable to finalize unattended recording', { roverId: id, error: error.message }); logger.warn('Unable to finalize unattended recording', { roverId: id, error: error.message, failure: errorDetails(error, 'rover learning runtime') });
}); });
} }
schedule(); schedule();
@@ -184,7 +188,7 @@ function createRuntime({ config, logger }) {
await trainer.tick(new Set([...sessions.values()].map((session) => session.id))); await trainer.tick(new Set([...sessions.values()].map((session) => session.id)));
} catch (error) { } catch (error) {
// Training availability must not tear down otherwise healthy capture. // Training availability must not tear down otherwise healthy capture.
logger.error('Unable to schedule training', { error: error.message }); logger.error('Unable to schedule training', { error: error.message, failure: errorDetails(error, 'rover learning runtime') });
} }
} }
} }
@@ -197,7 +201,7 @@ function createRuntime({ config, logger }) {
// The next tick is scheduled after completion: disk slowness cannot build // The next tick is scheduled after completion: disk slowness cannot build
// an unbounded backlog of retention tasks or overlapping recorder starts. // an unbounded backlog of retention tasks or overlapping recorder starts.
maintenance = maintain().catch(async (error) => { maintenance = maintain().catch(async (error) => {
logger.error('Recording maintenance failed; closing recorders', { error: error.message }); logger.error('Recording maintenance failed; closing recorders', { error: error.message, failure: errorDetails(error, 'recording maintenance') });
await Promise.allSettled([...sessions.keys()].map((id) => closeSession(id, 'storage-error'))); await Promise.allSettled([...sessions.keys()].map((id) => closeSession(id, 'storage-error')));
}).finally(() => { }).finally(() => {
busy = false; busy = false;
@@ -215,12 +219,13 @@ function createRuntime({ config, logger }) {
// Other server services can exit the process before asynchronous cleanup // Other server services can exit the process before asynchronous cleanup
// finishes. Signal children immediately, and keep an exit fallback too. // finishes. Signal children immediately, and keep an exit fallback too.
for (const session of sessions.values()) session.stop('server-shutdown').catch(() => undefined); for (const session of sessions.values()) session.stop('server-shutdown').catch(() => undefined);
stop().catch((error) => logger.warn('Recording shutdown failed', { error: error.message })); stop().catch((error) => logger.warn('Recording shutdown failed', { error: error.message, failure: errorDetails(error, 'rover learning runtime') }));
} }
async function start() { async function start() {
await fs.mkdir(recordingsRoot, { recursive: true }); await fs.mkdir(recordingsRoot, { recursive: true });
await recoverRecordings(recordingsRoot); await recoverRecordings(recordingsRoot);
await removeObsoleteModels(root);
await trainer.start(); await trainer.start();
running = true; running = true;
driving.attach(); driving.attach();
@@ -10,9 +10,17 @@ async function listRecordings(root) {
if (!entry.isDirectory() || !/^\d+-[0-9a-f-]{36}$/.test(entry.name)) continue; if (!entry.isDirectory() || !/^\d+-[0-9a-f-]{36}$/.test(entry.name)) continue;
const directory = path.join(root, entry.name); const directory = path.join(root, entry.name);
let bytes = 0; let bytes = 0;
for (const file of await fs.readdir(directory, { withFileTypes: true })) { let files;
try { files = await fs.readdir(directory, { withFileTypes: true }); }
catch (error) { if (error.code === 'ENOENT') continue; throw error; }
for (const file of files) {
if (!file.isFile()) continue; if (!file.isFile()) continue;
bytes += (await fs.stat(path.join(directory, file.name))).size; try { bytes += (await fs.stat(path.join(directory, file.name))).size; }
catch (error) {
// Dock-buffer pruning and atomic metadata replacement can remove a
// listed file before stat. Absence is normal; other I/O failures are not.
if (error.code !== 'ENOENT') throw error;
}
} }
recordings.push({ id: entry.name, directory, bytes, startedAt: Number(entry.name.split('-')[0]) }); recordings.push({ id: entry.name, directory, bytes, startedAt: Number(entry.name.split('-')[0]) });
} }
@@ -1,9 +1,10 @@
// One owned finite worker at a time. Leases are acquired in the runtime's // One owned finite worker at a time. Leases are acquired in the runtime's
// serialized maintenance pass, before retention can next inspect recordings. // serialized maintenance pass, before retention can next inspect recordings.
const { errorDetails } = require('./errors');
const fs = require('node:fs/promises'); const fs = require('node:fs/promises');
const path = require('node:path'); const path = require('node:path');
const { spawn } = require('node:child_process'); const { spawn } = require('node:child_process');
const { randomInt } = require('node:crypto'); const { randomInt, createHash } = require('node:crypto');
const { adoptCheckpoint } = require('./models'); const { adoptCheckpoint } = require('./models');
const { listRecordings } = require('./storage'); const { listRecordings } = require('./storage');
@@ -32,7 +33,7 @@ function createTrainer({ root, config, actionThreshold, logger }) {
} }
} }
try { try {
const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v3.json'), 'utf8')); const saved = JSON.parse(await fs.readFile(path.join(trainingRoot, 'schedule-v5.json'), 'utf8'));
ledger = saved.sessions; ledger = saved.sessions;
state = { ...saved.progress, status: 'waiting' }; state = { ...saved.progress, status: 'waiting' };
if (!ledger || typeof ledger !== 'object') throw new Error('Invalid training schedule'); if (!ledger || typeof ledger !== 'object') throw new Error('Invalid training schedule');
@@ -40,12 +41,12 @@ function createTrainer({ root, config, actionThreshold, logger }) {
} catch (error) { } catch (error) {
startupError = true; startupError = true;
state = { ...state, status: 'error', error: error.message }; state = { ...state, status: 'error', error: error.message };
logger.error('Training initialization failed; recording remains available', { error: error.message }); logger.error('Training initialization failed; recording remains available', { error: error.message, failure: errorDetails(error, 'training initialization') });
} }
} }
async function saveLedger() { async function saveLedger() {
const filename = path.join(trainingRoot, 'schedule-v3.json'); const filename = path.join(trainingRoot, 'schedule-v5.json');
await fs.writeFile(`${filename}.tmp`, JSON.stringify({ sessions: ledger, progress: state })); await fs.writeFile(`${filename}.tmp`, JSON.stringify({ sessions: ledger, progress: state }));
await fs.rename(`${filename}.tmp`, filename); await fs.rename(`${filename}.tmp`, filename);
} }
@@ -74,10 +75,27 @@ function createTrainer({ root, config, actionThreshold, logger }) {
const other = randomInt(index + 1); const other = randomInt(index + 1);
[eligible[index], eligible[other]] = [eligible[other], eligible[index]]; [eligible[index], eligible[other]] = [eligible[other], eligible[index]];
} }
const selected = eligible.slice(0, 4); // Match workers/train.py exactly. Never move a previously trained session
// into evaluation just to fill a slot: that would contaminate the holdout.
const partitions = { training: [], heldOut: [] };
for (const id of eligible) {
const heldOut = parseInt(createHash('sha256').update(id).digest('hex').slice(0, 8), 16) % 5 === 0;
partitions[heldOut ? 'heldOut' : 'training'].push(id);
}
const trainingSessions = partitions.training.slice(0, partitions.heldOut.length ? 3 : 4);
const heldOutSessions = trainingSessions.length ? partitions.heldOut.slice(0, 2) : [];
const selected = [...trainingSessions, ...heldOutSessions];
const selection = {
atUnixMs: Date.now(),
availableTraining: partitions.training.length, availableHeldOut: partitions.heldOut.length,
selectedTraining: trainingSessions.length, selectedHeldOut: heldOutSessions.length,
reason: !trainingSessions.length ? 'Waiting for completed usable training-partition recordings'
: !heldOutSessions.length ? 'Training without evaluation: no eligible held-out recordings' : null,
};
state = { ...state, selection };
nextRun = Date.now() + config.intervalSeconds * 1000; nextRun = Date.now() + config.intervalSeconds * 1000;
if (!selected.length) { if (!selected.length) {
state = { ...state, status: 'waiting', reason: 'Waiting for completed usable human recordings' }; state = { ...state, status: 'waiting', reason: selection.reason };
await saveLedger(); await saveLedger();
return; return;
} }
@@ -85,7 +103,7 @@ function createTrainer({ root, config, actionThreshold, logger }) {
await fs.writeFile(jobPath, JSON.stringify({ root, config, actionThreshold, sessions: selected })); await fs.writeFile(jobPath, JSON.stringify({ root, config, actionThreshold, sessions: selected }));
if (stopped) return; if (stopped) return;
selected.forEach((id) => leases.add(id)); selected.forEach((id) => leases.add(id));
state = { ...state, status: 'training', reason: null, sessions: selected }; state = { ...state, status: 'training', reason: selection.reason, sessions: selected };
state.startedAt = Date.now(); state.startedAt = Date.now();
child = spawn(config.python, [path.join(__dirname, 'workers/train.py'), '--job', jobPath], { child = spawn(config.python, [path.join(__dirname, 'workers/train.py'), '--job', jobPath], {
stdio: ['ignore', 'pipe', 'pipe'], stdio: ['ignore', 'pipe', 'pipe'],
@@ -148,7 +166,7 @@ function createTrainer({ root, config, actionThreshold, logger }) {
} }
} catch (error) { } catch (error) {
state = { ...state, status: 'error', error: error.message }; state = { ...state, status: 'error', error: error.message };
logger.error('Training result persistence failed', { error: error.message }); logger.error('Training result persistence failed', { error: error.message, failure: errorDetails(error, 'training result persistence') });
} finally { } finally {
history.push({ at: Date.now(), status: state.status, steps: state.steps, loss: state.loss, history.push({ at: Date.now(), status: state.status, steps: state.steps, loss: state.loss,
losses: state.losses || null, evaluation: state.evaluation || null, losses: state.losses || null, evaluation: state.evaluation || null,
@@ -9,7 +9,7 @@ from pathlib import Path
import av import av
import numpy as np import numpy as np
from policy import SLOTS, VALUES, HEIGHT, WIDTH, encode_command, sensor_vector from policy import SLOTS, VALUES, HEIGHT, WIDTH, SENSORS, command_family, is_stop, encode_command, sensor_vector
def load_events(directory): def load_events(directory):
@@ -47,7 +47,7 @@ def load_events(directory):
return metadata, sorted(events, key=lambda e: e['ts']) return metadata, sorted(events, key=lambda e: e['ts'])
def prepare(directory, schema, stats): def prepare(directory, schema, stats, extend):
metadata, events = load_events(directory) metadata, events = load_events(directory)
requests = {} requests = {}
accepted = set() accepted = set()
@@ -72,28 +72,27 @@ def prepare(directory, schema, stats):
if not sensors: if not sensors:
raise ValueError('No sensor observations') raise ValueError('No sensor observations')
candidate = copy.deepcopy(schema) candidate = copy.deepcopy(schema)
history, targets, wheels = [], [], [(stamp, None) for stamp in boundaries] history, targets, unsupported_times = [], [], []
unsupported = {}
stats.update({'sensorFrames': len(sensors), 'dispatches': len(dispatches), stats.update({'sensorFrames': len(sensors), 'dispatches': len(dispatches),
'failedCommands': len(failed), 'sensorPresentFraction': np.mean([row[1][32:] for row in sensors], axis=0).tolist()}) 'failedCommands': len(failed), 'sensorPresentFraction': np.mean([row[1][len(SENSORS):] for row in sensors], axis=0).tolist()})
for data in dispatches: for data in dispatches:
command = data.get('command') or {} command = data.get('command') or {}
request = requests.get(data.get('requestId')) if command.get('type') in ('reset', 'reboot', 'update'):
human = (data.get('source') == 'client' and request and data.get('commandId') in accepted boundaries.append(data['ts'])
and data.get('commandId') not in failed) try:
stamp = data['ts'] encoded = encode_command(command, candidate, extend=extend)
if command.get('type') == 'drive': except ValueError as error:
direct = command.get('driveDirect') or {} # Unrepresentable controls invalidate only nearby observations/targets,
pair = [direct.get('left'), direct.get('right')] # not the rest of a recording. Never turn them into negative labels.
if human and all(isinstance(v, (int, float)) and math.isfinite(v) for v in pair): request = requests.get(data.get('requestId'))
wheels.append((stamp, np.clip(np.array(pair, dtype=np.float32), -metadata.get('wheelSpeedLimit', 500), metadata.get('wheelSpeedLimit', 500)) / 500)) unsupported_times.extend([data['ts'], request['ts'] if request else data['ts']])
else: key = json.dumps(command, sort_keys=True)
wheels.append((stamp, None)) if key not in unsupported and len(unsupported) < 32:
elif command.get('type') in ('raw', 'reset', 'reboot', 'update', 'sensorStream'): unsupported[key] = {'command': command, 'reason': str(error), 'count': 0}
# Raw OI can change mode or motion; do not assume the last wheel if key in unsupported:
# command survives it. A later explicit human drive resolves intent. unsupported[key]['count'] += 1
wheels.append((stamp, None)) continue
boundaries.append(stamp)
encoded = encode_command(command, candidate, extend=True)
if encoded is None: if encoded is None:
continue continue
stamp = data['ts'] stamp = data['ts']
@@ -109,25 +108,25 @@ def prepare(directory, schema, stats):
# Server safety overrides belong in history, never demonstration # Server safety overrides belong in history, never demonstration
# targets. Exclude windows around them rather than teaching inaction. # targets. Exclude windows around them rather than teaching inaction.
boundaries.append(stamp) boundaries.append(stamp)
if not targets: stats['unsupportedCommands'] = list(unsupported.values())
stats['unsupportedCommandEvents'] = len(unsupported_times) // 2
if not targets and not unsupported_times:
raise ValueError('No accepted client operating commands') raise ValueError('No accepted client operating commands')
schema[:] = candidate schema[:] = candidate
return metadata, sensors, sorted(wheels, key=lambda x: x[0]), sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries) return metadata, sensors, sorted(history, key=lambda x: x[0]), sorted(targets, key=lambda x: x[0]), sorted(boundaries), sorted(unsupported_times)
def windows(directory, schema, cancelled=lambda: False, stats=None): def windows(directory, schema, cancelled=lambda: False, stats=None, extend=True):
directory = Path(directory) directory = Path(directory)
stats = stats if stats is not None else {} stats = stats if stats is not None else {}
metadata, sensors, wheels, history, targets, boundaries = prepare(directory, schema, stats) metadata, sensors, history, targets, boundaries, unsupported_times = prepare(directory, schema, stats, extend)
wheel_times = [row[0] for row in wheels] stats.update({'usableWindows': 0, 'commandWindows': 0, 'stopWindows': 0, 'noCommandWindows': 0,
stats.update({'usableWindows': 0, 'movingWindows': 0, 'stoppedWindows': 0, 'staleSensorWindows': 0, 'boundaryWindows': 0,
'unknownWheelWindows': 0, 'staleSensorWindows': 0, 'boundaryWindows': 0, 'unsupportedWindows': 0, 'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0})
'ambiguousWindows': 0, 'videoFrames': 0, 'maxVideoGapMs': 0})
sensor_times = [row[0] for row in sensors] sensor_times = [row[0] for row in sensors]
target_times = [row[0] for row in targets] target_times = [row[0] for row in targets]
available = np.zeros(SLOTS, dtype=np.float32) available = np.zeros(SLOTS, dtype=np.float32)
for _, slot, _ in history: available[:len(schema)] = 1
available[slot] = float(schema[slot]['template']['type'] != 'drive')
command_values = np.zeros((SLOTS, VALUES), dtype=np.float32) command_values = np.zeros((SLOTS, VALUES), dtype=np.float32)
command_times = np.full(SLOTS, -np.inf) command_times = np.full(SLOTS, -np.inf)
cursor = 0 cursor = 0
@@ -164,8 +163,10 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
last_sample = stamp last_sample = stamp
while cursor < len(history) and history[cursor][0] < stamp: while cursor < len(history) and history[cursor][0] < stamp:
when, slot, values = history[cursor] when, slot, values = history[cursor]
command_values[slot] = values # Do not let drive predictions simply copy earlier human/model drive commands.
command_times[slot] = when if schema[slot]['template']['type'] != 'drive':
command_values[slot] = values
command_times[slot] = when
cursor += 1 cursor += 1
sensor_index = bisect.bisect_right(sensor_times, stamp) - 1 sensor_index = bisect.bisect_right(sensor_times, stamp) - 1
if sensor_index < 0 or stamp - sensor_times[sensor_index] > 1000: if sensor_index < 0 or stamp - sensor_times[sensor_index] > 1000:
@@ -185,15 +186,12 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
chosen.append(options[-1]) chosen.append(options[-1])
if len(chosen) != 4 or stamp + 100 > metadata['endedAt']: if len(chosen) != 4 or stamp + 100 > metadata['endedAt']:
continue continue
if bisect.bisect_right(unsupported_times, stamp + 100) > bisect.bisect_left(unsupported_times, chosen[0][0]):
stats['unsupportedWindows'] += 1
continue
if bisect.bisect_right(boundaries, stamp + 100) > bisect.bisect_left(boundaries, chosen[0][0]): if bisect.bisect_right(boundaries, stamp + 100) > bisect.bisect_left(boundaries, chosen[0][0]):
stats['boundaryWindows'] += 1 stats['boundaryWindows'] += 1
continue continue
before = bisect.bisect_left(wheel_times, stamp) - 1
after = bisect.bisect_right(wheel_times, stamp + 100) - 1
if before < 0 or after < 0 or wheels[before][1] is None or wheels[after][1] is None:
stats['unknownWheelWindows'] += 1
continue
wheel_target, previous_wheels = wheels[after][1], wheels[before][1]
start = bisect.bisect_left(target_times, stamp) start = bisect.bisect_left(target_times, stamp)
end = bisect.bisect_left(target_times, stamp + 100) end = bisect.bisect_left(target_times, stamp + 100)
actions = np.zeros(SLOTS, dtype=np.float32) actions = np.zeros(SLOTS, dtype=np.float32)
@@ -202,13 +200,8 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
families = set() families = set()
for _, slot, vector in targets[start:end]: for _, slot, vector in targets[start:end]:
template = schema[slot]['template'] template = schema[slot]['template']
family = template['type'] family = command_family(template)
if family == 'drive': if ((actions[slot] and (template['type'] in ('headlight', 'laser', 'horn', 'song', 'raw') or not np.array_equal(values[slot], vector)))
continue
if family == 'peripheral':
peripheral = template.get('peripheral') or {}
family += ':' + str(peripheral.get('id')) + ':' + str(peripheral.get('control'))
if ((actions[slot] and (not schema[slot]['fields'] or not np.array_equal(values[slot], vector)))
or (not actions[slot] and family in families)): or (not actions[slot] and family in families)):
ambiguous = True ambiguous = True
break break
@@ -218,5 +211,6 @@ def windows(directory, schema, cancelled=lambda: False, stats=None):
stats['ambiguousWindows'] += 1 stats['ambiguousWindows'] += 1
continue continue
stats['usableWindows'] += 1 stats['usableWindows'] += 1
stats['movingWindows' if np.max(np.abs(wheel_target)) > .02 else 'stoppedWindows'] += 1 stats['commandWindows' if actions.any() else 'noCommandWindows'] += 1
yield np.stack([row[1] for row in chosen]), np.stack([row[2] for row in chosen]), actions, values, available, wheel_target, previous_wheels stats['stopWindows'] += int(any(actions[slot] and is_stop(descriptor) for slot, descriptor in enumerate(schema)))
yield np.stack([row[1] for row in chosen]), np.stack([row[2] for row in chosen]), actions, values, available
@@ -14,7 +14,7 @@ import av
import numpy as np import numpy as np
import torch import torch
from policy import Policy, VERSION, SLOTS, VALUES, HEIGHT, WIDTH, encode_command, sensor_vector from policy import Policy, VERSION, SLOTS, VALUES, HEIGHT, WIDTH, SENSORS, select_slots, decode_command, encode_command, sensor_vector
def main(): def main():
@@ -78,7 +78,7 @@ def main():
args.threshold = max(.05, min(.99, float(event['threshold']))) args.threshold = max(.05, min(.99, float(event['threshold'])))
elif event['kind'] == 'sensor': elif event['kind'] == 'sensor':
sensors, sensor_time = event['data'], event['ts'] sensors, sensor_time = event['data'], event['ts']
elif event['kind'] == 'command': elif event['kind'] == 'command' and event['data'].get('type') != 'drive':
try: try:
encoded = encode_command(event['data'], schema) encoded = encode_command(event['data'], schema)
if encoded: if encoded:
@@ -104,58 +104,30 @@ def main():
continue continue
before = time.perf_counter() before = time.perf_counter()
with torch.inference_mode(): with torch.inference_mode():
logits, numeric, wheel_output = model(torch.from_numpy(np.stack([row[1] for row in selected])).unsqueeze(0).float() / 255, logits, numeric = model(torch.from_numpy(np.stack([row[1] for row in selected])).unsqueeze(0).float() / 255,
torch.from_numpy(np.stack([row[2] for row in selected])).unsqueeze(0)) torch.from_numpy(np.stack([row[2] for row in selected])).unsqueeze(0))
probabilities = logits[0].sigmoid().tolist() probabilities = logits[0].sigmoid().tolist()
predictions = numeric[0].tolist() predictions = numeric[0].tolist()
wheel_speeds = [round(value * 500) for value in wheel_output[0].tolist()]
if not all(math.isfinite(value) for value in probabilities): if not all(math.isfinite(value) for value in probabilities):
raise ValueError('Nonfinite model output') raise ValueError('Nonfinite model output')
families = {} winners = select_slots(probabilities, schema, args.threshold)
active = set() active = {slot for slot in range(len(schema)) if probabilities[slot] >= args.threshold}
proposals = []
for slot, descriptor in enumerate(schema):
if descriptor['template']['type'] == 'drive':
continue
command = json.loads(json.dumps(descriptor['template']))
family = command['type']
if family == 'peripheral':
family += ':' + str(command['peripheral']['id']) + ':' + str(command['peripheral']['control'])
for index, field in enumerate(descriptor['fields']):
number = predictions[slot][index]
if not math.isfinite(number):
raise ValueError('Nonfinite control value')
number = math.tan(max(-.98, min(.98, number)) * math.pi / 2) * field['scale']
target = command
for key in field['path'][:-1]:
target = target[key]
target[field['path'][-1]] = round(number) if field['integer'] else number
proposals.append({'slot': slot, 'score': probabilities[slot], 'command': command,
'reason': 'Below threshold'})
if probabilities[slot] < args.threshold:
continue
active.add(slot)
discrete = command['type'] in ('headlight', 'laser', 'horn', 'song', 'raw')
if family not in families or probabilities[slot] > families[family][0]:
families[family] = (probabilities[slot], command, slot, discrete)
latched.intersection_update(active) latched.intersection_update(active)
# Choose the winning action before latching, so a held toggle cannot proposals, commands = [], []
# expose a competing lower-confidence toggle on the following tick. for slot, descriptor in enumerate(schema):
commands = [command for _, command, slot, discrete in families.values() command = decode_command(descriptor, predictions[slot])
if not discrete or slot not in latched] discrete = command['type'] in ('headlight', 'laser', 'horn', 'song', 'raw')
winners = {row[2]: row for row in families.values()} reason = ('Below threshold' if slot not in active else
for proposal in proposals: 'Competing action' if slot not in winners else
slot = proposal['slot'] 'Held discrete action' if discrete and slot in latched else 'Proposed')
if slot in active: proposals.append({'slot': slot, 'score': probabilities[slot], 'command': command, 'reason': reason})
proposal['reason'] = ('Competing action' if slot not in winners else if reason == 'Proposed':
'Held discrete action' if winners[slot][3] and slot in latched else 'Proposed') commands.append(command)
for _, _, slot, discrete in families.values(): if slot in winners and discrete:
if discrete:
latched.add(slot) latched.add(slot)
commands.insert(0, {'type': 'drive', 'driveDirect': dict(zip(('left', 'right'), wheel_speeds))})
print(json.dumps({'kind': 'prediction', 'frameAt': stamp, 'sensorAt': sensor_time, print(json.dumps({'kind': 'prediction', 'frameAt': stamp, 'sensorAt': sensor_time,
'latencyMs': (time.perf_counter() - before) * 1000, 'latencyMs': (time.perf_counter() - before) * 1000,
'wheelSpeeds': wheel_speeds, 'sensorPresent': sensor_vector(sensors)[32:].tolist(), 'sensorPresent': sensor_vector(sensors)[len(SENSORS):].tolist(),
'commands': commands, 'proposals': proposals, 'threshold': args.threshold}, allow_nan=False), flush=True) 'commands': commands, 'proposals': proposals, 'threshold': args.threshold}, allow_nan=False), flush=True)
@@ -4,6 +4,8 @@ Slots describe command shapes, not arbitrary executable instructions. Numeric
leaves are regressed; string/bool/list values remain exact template constants. leaves are regressed; string/bool/list values remain exact template constants.
No command is sent by this module. Execution must use server authorization. No command is sent by this module. Execution must use server authorization.
""" """
import base64
import itertools
import copy import copy
import json import json
import math import math
@@ -12,12 +14,12 @@ import numpy as np
import torch import torch
from torch import nn from torch import nn
VERSION = 2 VERSION = 4
SLOTS = 64 SLOTS = 64
VALUES = 8 VALUES = 8
FRAMES = 4 FRAMES = 4
HEIGHT, WIDTH = 120, 160 HEIGHT, WIDTH = 120, 160
OPERATING_TYPES = {"drive", "motors", "servo", "headlight", "laser", "horn", "peripheral", "song", "raw"} OPERATING_TYPES = {"drive", "motors", "servo", "headlight", "laser", "horn", "song", "raw"}
SENSORS = [ SENSORS = [
("bumpsAndWheelDrops.bumpLeft", 1), ("bumpsAndWheelDrops.bumpRight", 1), ("bumpsAndWheelDrops.bumpLeft", 1), ("bumpsAndWheelDrops.bumpRight", 1),
("bumpsAndWheelDrops.wheelDropLeft", 1), ("bumpsAndWheelDrops.wheelDropRight", 1), ("bumpsAndWheelDrops.wheelDropLeft", 1), ("bumpsAndWheelDrops.wheelDropRight", 1),
@@ -26,7 +28,6 @@ SENSORS = [
("voltageMv", 16000), ("currentMa", 3000), ("batteryChargeMah", 4000), ("voltageMv", 16000), ("currentMa", 3000), ("batteryChargeMah", 4000),
("batteryCapacityMah", 4000), ("batteryTemperatureC", 50), ("batteryCapacityMah", 4000), ("batteryTemperatureC", 50),
("chargingState.code", 5), ("chargingSources.homeBase", 1), ("oiMode.code", 4), ("chargingState.code", 5), ("chargingSources.homeBase", 1), ("oiMode.code", 4),
("wheelSpeedsMmPerSecond.left", 500), ("wheelSpeedsMmPerSecond.right", 500),
("wheelLeftCurrentMa", 2000), ("wheelRightCurrentMa", 2000), ("wheelLeftCurrentMa", 2000), ("wheelRightCurrentMa", 2000),
("mainBrushCurrentMa", 2000), ("sideBrushCurrentMa", 2000), ("mainBrushCurrentMa", 2000), ("sideBrushCurrentMa", 2000),
("lightBumpLeftSignal", 4096), ("lightBumpRightSignal", 4096), ("lightBumpLeftSignal", 4096), ("lightBumpRightSignal", 4096),
@@ -60,14 +61,20 @@ def describe_command(command):
values, fields = [], [] values, fields = [], []
def visit(value, path=()): def visit(value, path=()):
if isinstance(value, list) and command['type'] == 'horn' and path == ('horn', 'freqs'):
return [visit(item, (*path, index)) for index, item in enumerate(value)]
if isinstance(value, dict): if isinstance(value, dict):
return {key: visit(item, (*path, key)) for key, item in sorted(value.items())} return {key: visit(item, (*path, key)) for key, item in sorted(value.items())}
# Exact actuator zeros preserve demonstrated releases. Horn frequency
# zeros are parameters so changing a chord does not require a new slot.
if value == 0 and command['type'] != 'horn':
return copy.deepcopy(value)
if isinstance(value, (int, float)) and not isinstance(value, bool): if isinstance(value, (int, float)) and not isinstance(value, bool):
if not math.isfinite(value) or len(values) >= VALUES: if not math.isfinite(value) or len(values) >= VALUES:
raise ValueError('Nonfinite command or too many numeric control fields') raise ValueError('Nonfinite command or too many numeric control fields')
scale = {'drive': 500, 'motors': 255, 'servo': 180, 'peripheral': 255, scale = {'drive': 500, 'motors': 255, 'servo': 180,
'horn': 1000, 'song': 127}.get(command['type'], 1) 'horn': 1000, 'song': 127}.get(command['type'], 1)
fields.append({'path': list(path), 'scale': scale, 'integer': isinstance(value, int)}) fields.append({'path': list(path), 'scale': scale, 'integer': command['type'] in ('drive', 'motors', 'servo', 'horn') or isinstance(value, int)})
values.append(normalize(value, scale)) values.append(normalize(value, scale))
return {'$number': len(values) - 1} return {'$number': len(values) - 1}
# Arrays (e.g. songs) are discrete exact demonstrations, not variable # Arrays (e.g. songs) are discrete exact demonstrations, not variable
@@ -89,7 +96,7 @@ def encode_command(command, schema, extend=False):
if not extend: if not extend:
raise ValueError('Command shape missing from model schema') raise ValueError('Command shape missing from model schema')
if len(schema) >= SLOTS: if len(schema) >= SLOTS:
raise ValueError('64 command shapes exhausted; recording skipped, not silently truncated') raise ValueError('64 command shapes exhausted; affected windows excluded')
slot = len(schema) slot = len(schema)
schema.append(descriptor) schema.append(descriptor)
vector = np.zeros(VALUES, dtype=np.float32) vector = np.zeros(VALUES, dtype=np.float32)
@@ -97,6 +104,66 @@ def encode_command(command, schema, extend=False):
return slot, vector return slot, vector
def initial_schema():
# Match commandPipeline.js envelopes, OI_COMMANDS and the four-frequency
# horn control. Values are examples describing fields, not training labels.
commands = []
for left, right in itertools.product((0, 1), repeat=2):
commands.append({'type': 'drive', 'driveDirect': {'left': left, 'right': right}})
for main, side, vacuum in itertools.product((0, 1), repeat=3):
commands.append({'type': 'motors', 'motorPwm': {'main': main, 'side': side, 'vacuum': vacuum}})
for angle in (0, 1):
commands.append({'type': 'servo', 'servo': {'angle': angle}})
for kind in ('headlight', 'laser'):
for action in ('on', 'off', 'toggle'):
commands.append({'type': kind, kind: {'action': action}})
for waveform in ('sine', 'saw'):
commands.append({'type': 'horn', 'horn': {'action': 'start', 'waveform': waveform, 'freqs': [1, 1, 1, 1]}})
commands.append({'type': 'horn', 'horn': {'action': 'stop'}})
for opcode in (128, 131, 132, 143):
commands.append({'type': 'raw', 'raw': base64.b64encode(bytes([opcode])).decode()})
commands.append({'type': 'song', 'song': {'notes': [{'note': 60, 'duration': 8}]}})
schema = []
for command in commands:
encode_command(command, schema, extend=True)
return schema
def command_family(command):
return (command['type'],)
def select_slots(scores, schema, threshold):
winners = {}
for slot, descriptor in enumerate(schema):
if scores[slot] < threshold:
continue
family = command_family(descriptor['template'])
if family not in winners or scores[slot] > scores[winners[family]]:
winners[family] = slot
return set(winners.values())
def decode_command(descriptor, values):
command = copy.deepcopy(descriptor['template'])
for index, field in enumerate(descriptor['fields']):
number = float(values[index])
if not math.isfinite(number):
raise ValueError('Nonfinite control value')
number = math.tan(max(-.98, min(.98, number)) * math.pi / 2) * field['scale']
target = command
for key in field['path'][:-1]:
target = target[key]
target[field['path'][-1]] = round(number) if field['integer'] else number
return command
def is_stop(descriptor):
command = descriptor['template']
direct = command.get('driveDirect') or {}
return command['type'] == 'drive' and direct.get('left') == 0 and direct.get('right') == 0
class Policy(nn.Module): class Policy(nn.Module):
def __init__(self): def __init__(self):
super().__init__() super().__init__()
@@ -109,7 +176,6 @@ class Policy(nn.Module):
nn.Linear(128 * 3 * 4, 128), nn.ReLU(), nn.Linear(128 * 3 * 4, 128), nn.ReLU(),
) )
self.history = nn.GRU(128 + STATE_SIZE, 128, batch_first=True) self.history = nn.GRU(128 + STATE_SIZE, 128, batch_first=True)
self.wheels = nn.Linear(128, 2)
self.actions = nn.Linear(128, SLOTS) self.actions = nn.Linear(128, SLOTS)
self.values = nn.Linear(128, SLOTS * VALUES) self.values = nn.Linear(128, SLOTS * VALUES)
@@ -118,17 +184,16 @@ class Policy(nn.Module):
encoded = self.vision(images.reshape(-1, 3, HEIGHT, WIDTH)).reshape(batch, frames, -1) encoded = self.vision(images.reshape(-1, 3, HEIGHT, WIDTH)).reshape(batch, frames, -1)
history, _ = self.history(torch.cat((encoded, state), dim=-1)) history, _ = self.history(torch.cat((encoded, state), dim=-1))
last = history[:, -1] last = history[:, -1]
return self.actions(last), self.values(last).reshape(batch, SLOTS, VALUES).tanh(), self.wheels(last).tanh() return self.actions(last), self.values(last).reshape(batch, SLOTS, VALUES).tanh()
def specification(schema): def specification(schema):
return {'version': VERSION, 'architecture': 'spatial-cnn-gru-wheels-v2', return {'version': VERSION, 'architecture': 'spatial-cnn-gru-events-v4',
'frames': FRAMES, 'height': HEIGHT, 'width': WIDTH, 'frames': FRAMES, 'height': HEIGHT, 'width': WIDTH,
'historyStrideMs': 300, 'decisionIntervalMs': 100, 'historyStrideMs': 300, 'decisionIntervalMs': 100,
'stateSize': STATE_SIZE, 'sensorFields': SENSORS, 'stateSize': STATE_SIZE, 'sensorFields': SENSORS,
'normalization': '2/pi * atan(value/scale); missing sensors have mask=0', 'normalization': '2/pi * atan(value/scale); missing sensors have mask=0',
'maxCommandShapes': SLOTS, 'numericFieldsPerShape': VALUES, 'maxCommandShapes': SLOTS, 'numericFieldsPerShape': VALUES,
'commands': schema, 'actionSemantics': 'Continuous wheels at +100ms; accessories are events in the next 100ms', 'commands': schema, 'actionSemantics': 'All controls are command events in the next 100ms; no event preserves existing rover state',
'wheelScaleMmPerSecond': 500, 'wheelOrder': ['left', 'right'], 'driveCommandHistoryInput': False, 'zeroNumericValues': 'Exact template constants',
'wheelCommandExpiryMs': None, 'unknownWheelIntent': 'Excluded until next accepted human drive',
'timing': 'Server-receipt alignment; browser display latency unmeasured'} 'timing': 'Server-receipt alignment; browser display latency unmeasured'}
@@ -41,11 +41,11 @@ def main():
import torch import torch
from torch.nn import functional as functional from torch.nn import functional as functional
from dataset import windows from dataset import windows
from policy import Policy, VERSION, SLOTS, VALUES, specification from policy import Policy, VERSION, SLOTS, VALUES, specification, is_stop, select_slots, initial_schema
if args.check_dependencies: if args.check_dependencies:
emit('dependencies', torch=str(torch.__version__), numpy=str(np.__version__), av=str(av.__version__), emit('dependencies', torch=str(torch.__version__), numpy=str(np.__version__), av=str(av.__version__),
parameters=sum(p.numel() for p in Policy().parameters()), specification=specification([])) parameters=sum(p.numel() for p in Policy().parameters()), specification=specification(initial_schema()))
return return
if not args.job: if not args.job:
parser.error('--job is required unless --check-dependencies is used') parser.error('--job is required unless --check-dependencies is used')
@@ -70,9 +70,10 @@ def main():
cancelled = lambda: stopped or time.monotonic() - started >= config['maxJobSeconds'] cancelled = lambda: stopped or time.monotonic() - started >= config['maxJobSeconds']
model = Policy().cpu() model = Policy().cpu()
optimizer = torch.optim.AdamW(model.parameters(), lr=0.0001) optimizer = torch.optim.AdamW(model.parameters(), lr=0.0001)
# A fresh training lineage prevents previously trained footage leaking into validation. # The command-event policy has different inputs, targets and weights. Keep its
resume_path = root / 'training' / 'resume-v3.pt' # optimizer lineage separate; reuse recordings with their unchanged partition.
schema, steps, examples, last_published = [], 0, 0, 0 resume_path = root / 'training' / 'resume-v5.pt'
schema, steps, examples, last_published = initial_schema(), 0, 0, 0
trained_rovers = set() trained_rovers = set()
control_profiles = [] control_profiles = []
distinct_windows = {} distinct_windows = {}
@@ -107,25 +108,24 @@ def main():
candidate_schema = copy.deepcopy(schema) candidate_schema = copy.deepcopy(schema)
samples, seen = [], 0 samples, seen = [], 0
stats = {} stats = {}
buckets = {key: [] for key in ('stopped', 'forward', 'reverse', 'turn')} buckets = {key: [] for key in ('stop', 'command', 'none')}
counts = dict.fromkeys(buckets, 0) counts = dict.fromkeys(buckets, 0)
held_out = is_held_out(session_id) held_out = is_held_out(session_id)
try: try:
directory = root / 'recordings' / session_id directory = root / 'recordings' / session_id
for sample in windows(directory, candidate_schema, cancelled, stats): for sample in windows(directory, candidate_schema, cancelled, stats, extend=not held_out):
seen += 1 seen += 1
# Keep evaluation naturally distributed. Training reserves space # Keep evaluation naturally distributed. Training reserves space
# for ordinary turns/reversing so idle or straight motion cannot # for recorded commands and releases so periods without commands
# consume the entire sample budget. No scripted driving required. # cannot consume the entire sample budget. No scripted driving required.
if held_out: if held_out:
reservoir, count, capacity = samples, seen, per_session reservoir, count, capacity = samples, seen, per_session
else: else:
left, right = sample[5] category = ('stop' if any(sample[2][slot] and is_stop(descriptor)
category = ('stopped' if max(abs(left), abs(right)) <= .02 else for slot, descriptor in enumerate(candidate_schema)) else
'turn' if abs(left - right) > .1 else 'command' if sample[2].any() else 'none')
'reverse' if left + right < 0 else 'forward')
counts[category] += 1 counts[category] += 1
reservoir, count, capacity = buckets[category], counts[category], max(1, per_session // 4) reservoir, count, capacity = buckets[category], counts[category], max(1, per_session // 3)
if len(reservoir) < capacity: if len(reservoir) < capacity:
reservoir.append(sample) reservoir.append(sample)
else: else:
@@ -137,15 +137,15 @@ def main():
if cancelled(): if cancelled():
break # Do not call a partially decoded recording validated. break # Do not call a partially decoded recording validated.
if not samples: if not samples:
if held_out and stats.get('unsupportedCommandEvents'):
dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': 0,
'skipReason': 'No supported aligned windows'})
continue
raise ValueError('No complete, aligned observation/action windows') raise ValueError('No complete, aligned observation/action windows')
# Entire sessions stay in one partition across all jobs. Neighboring # Entire sessions stay in one partition across all jobs. Neighboring
# video windows must never straddle training and validation. # video windows must never straddle training and validation.
held_out = is_held_out(session_id) held_out = is_held_out(session_id)
if held_out: if held_out:
if candidate_schema != schema:
dataset_stats.append({**stats, 'partition': 'heldOut', 'sampledWindows': len(samples),
'skipReason': 'Vocabulary absent from training data'})
continue # Retry after training data introduces these command shapes.
validation.extend(samples) validation.extend(samples)
else: else:
schema = candidate_schema schema = candidate_schema
@@ -164,7 +164,7 @@ def main():
except (ValueError, KeyError, TypeError, OSError, EOFError, IndexError, OverflowError, av.error.FFmpegError) as error: except (ValueError, KeyError, TypeError, OSError, EOFError, IndexError, OverflowError, av.error.FFmpegError) as error:
rejected[session_id] = str(error)[:500] rejected[session_id] = str(error)[:500]
emit('skipped', sessionId=session_id, reason=rejected[session_id]) emit('skipped', sessionId=session_id, reason=rejected[session_id])
if len(pool) < config['minimumSamples'] or not any(np.max(np.abs(sample[5])) > .02 for sample in pool) or cancelled(): if len(pool) < config['minimumSamples'] or not any(sample[2].any() for sample in pool) or cancelled():
emit('complete', trained=False, accepted=[], rejected=rejected, samples=len(pool), emit('complete', trained=False, accepted=[], rejected=rejected, samples=len(pool),
diagnostics={'dataset': dataset_stats, 'decodeSeconds': time.monotonic() - decode_started}, diagnostics={'dataset': dataset_stats, 'decodeSeconds': time.monotonic() - decode_started},
reason='Stopped/time limit' if cancelled() else 'Waiting for enough training-partition samples with human actions') reason='Stopped/time limit' if cancelled() else 'Waiting for enough training-partition samples with human actions')
@@ -188,17 +188,16 @@ def main():
if cancelled() or steps - initial_steps >= config['maxStepsPerJob']: if cancelled() or steps - initial_steps >= config['maxStepsPerJob']:
break break
batch = pool[offset:offset + config['batchSize']] batch = pool[offset:offset + config['batchSize']]
images, states, targets, values, available, wheel_targets, previous_wheels = [torch.from_numpy(np.stack(items)) for items in zip(*batch)] images, states, targets, values, available = [torch.from_numpy(np.stack(items)) for items in zip(*batch)]
images = images.float().div_(255) images = images.float().div_(255)
optimizer.zero_grad(set_to_none=True) optimizer.zero_grad(set_to_none=True)
action_logits, predictions, wheel_predictions = model(images, states) action_logits, predictions = model(images, states)
event_loss = functional.binary_cross_entropy_with_logits(action_logits, targets, event_loss = functional.binary_cross_entropy_with_logits(action_logits, targets,
pos_weight=positive_weight, reduction='none') pos_weight=positive_weight, reduction='none')
event_loss = (event_loss * available).sum() / available.sum().clamp_min(1) event_loss = (event_loss * available).sum() / available.sum().clamp_min(1)
mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1) mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1)
value_loss = ((predictions - values).square() * mask).sum() / mask.sum().clamp_min(1) value_loss = ((predictions - values).square() * mask).sum() / mask.sum().clamp_min(1)
wheel_loss = functional.smooth_l1_loss(wheel_predictions, wheel_targets) loss = event_loss + value_loss
loss = wheel_loss + event_loss + value_loss
if not torch.isfinite(loss): if not torch.isfinite(loss):
raise ValueError('Nonfinite training loss; checkpoint not saved') raise ValueError('Nonfinite training loss; checkpoint not saved')
loss.backward() loss.backward()
@@ -209,7 +208,7 @@ def main():
losses.append(loss.item()) losses.append(loss.item())
if steps % 10 == 0: if steps % 10 == 0:
emit('progress', steps=steps, examples=examples, loss=loss.item(), emit('progress', steps=steps, examples=examples, loss=loss.item(),
losses={'wheels': wheel_loss.item(), 'events': event_loss.item(), 'accessoryValues': value_loss.item()}) losses={'events': event_loss.item(), 'commandValues': value_loss.item()})
if cancelled() or steps - initial_steps >= config['maxStepsPerJob']: if cancelled() or steps - initial_steps >= config['maxStepsPerJob']:
break break
if steps == initial_steps: if steps == initial_steps:
@@ -220,64 +219,95 @@ def main():
evaluation_started = time.monotonic() evaluation_started = time.monotonic()
distinct_windows.update(observed_windows) distinct_windows.update(observed_windows)
unique_count = sum(distinct_windows.values()) unique_count = sum(distinct_windows.values())
evaluation = {'status': 'waiting for held-out sessions', 'windows': len(validation)} coverage = [row for row in dataset_stats if row.get('partition') == 'heldOut']
unsupported_events = sum(row.get('unsupportedCommandEvents', 0) for row in coverage)
unsupported_windows = sum(row.get('unsupportedWindows', 0) for row in coverage)
evaluation = {'status': 'No supported held-out windows' if unsupported_events and not validation else 'waiting for held-out sessions',
'windows': len(validation), 'unsupportedCommandEvents': unsupported_events,
'unsupportedWindows': unsupported_windows}
if validation and not cancelled(): if validation and not cancelled():
model.eval() model.eval()
true_positive = predicted_positive = actual_positive = 0 true_positive = predicted_positive = actual_positive = 0
value_error = value_count = 0 value_error = value_count = evaluated = 0
evaluated = 0 stop_actual = stop_detected = false_stops = 0
wheel_error = stopped_error = persistence_error = 0 drive_actual = drive_detected = 0
changed_error = changed_persistence = changed_count = moving_count = 0 empty_windows = unwanted_windows = 0
stop_slots = [slot for slot, descriptor in enumerate(schema) if is_stop(descriptor)]
drive_slots = [slot for slot, descriptor in enumerate(schema)
if descriptor['template']['type'] == 'drive' and not is_stop(descriptor)]
per_command = [{'slot': slot, 'command': descriptor['template'], 'actual': 0,
'predicted': 0, 'matched': 0} for slot, descriptor in enumerate(schema)]
with torch.inference_mode(): with torch.inference_mode():
for offset in range(0, len(validation), config['batchSize']): for offset in range(0, len(validation), config['batchSize']):
if cancelled(): if cancelled():
break break
batch = validation[offset:offset + config['batchSize']] batch = validation[offset:offset + config['batchSize']]
images, states, targets, values, available, wheel_targets, previous_wheels = [torch.from_numpy(np.stack(items)) for items in zip(*batch)] images, states, targets, values, available = [torch.from_numpy(np.stack(items)) for items in zip(*batch)]
logits, predictions, wheel_predictions = model(images.float() / 255, states) logits, predictions = model(images.float() / 255, states)
predicted = (logits.sigmoid() >= job['actionThreshold']) & available.bool() predicted = torch.zeros_like(targets, dtype=torch.bool)
actual = targets.bool() & available.bool() # Use the same threshold and competing-command selection as live
# inference. Reservoir samples cannot assess temporal latching or
# closed-loop motion; report this explicitly rather than implying it.
for row, scores in enumerate(logits.sigmoid().tolist()):
for slot in select_slots(scores, schema, job['actionThreshold']):
predicted[row, slot] = True
actual = targets.bool()
true_positive += (predicted & actual).sum().item() true_positive += (predicted & actual).sum().item()
predicted_positive += predicted.sum().item() predicted_positive += predicted.sum().item()
actual_positive += actual.sum().item() actual_positive += actual.sum().item()
mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1) mask = numeric_mask.unsqueeze(0) * targets.unsqueeze(-1)
value_error += ((predictions - values).square() * mask).sum().item() value_error += ((predictions - values).square() * mask).sum().item()
value_count += mask.sum().item() value_count += mask.sum().item()
errors = (wheel_predictions - wheel_targets).abs().mean(dim=1) * 500 empty = ~actual.any(dim=1)
persistence = (previous_wheels - wheel_targets).abs().mean(dim=1) * 500 empty_windows += empty.sum().item()
changed = (previous_wheels - wheel_targets).abs().amax(dim=1) > .02 unwanted_windows += (empty & predicted.any(dim=1)).sum().item()
changed_error += errors[changed].sum().item() if stop_slots:
changed_persistence += persistence[changed].sum().item() stop_actual += actual[:, stop_slots].sum().item()
changed_count += changed.sum().item() stop_detected += (actual[:, stop_slots] & predicted[:, stop_slots]).sum().item()
moving_count += (wheel_targets.abs().amax(dim=1) > .02).sum().item() false_stops += (predicted[:, stop_slots] & ~actual[:, stop_slots]).sum().item()
wheel_error += errors.sum().item() if drive_slots:
stopped_error += wheel_targets.abs().mean(dim=1).sum().item() * 500 drive_actual += actual[:, drive_slots].sum().item()
persistence_error += persistence.sum().item() drive_detected += (actual[:, drive_slots] & predicted[:, drive_slots]).sum().item()
for item in per_command:
slot = item['slot']
item['actual'] += actual[:, slot].sum().item()
item['predicted'] += predicted[:, slot].sum().item()
item['matched'] += (actual[:, slot] & predicted[:, slot]).sum().item()
evaluated += len(batch) evaluated += len(batch)
precision = true_positive / max(1, predicted_positive) precision = true_positive / max(1, predicted_positive)
recall = true_positive / max(1, actual_positive) recall = true_positive / max(1, actual_positive)
mse = value_error / max(1, value_count) mse = value_error / max(1, value_count)
wheel_mae = wheel_error / max(1, evaluated) stop_recall = stop_detected / max(1, stop_actual)
stopped_mae = stopped_error / max(1, evaluated) held_out_sessions = sum(row.get('partition') == 'heldOut' and not row.get('skipReason') for row in dataset_stats)
persistence_mae = persistence_error / max(1, evaluated) gate_reasons = []
changed_mae = changed_error / max(1, changed_count) for condition, reason in [
changed_baseline = changed_persistence / max(1, changed_count) (unsupported_events == 0, 'Held-out controls are not fully represented; see unsupportedCommands'),
# Check control changes separately: copying the previous speed can look (evaluated == len(validation), 'Evaluation interrupted'),
# excellent on long straight runs without learning to steer or stop. (held_out_sessions >= 2, 'Need two usable held-out recordings'),
wheels_pass = (evaluated == len(validation) and evaluated >= 128 and moving_count >= 32 (evaluated >= 128, 'Need 128 held-out windows'),
and changed_count >= 32 and wheel_mae < stopped_mae (actual_positive >= 32, 'Need 32 held-out command events'),
and wheel_mae <= persistence_mae and changed_mae < changed_baseline) (stop_actual >= 8, 'Need eight held-out stop events'),
accessories_pass = (predicted_positive == 0 if actual_positive == 0 else (drive_actual >= 16, 'Need 16 held-out movement commands'),
actual_positive >= 32 and precision >= .5 and recall >= .5) (drive_detected / max(1, drive_actual) >= .5, 'Movement command recall below 0.5'),
passed = wheels_pass and accessories_pass and (value_count == 0 or mse <= .1) (precision >= .5, 'Command precision below 0.5'),
evaluation = {'status': 'passed imitation gate' if passed else 'imitation gate not met', (recall >= .5, 'Command recall below 0.5'),
(stop_recall >= .5, 'Stop recall below 0.5'),
(value_count == 0 or mse <= .1, 'Command parameter error above 0.1'),
]:
if not condition:
gate_reasons.append(reason)
evaluation = {'status': 'imitation gate not met' if gate_reasons else 'passed imitation gate',
'unsupportedCommandEvents': unsupported_events, 'unsupportedWindows': unsupported_windows,
'method': 'Held-out 100ms command events; not a closed-loop rollout',
'threshold': job['actionThreshold'], 'heldOutSessions': held_out_sessions,
'windows': evaluated, 'precision': precision, 'recall': recall, 'windows': evaluated, 'precision': precision, 'recall': recall,
'numericMse': mse, 'positiveActions': actual_positive, 'numericMse': mse, 'positiveActions': actual_positive,
'wheelMaeMmPerSecond': wheel_mae, 'stoppedBaselineMaeMmPerSecond': stopped_mae, 'predictedActions': predicted_positive, 'falsePositiveActions': predicted_positive - true_positive,
'previousSpeedBaselineMaeMmPerSecond': persistence_mae, 'stopEvents': stop_actual, 'missedStops': stop_actual - stop_detected,
'changedWheelMaeMmPerSecond': changed_mae, 'changedBaselineMaeMmPerSecond': changed_baseline, 'stopRecall': stop_recall, 'falseStops': false_stops,
'changedWindows': changed_count, 'movingWindows': moving_count, 'movementEvents': drive_actual, 'movementRecall': drive_detected / max(1, drive_actual),
'wheelsPassed': wheels_pass, 'accessoriesPassed': accessories_pass} 'noCommandWindows': empty_windows, 'unwantedCommandWindows': unwanted_windows,
'commands': per_command, 'gateReasons': gate_reasons}
evaluation_seconds = time.monotonic() - evaluation_started evaluation_seconds = time.monotonic() - evaluation_started
publication = {'distinctMinutes': unique_count / 600, publication = {'distinctMinutes': unique_count / 600,
'newMinutes': (unique_count - published_windows) / 600, 'newMinutes': (unique_count - published_windows) / 600,
@@ -1,3 +1,4 @@
const { getOperatingMode } = require('../operatingModeService');
// rover Manager private access // rover Manager private access
// Purpose: Encapsulates private-rover visibility and control-authorization policy checks. // Purpose: Encapsulates private-rover visibility and control-authorization policy checks.
// Scope: Keeps runtime behavior unchanged while isolating policy decisions from orchestration wiring. // Scope: Keeps runtime behavior unchanged while isolating policy decisions from orchestration wiring.
@@ -80,6 +81,7 @@ function createPrivateAccessPolicy(deps) {
allowClosedPrivateGrantInLockdown = false, allowClosedPrivateGrantInLockdown = false,
allowClosedPrivateCurrentDriver = false, allowClosedPrivateCurrentDriver = false,
} = options; } = options;
if (getOperatingMode(socket) !== 'rover') return 'Leave PTZ before requesting rover control';
if (!record) return 'Unknown rover'; if (!record) return 'Unknown rover';
if (!allowUser && !isAdmin(socket)) return 'Only admins can request control'; if (!allowUser && !isAdmin(socket)) return 'Only admins can request control';
if (record.locked && !isAdmin(socket)) return 'Rover locked'; if (record.locked && !isAdmin(socket)) return 'Rover locked';
+33 -30
View File
@@ -5,16 +5,18 @@ const io = require('../../globals/io');
const logger = require('../../globals/logger').child('sessionService'); const logger = require('../../globals/logger').child('sessionService');
const { getFeatureFlags, configurationEvents } = require('../../configuration'); const { getFeatureFlags, configurationEvents } = require('../../configuration');
const { getRole, isAdmin, roleEvents } = require('../roleService'); const { getRole, isAdmin, roleEvents } = require('../roleService');
const { getOperatingMode, setOperatingMode, operatingModeEvents } = require('../operatingModeService');
const { getMode, modeEvents } = require('../modeManager'); const { getMode, modeEvents } = require('../modeManager');
const roverManager = require('../roverManager'); const roverManager = require('../roverManager');
const { managerEvents } = roverManager; const { managerEvents } = roverManager;
const assignmentService = require('../assignmentService'); const assignmentService = require('../assignmentService');
const { getActiveDrivers, getTurnQueues, turnEvents } = require('../turnService'); const { getActiveDrivers, getTurnQueues, canDrive, turnEvents } = require('../turnService');
const { buildRoverTurn } = require('../turnService/display');
const { getRoomCameras, roomCameraEvents } = require('../roomCameraService'); const { getRoomCameras, roomCameraEvents } = require('../roomCameraService');
const { const {
getPublicState: getPtzCameraState, getPublicState: getPtzCameraState,
getChatTargetForSocket: getPtzChatTargetForSocket, getOperatingModeDisplay,
PTZ_CAMERA_ID, getParticipantSocketIds,
ptzCameraEvents, ptzCameraEvents,
} = require('../ptzCameraService'); } = require('../ptzCameraService');
const { getState: getHomeAssistantState, homeAssistantEvents } = require('../homeAssistantService'); const { getState: getHomeAssistantState, homeAssistantEvents } = require('../homeAssistantService');
@@ -111,18 +113,13 @@ function buildBandwidthSavingsSessionState(socket, controllableUserCount = 0) {
function countControllableUsers(userEntries = []) { function countControllableUsers(userEntries = []) {
const ids = new Set(); const ids = new Set();
const ptzParticipants = new Set(getParticipantSocketIds());
userEntries.forEach((entry) => { userEntries.forEach((entry) => {
const role = String(entry?.role || ''); const role = String(entry?.role || '');
if (role === 'spectator') return; if (role === 'spectator') return;
const socketId = String(entry?.socketId || '').trim(); const socketId = String(entry?.socketId || '').trim();
const roverId = String(entry?.roverId || '').trim(); const roverId = String(entry?.roverId || '').trim();
/* if (socketId && (roverId || ptzParticipants.has(socketId))) ids.add(socketId);
buildUserEntry already maps PTZ queued/operators to the PTZ pseudo-rover
id and normal drivers to their physical rover. Counting entries after that
normalization gives the browser the same conceptual "controllable users"
count it shows in the user/queue panels without duplicating PTZ UI logic.
*/
if (socketId && roverId) ids.add(socketId);
}); });
return ids.size; return ids.size;
} }
@@ -146,19 +143,14 @@ function buildUserEntry(socket) {
&& roverManager.isDriver(assignment.roverId, socket) && roverManager.isDriver(assignment.roverId, socket)
? assignment.roverId ? assignment.roverId
: null; : null;
const ptzChatTarget = getPtzChatTargetForSocket(socket.id);
return { return {
socketId: socket.id, socketId: socket.id,
userId: socket?.data?.userId || null, userId: socket?.data?.userId || null,
nickname: getNickname(socket) || null, nickname: getNickname(socket) || null,
role, role,
/* operatingMode: getOperatingMode(socket),
PTZ is not inserted into the physical rover roster, but for chat and user operatingModeDisplay: getOperatingModeDisplay(socket.id),
presence it should read like the user moved to a rover-like target. Prefer roverId: verifiedPrimaryRover || verifiedAssignmentRover || null,
the PTZ chat target while the socket is queued or operating so presence,
queue lookup, and chat identity all agree.
*/
roverId: ptzChatTarget?.roverId || verifiedPrimaryRover || verifiedAssignmentRover || null,
}; };
} }
@@ -171,15 +163,7 @@ function buildSession(socket) {
const controllableUserCount = countControllableUsers(userEntries); const controllableUserCount = countControllableUsers(userEntries);
const users = userEntries.map((entry) => ({ const users = userEntries.map((entry) => ({
...entry, ...entry,
/* roverId: filterVisibleRoverId(socket, entry.roverId),
PTZ is intentionally not a roverManager record, so the normal physical
rover visibility filter would erase the user's PTZ chat target. Preserve
it here because getPtzChatTargetForSocket already applied the PTZ access
and queue/operator rules before buildUserEntry returned it.
*/
roverId: entry.roverId === PTZ_CAMERA_ID
? entry.roverId
: filterVisibleRoverId(socket, entry.roverId),
})); }));
const roster = roverManager.getRosterForSocket(socket); const roster = roverManager.getRosterForSocket(socket);
const assignment = assignmentService.describeAssignment(socket?.id || ''); const assignment = assignmentService.describeAssignment(socket?.id || '');
@@ -201,6 +185,7 @@ function buildSession(socket) {
return { return {
socketId: socket?.id || null, socketId: socket?.id || null,
role: getRole(socket), role: getRole(socket),
operatingMode: getOperatingMode(socket),
mode: getMode(), mode: getMode(),
isLocalNetwork: isLocalNetwork(getSocketIp(socket)), isLocalNetwork: isLocalNetwork(getSocketIp(socket)),
bandwidthSavings: buildBandwidthSavingsSessionState(socket, controllableUserCount), bandwidthSavings: buildBandwidthSavingsSessionState(socket, controllableUserCount),
@@ -210,15 +195,24 @@ function buildSession(socket) {
of navigation and layout without maintaining another feature registry. of navigation and layout without maintaining another feature registry.
*/ */
features, features,
roster, roster: roster.map((rover) => ({
...rover,
permissions: { canControl: getRole(socket) !== 'spectator' && canDrive(rover.id, socket) },
turn: buildRoverTurn({
rover,
mode: getMode(),
socketId: socket?.id,
turnInfo: turnQueues[rover.id],
activeDriverId: activeDrivers[rover.id],
users,
}),
})),
odometers: roverManager.getOdometersForSocket(socket), odometers: roverManager.getOdometersForSocket(socket),
assignment: { assignment: {
...assignment, ...assignment,
roverId: assignmentRoverId, roverId: assignmentRoverId,
status: assignmentRoverId ? assignment.status : assignment.status === 'waiting' ? 'waiting' : null, status: assignmentRoverId ? assignment.status : assignment.status === 'waiting' ? 'waiting' : null,
}, },
activeDrivers,
turnQueues,
roomCameras: getRoomCameras(), roomCameras: getRoomCameras(),
ptzCamera: getPtzCameraState(socket), ptzCamera: getPtzCameraState(socket),
homeAssistant: getHomeAssistantState(), homeAssistant: getHomeAssistantState(),
@@ -280,10 +274,19 @@ function syncAll() {
} }
io.on('connection', (socket) => { io.on('connection', (socket) => {
socket.on('session:setOperatingMode', async ({ operatingMode } = {}, cb = () => {}) => {
try {
cb({ ok: true, ...await setOperatingMode(socket, operatingMode) });
} catch (err) {
cb({ error: err.message });
}
});
logger.info('New socket connected', socket.id); logger.info('New socket connected', socket.id);
syncSocket(socket); syncSocket(socket);
}); });
operatingModeEvents.on('change', () => syncAll());
roleEvents.on('change', ({ socket }) => { roleEvents.on('change', ({ socket }) => {
if (!socket) return; if (!socket) return;
logger.info('Role changed; syncing session', socket.id); logger.info('Role changed; syncing session', socket.id);
@@ -0,0 +1,39 @@
const { describeQueue } = require('./queueDisplay');
const { TURN_DURATION_MS, IDLE_TIMEOUT_MS } = require('./constants');
const labels = {
...require('./labels'),
waiting: 'Someone else is driving',
active: 'You’re driving',
showName: 'Show rover name',
hideName: 'Hide rover name',
};
// Build this from visibility-filtered session data so private rover state stays private.
function buildRoverTurn({ rover, mode, socketId, turnInfo, activeDriverId, users }) {
const queue = turnInfo?.queue || [];
// Direct ownership can arrive before queue details on load/reconnect.
const currentDriverId = activeDriverId || turnInfo?.current || null;
const currentIndex = currentDriverId ? queue.indexOf(currentDriverId) : -1;
const userIndex = socketId ? queue.indexOf(socketId) : -1;
const enabled = mode === 'turns' && queue.length > 1 && currentIndex >= 0 && userIndex >= 0;
const turnsAhead = enabled ? (userIndex - currentIndex + queue.length) % queue.length : null;
const display = describeQueue(queue, currentDriverId, users);
return {
target: { id: rover.id, name: rover.name, color: rover.color, fallback: rover.id },
...display,
// Null distinguishes a rover that has never had a queue from an empty queue.
queue: turnInfo ? display.queue : null,
enabled,
ownsControl: Boolean(socketId && currentDriverId === socketId),
turnsAhead,
queueLength: queue.length,
deadline: turnInfo?.deadline || null,
durationMs: TURN_DURATION_MS,
idleDeadline: turnInfo?.idleDeadline || null,
idleGraceMs: IDLE_TIMEOUT_MS,
labels,
};
}
module.exports = { buildRoverTurn };
+1
View File
@@ -397,6 +397,7 @@ function advanceTurn(roverId) {
return; return;
} }
if (queue.queue.length === 0) { if (queue.queue.length === 0) {
queue.current = null;
stopRover(roverId); stopRover(roverId);
clearTurnTimer(roverId, queue); clearTurnTimer(roverId, queue);
setActiveDriver(roverId, null); setActiveDriver(roverId, null);
+10
View File
@@ -0,0 +1,10 @@
module.exports = {
handoff: 'It’s your turn!',
idleWarning: 'Start driving or your turn will be skipped',
activeTimer: 'Your turn',
waitingTimer: 'Waiting',
untilTurn: 'until your turn',
timeRemaining: 'left',
showTimer: 'Show turn timer',
hideTimer: 'Hide turn timer',
};
@@ -0,0 +1,20 @@
// Queue entries are complete display records. No browser-side user lookup or
// naming fallback is needed, including when the queue comes from another server.
function describeQueue(socketIds, currentId, users) {
const byId = new Map(users.map((user) => [user.socketId, user]));
const currentIndex = currentId ? socketIds.indexOf(currentId) : -1;
return {
queue: socketIds.map((socketId) => {
const user = byId.get(socketId);
return {
socketId,
name: user?.nickname || user?.name || socketId.slice(0, 6) || 'unknown',
role: user?.role || null,
};
}),
currentId,
nextId: socketIds.length > 1 ? socketIds[(currentIndex + 1) % socketIds.length] : null,
};
}
module.exports = { describeQueue };
@@ -157,11 +157,11 @@ function identifySocket(socket, payload = {}) {
} }
/* /*
Spectator-style pages carry identity too, but only the driver surface Spectator-style pages carry identity too, but only the driver and PTZ surfaces
participates in duplicate-driver enforcement. This flag remains on the participate in duplicate-driver enforcement. This flag remains on the
socket because it is connection-specific, not person-specific. socket because it is connection-specific, not person-specific.
*/ */
socket.data.identitySurface = payload.identitySurface === 'driver' ? 'driver' : 'passive'; socket.data.identitySurface = ['driver', 'ptz'].includes(payload.identitySurface) ? payload.identitySurface : 'passive';
const result = identifyCanonicalSocket(socket, { const result = identifyCanonicalSocket(socket, {
...payload, ...payload,
@@ -207,7 +207,7 @@ function enforceSingleDriverSocketPerIdentity(currentSocket) {
if ( if (
!currentSocket?.id || !currentSocket?.id ||
!currentUserId || !currentUserId ||
currentSocket.data?.identitySurface !== 'driver' || !['driver', 'ptz'].includes(currentSocket.data?.identitySurface) ||
!enforceForCurrentSocket !enforceForCurrentSocket
) { ) {
return; return;
@@ -215,7 +215,7 @@ function enforceSingleDriverSocketPerIdentity(currentSocket) {
const duplicates = Array.from(io.sockets.sockets.values()).filter((candidate) => { const duplicates = Array.from(io.sockets.sockets.values()).filter((candidate) => {
if (!candidate?.id || candidate.id === currentSocket.id || candidate.disconnected) return false; if (!candidate?.id || candidate.id === currentSocket.id || candidate.disconnected) return false;
if (candidate?.data?.identitySurface !== 'driver') return false; if (!['driver', 'ptz'].includes(candidate?.data?.identitySurface)) return false;
return getUserIdForSocket(candidate) === currentUserId; return getUserIdForSocket(candidate) === currentUserId;
}); });
+1 -1
View File
@@ -4,7 +4,7 @@
"version": "0.0.0", "version": "0.0.0",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "VITE_ROVERD_URL=http://192.168.0.86:8080 vite", "dev": "VITE_ROVERD_URL=https://rover.otter.land vite",
"build": "vite build", "build": "vite build",
"lint": "eslint .", "lint": "eslint .",
"test": "node --test src/controls/inputs/*.test.js src/components/GamepadMappingSettings/*.test.js", "test": "node --test src/controls/inputs/*.test.js src/components/GamepadMappingSettings/*.test.js",
@@ -132,11 +132,11 @@ export function ChatIdentity({ message, toolsToggle = null }) {
bot bot
</span> </span>
) : null} ) : null}
{message.roverId && ( {(message.roverId || message.operatingModeDisplay) && (
<RoverLabel <RoverLabel
roverId={message.roverId} roverId={message.roverId}
name={message.roverName} name={message.operatingModeDisplay?.name || message.roverName}
color={message.roverColor} color={message.operatingModeDisplay?.color || message.roverColor}
fallback={message.roverId} fallback={message.roverId}
className="shrink-0 text-[0.7rem]" className="shrink-0 text-[0.7rem]"
/> />
+5 -19
View File
@@ -51,33 +51,19 @@ function resolveTtsSettings(settings) {
function useChatComposerSessionState(allowSpectatorInput) { function useChatComposerSessionState(allowSpectatorInput) {
const role = useSessionSelector((state) => state.session?.role || null); const role = useSessionSelector((state) => state.session?.role || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null); const operatingMode = useSessionSelector((state) => state.session?.operatingMode || 'rover');
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null); const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
const users = useSessionSelector((state) => state.session?.users ?? []);
const roster = useSessionSelector((state) => state.session?.roster ?? []); const roster = useSessionSelector((state) => state.session?.roster ?? []);
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null); const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const chatTargetId = useMemo(() => {
/*
PTZ is intentionally not a roster entry, but session users expose the
current chat target for each socket. Prefer the self user entry so the TTS
control follows PTZ queue/operation state instead of only physical rover
assignment state.
*/
const self = users.find((entry) => entry?.socketId === socketId);
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
return self?.roverId || currentRoverId || null;
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
const rover = useMemo( const rover = useMemo(
() => roster.find((entry) => String(entry.id) === String(chatTargetId)) || null, () => roster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
[chatTargetId, roster], [currentRoverId, roster],
); );
const ptzTtsSupported = Boolean( const ptzTtsSupported = Boolean(
ptz?.id && operatingMode === 'ptz' &&
String(chatTargetId) === String(ptz.id) &&
ptz?.audio?.enabled && ptz?.audio?.enabled &&
(ptz?.isOperator || ptz?.queuedPosition), (ptz?.turn?.ownsControl || ptz?.turn?.turnsAhead > 0),
); );
return { return {
@@ -27,9 +27,8 @@ function detectSafari() {
function HudChatInput({ compact = false, variant = 'legacy', open = true, onOpenChange = null }) { function HudChatInput({ compact = false, variant = 'legacy', open = true, onOpenChange = null }) {
const role = useSessionSelector((state) => state.session?.role || null); const role = useSessionSelector((state) => state.session?.role || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null); const operatingMode = useSessionSelector((state) => state.session?.operatingMode || 'rover');
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null); const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
const users = useSessionSelector((state) => state.session?.users || []);
const roverRoster = useSessionSelector((state) => state.session?.roster || []); const roverRoster = useSessionSelector((state) => state.session?.roster || []);
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null); const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const { sendMessage, onInputFocus, onInputBlur, blurChat, registerInputRef, setTypingActive } = useChatActions(); const { sendMessage, onInputFocus, onInputBlur, blurChat, registerInputRef, setTypingActive } = useChatActions();
@@ -49,26 +48,14 @@ function HudChatInput({ compact = false, variant = 'legacy', open = true, onOpen
const { navigateHistory, resetHistoryNavigation } = useChatMessageHistoryNavigation(); const { navigateHistory, resetHistoryNavigation } = useChatMessageHistoryNavigation();
const canChat = role !== 'spectator'; const canChat = role !== 'spectator';
const hideHudChat = role === 'spectator'; const hideHudChat = role === 'spectator';
const chatTargetId = useMemo(() => {
/*
HUD chat does not render the full composer controls, but it still sends
TTS payloads when the active target supports speech. PTZ appears only in
the session user target, not the physical rover roster, so use the same
self-target resolution as the full chat panel.
*/
const self = users.find((entry) => entry?.socketId === socketId);
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
return self?.roverId || currentRoverId || null;
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
const rover = useMemo( const rover = useMemo(
() => roverRoster.find((entry) => String(entry.id) === String(chatTargetId)) || null, () => roverRoster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
[chatTargetId, roverRoster], [currentRoverId, roverRoster],
); );
const ptzTtsSupported = Boolean( const ptzTtsSupported = Boolean(
ptz?.id && operatingMode === 'ptz' &&
String(chatTargetId) === String(ptz.id) &&
ptz?.audio?.enabled && ptz?.audio?.enabled &&
(ptz?.isOperator || ptz?.queuedPosition), (ptz?.turn?.ownsControl || ptz?.turn?.turnsAhead > 0),
); );
const ttsSupported = Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported); const ttsSupported = Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported);
const ttsPayload = useMemo(() => { const ttsPayload = useMemo(() => {
@@ -40,10 +40,8 @@ function HudOverlay({
}); });
const derivedDriverLabel = useSessionSelector((state) => { const derivedDriverLabel = useSessionSelector((state) => {
if (!effectiveRoverId || variant !== 'spectator') return null; if (!effectiveRoverId || variant !== 'spectator') return null;
const activeId = state.session?.activeDrivers?.[effectiveRoverId] || null; const turn = state.session?.roster?.find((rover) => String(rover.id) === String(effectiveRoverId))?.turn;
const users = state.session?.users || []; return turn?.queue?.find((entry) => entry.socketId === turn.currentId)?.name || null;
const match = users.find((u) => String(u.socketId || '') === String(activeId || ''));
return match?.nickname || match?.name || null;
}); });
const resolvedSensors = sensors ?? null; const resolvedSensors = sensors ?? null;
const resolvedLabel = label ?? rosterInfo.label ?? null; const resolvedLabel = label ?? rosterInfo.label ?? null;
@@ -7,16 +7,13 @@ import SocialButton from '../../SocialButton/index.jsx';
function TurnsOverlay({ function TurnsOverlay({
roverId = null, roverId = null,
mobileHud = false, mobileHud = false,
turnModel = null,
}) { }) {
const assignedRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null); const assignedRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
const effectiveRoverId = turnModel?.targetId ?? roverId ?? assignedRoverId; const effectiveRoverId = roverId ?? assignedRoverId;
const mode = useSessionSelector((state) => state.session?.mode || null); const mode = useSessionSelector((state) => state.session?.mode || null);
const roster = useSessionSelector((state) => state.session?.roster ?? []); const roster = useSessionSelector((state) => state.session?.roster ?? []);
const users = useSessionSelector((state) => state.session?.users ?? []); const users = useSessionSelector((state) => state.session?.users ?? []);
const turnQueues = useSessionSelector((state) => state.session?.turnQueues ?? {});
const socketId = useSessionSelector((state) => state.session?.socketId || null); const socketId = useSessionSelector((state) => state.session?.socketId || null);
const activeDrivers = useSessionSelector((state) => state.session?.activeDrivers ?? {});
const lastControlIntentAt = useControlSelector((control) => control.state.lastControlIntentAt); const lastControlIntentAt = useControlSelector((control) => control.state.lastControlIntentAt);
const [showTurnCue, setShowTurnCue] = useState(false); const [showTurnCue, setShowTurnCue] = useState(false);
const [turnCueStartAt, setTurnCueStartAt] = useState(null); const [turnCueStartAt, setTurnCueStartAt] = useState(null);
@@ -30,30 +27,15 @@ function TurnsOverlay({
const subClass = mobileHud ? 'text-xs' : 'text-sm'; const subClass = mobileHud ? 'text-xs' : 'text-sm';
const cueTimerClass = mobileHud ? 'text-[0.55rem]' : 'text-[0.75rem]'; const cueTimerClass = mobileHud ? 'text-[0.55rem]' : 'text-[0.75rem]';
const cuePadClass = mobileHud ? 'px-4 py-3' : 'px-6 py-4'; const cuePadClass = mobileHud ? 'px-4 py-3' : 'px-6 py-4';
const turnInfo = turnModel ? null : effectiveRoverId ? turnQueues?.[effectiveRoverId] : null; const turnInfo = roster.find((rover) => String(rover.id) === String(effectiveRoverId))?.turn;
const activeDriverId = turnModel const activeDriverId = turnInfo?.currentId || null;
? turnModel.activeId || null const isActiveDriver = Boolean(turnInfo?.ownsControl);
: effectiveRoverId const isTurnsMode = mode === 'turns';
? activeDrivers?.[effectiveRoverId] || null
: null;
const isActiveDriver = turnModel
? Boolean(turnModel.isActive)
: Boolean(socketId && activeDriverId === socketId);
const isTurnsMode = turnModel ? Boolean(turnModel.enabled) : mode === 'turns';
const now = useSharedClock(1000, isTurnsMode); const now = useSharedClock(1000, isTurnsMode);
const nextDriverId = useMemo(() => { const nextDriverId = turnInfo?.currentId ? turnInfo.nextId : null;
if (turnModel) return turnModel.nextId || null; const isNextDriver = Boolean(socketId && nextDriverId === socketId);
const queue = turnInfo?.queue || []; const deadline = turnInfo?.deadline || null;
if (!queue.length || !turnInfo?.current || queue.length <= 1) return null; const idleDeadline = turnInfo?.idleDeadline || null;
const idx = queue.findIndex((id) => id === turnInfo.current);
if (idx === -1) return queue[0] || null;
return queue[(idx + 1) % queue.length] || null;
}, [turnInfo, turnModel]);
const isNextDriver = turnModel
? Boolean(turnModel.isNext)
: Boolean(socketId && nextDriverId === socketId);
const deadline = turnModel ? turnModel.deadline || null : turnInfo?.deadline || null;
const idleDeadline = turnModel ? turnModel.idleDeadline || null : turnInfo?.idleDeadline || null;
const msUntilTurn = deadline ? deadline - now : null; const msUntilTurn = deadline ? deadline - now : null;
const msUntilIdleSkip = idleDeadline ? idleDeadline - now : null; const msUntilIdleSkip = idleDeadline ? idleDeadline - now : null;
const totalRovers = roster.length; const totalRovers = roster.length;
@@ -69,14 +51,10 @@ function TurnsOverlay({
}); });
return unique.size; return unique.size;
}, [users]); }, [users]);
const shouldUsePreviewByLoad = turnModel const shouldUsePreviewByLoad = isTurnsMode && totalDrivers > totalRovers;
? Boolean(turnModel.showPreviewReason)
: isTurnsMode && totalDrivers > totalRovers;
const isPreSwitchWindow = const isPreSwitchWindow =
isTurnsMode && isNextDriver && msUntilTurn != null && msUntilTurn <= 5000 && msUntilTurn > 0; isTurnsMode && isNextDriver && msUntilTurn != null && msUntilTurn <= 5000 && msUntilTurn > 0;
const showNotTurnNotice = turnModel const showNotTurnNotice = isTurnsMode && !isActiveDriver;
? Boolean(turnModel.showNotTurnNotice ?? (isTurnsMode && !isActiveDriver))
: isTurnsMode && !isActiveDriver;
const showPreviewReason = showNotTurnNotice && !isPreSwitchWindow && shouldUsePreviewByLoad; const showPreviewReason = showNotTurnNotice && !isPreSwitchWindow && shouldUsePreviewByLoad;
const turnSeconds = const turnSeconds =
msUntilTurn != null && Number.isFinite(msUntilTurn) ? Math.max(0, Math.ceil(msUntilTurn / 1000)) : null; msUntilTurn != null && Number.isFinite(msUntilTurn) ? Math.max(0, Math.ceil(msUntilTurn / 1000)) : null;
@@ -105,12 +83,9 @@ function TurnsOverlay({
const turnTimerFlashActive = noticeFlashActive; const turnTimerFlashActive = noticeFlashActive;
useEffect(() => { useEffect(() => {
/* // The legacy overlay still follows the server's turns mode; ownership
Rover turns and PTZ turns now arrive through the same render path. Use the // and deadlines now come from the same turn object as the new HUD.
normalized isTurnsMode flag here instead of checking the server's rover
mode directly, otherwise PTZ can render the notice but never trigger the
"your turn" cue when camera ownership changes.
*/
if (!isTurnsMode) { if (!isTurnsMode) {
lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null }; lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null };
/* /*
@@ -6,33 +6,11 @@ import { useControlActions, useControlSelector } from '../../../../controls/inde
import ControlHint from '../../../ControlHint/index.jsx'; import ControlHint from '../../../ControlHint/index.jsx';
import { useSessionSelector } from '../../../../context/SessionContext.jsx'; import { useSessionSelector } from '../../../../context/SessionContext.jsx';
import useCanControlRover from '../../../../hooks/useCanControlRover.js'; import useCanControlRover from '../../../../hooks/useCanControlRover.js';
import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx'; import RoundControl from './RoundControl.jsx';
import PeripheralPod from './PeripheralPod.jsx';
import HornSettingsExpansion from './HornSettingsExpansion.jsx'; import HornSettingsExpansion from './HornSettingsExpansion.jsx';
import CornerPodToggle from './CornerPodToggle.jsx';
import usePodVisibility from './usePodVisibility.js'; import usePodVisibility from './usePodVisibility.js';
function RoundControl({ label, icon, keyLabel, active, tone, disabled = false, onClick, onPointerDown, onPointerUp, large = false, className = '' }) {
const ControlIcon = icon;
const toneClass = tone === 'horn'
? active ? 'border-fuchsia-300/70 bg-fuchsia-700 text-fuchsia-50' : 'border-cyan-300/70 bg-cyan-900 text-cyan-50'
: active ? 'border-emerald-300/70 bg-emerald-800 text-emerald-50' : 'border-amber-300/70 bg-amber-900 text-amber-50';
return (
<button
type="button"
aria-label={label}
disabled={disabled}
onClick={onClick}
onPointerDown={onPointerDown}
onPointerUp={onPointerUp}
onPointerCancel={onPointerUp}
className={`flex shrink-0 select-none flex-col items-center justify-center gap-1 rounded-full border-2 ${toneClass} ${large ? 'h-20 w-20' : 'h-16 w-16'} disabled:cursor-not-allowed disabled:opacity-40 ${className}`}
>
<ControlIcon className={large ? 'text-xl' : 'text-base'} aria-hidden="true" />
<KeyPill label={keyLabel} />
</button>
);
}
export default function BottomLeftPod({ roverId }) { export default function BottomLeftPod({ roverId }) {
const [open, setOpen] = usePodVisibility('peripherals', true); const [open, setOpen] = usePodVisibility('peripherals', true);
const [hornSettingsOpen, setHornSettingsOpen] = usePodVisibility('hornSettings', false); const [hornSettingsOpen, setHornSettingsOpen] = usePodVisibility('hornSettings', false);
@@ -67,30 +45,25 @@ export default function BottomLeftPod({ roverId }) {
return ( return (
<> <>
{open ? ( <PeripheralPod open={open} onOpenChange={setOpen} label="rover controls">
<div className="pointer-events-auto absolute bottom-0 left-0 z-20 h-40 w-40 rounded-tr-[4.25rem] bg-black/60"> {/*
{/* The horn's center is the origin of an invisible 76px-radius arc. Headlight and laser
The horn's center is the origin of an invisible 76px-radius arc. Headlight and laser sit at -75 and -15 degrees on either side of that arc's diagonal midpoint. Their
sit at -75 and -15 degrees on either side of that arc's diagonal midpoint. Their sixty-degree separation leaves a visible gap between the small circles while keeping
sixty-degree separation leaves a visible gap between the small circles while keeping both controls equally distant from the horn. These explicit positions are the
both controls equally distant from the horn. These explicit positions are the rendered result of that geometry, not unrelated visual nudges.
rendered result of that geometry, not unrelated visual nudges. */}
*/} {/* Physical rover actions become visibly and behaviorally unavailable
{/* Physical rover actions become visibly and behaviorally unavailable while another queued driver owns the turn. Pod/settings controls
while another queued driver owns the turn. Pod/settings controls remain interactive because they do not mutate rover hardware. */}
remain interactive because they do not mutate rover hardware. */} {hornDevice ? <RoundControl label="Horn" icon={FaBullhorn} keyLabel={<ControlHint actionId="hornHonk" />} active={hornActive} tone="horn" disabled={!canControl} large onPointerDown={startHornPointer} onPointerUp={stopHornPointer} className="absolute bottom-1 left-1" /> : null}
{hornDevice ? <RoundControl label="Horn" icon={FaBullhorn} keyLabel={<ControlHint actionId="hornHonk" />} active={hornActive} tone="horn" disabled={!canControl} large onPointerDown={startHornPointer} onPointerUp={stopHornPointer} className="absolute bottom-1 left-1" /> : null} {headlight ? <RoundControl label="Headlight" icon={FaLightbulb} keyLabel={<ControlHint actionId="headlightToggle" />} active={headlightOn} disabled={!canControl} onClick={() => setHeadlight(!headlightOn)} className="absolute left-[1.979rem] top-[0.662rem]" /> : null}
{headlight ? <RoundControl label="Headlight" icon={FaLightbulb} keyLabel={<ControlHint actionId="headlightToggle" />} active={headlightOn} disabled={!canControl} onClick={() => setHeadlight(!headlightOn)} className="absolute left-[1.979rem] top-[0.662rem]" /> : null} {/* The room-light lock deliberately blocks laser activation because
{/* The room-light lock deliberately blocks laser activation because the laser is only intended for use while the room is dark. This
the laser is only intended for use while the room is dark. This mirrors the old desktop control's visible disabled state; turn
mirrors the old desktop control's visible disabled state; turn ownership remains the other independent control restriction. */}
ownership remains the other independent control restriction. */} {laser ? <RoundControl label="Laser" icon={FaCrosshairs} keyLabel={<ControlHint actionId="laserToggle" />} active={laserOn} disabled={!canControl || roomLightsLockedOn} onClick={() => setLaser(!laserOn)} className="absolute left-[5.338rem] top-[4.021rem]" /> : null}
{laser ? <RoundControl label="Laser" icon={FaCrosshairs} keyLabel={<ControlHint actionId="laserToggle" />} active={laserOn} disabled={!canControl || roomLightsLockedOn} onClick={() => setLaser(!laserOn)} className="absolute left-[5.338rem] top-[4.021rem]" /> : null} </PeripheralPod>
<CornerPodToggle corner="bottom-left" expanded label="Hide rover controls" onClick={() => setOpen(false)} />
</div>
) : (
<CornerPodToggle corner="bottom-left" expanded={false} label="Show rover controls" onClick={() => setOpen(true)} />
)}
{hornDevice ? <HornSettingsExpansion open={hornSettingsOpen} onOpenChange={setHornSettingsOpen} /> : null} {hornDevice ? <HornSettingsExpansion open={hornSettingsOpen} onOpenChange={setHornSettingsOpen} /> : null}
</> </>
); );
@@ -5,7 +5,7 @@ import { FaVideo } from 'react-icons/fa';
import { useControlActions, useControlSelector } from '../../../../controls/index.js'; import { useControlActions, useControlSelector } from '../../../../controls/index.js';
import ControlHint from '../../../ControlHint/index.jsx'; import ControlHint from '../../../ControlHint/index.jsx';
import useCanControlRover from '../../../../hooks/useCanControlRover.js'; import useCanControlRover from '../../../../hooks/useCanControlRover.js';
import { useDriverLayout } from '../../../../layouts/driver/DriverLayoutContext.jsx'; import { useLayout } from '../../../../layouts/LayoutContext.jsx';
import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx'; import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx';
import ChatExpansion from './ChatExpansion.jsx'; import ChatExpansion from './ChatExpansion.jsx';
import CornerPodToggle from './CornerPodToggle.jsx'; import CornerPodToggle from './CornerPodToggle.jsx';
@@ -28,7 +28,7 @@ function pointOnArc(fraction) {
} }
export default function BottomRightPod({ roverId }) { export default function BottomRightPod({ roverId }) {
const layout = useDriverLayout(); const layout = useLayout();
const [open, setOpen] = usePodVisibility('camera', true); const [open, setOpen] = usePodVisibility('camera', true);
const camera = useControlSelector((control) => control.state.camera); const camera = useControlSelector((control) => control.state.camera);
const dockAssistActive = useControlSelector((control) => Boolean(control.state.manualDockAssist?.active)); const dockAssistActive = useControlSelector((control) => Boolean(control.state.manualDockAssist?.active));
@@ -0,0 +1,12 @@
import CornerPodToggle from './CornerPodToggle.jsx';
export default function PeripheralPod({ open, onOpenChange, label, children, className = 'h-40 w-40' }) {
return open ? (
<div className={`pointer-events-auto absolute bottom-0 left-0 z-20 rounded-tr-[4.25rem] bg-black/60 ${className}`}>
{children}
<CornerPodToggle corner="bottom-left" expanded label={`Hide ${label}`} onClick={() => onOpenChange(false)} />
</div>
) : (
<CornerPodToggle corner="bottom-left" expanded={false} label={`Show ${label}`} onClick={() => onOpenChange(true)} />
);
}
@@ -0,0 +1,25 @@
import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx';
export default function RoundControl({ label, icon, keyLabel, active, tone, disabled = false, onClick, onPointerDown, onPointerUp, large = false, className = '', value = null }) {
const ControlIcon = icon;
const toneClass = tone === 'horn'
? active ? 'border-fuchsia-300/70 bg-fuchsia-700 text-fuchsia-50' : 'border-cyan-300/70 bg-cyan-900 text-cyan-50'
: active ? 'border-emerald-300/70 bg-emerald-800 text-emerald-50' : 'border-amber-300/70 bg-amber-900 text-amber-50';
return (
<button
type="button"
aria-label={label}
disabled={disabled}
onClick={onClick}
onPointerDown={onPointerDown}
onPointerUp={onPointerUp}
onPointerCancel={onPointerUp}
className={`flex shrink-0 select-none flex-col items-center justify-center gap-1 rounded-full border-2 ${toneClass} ${large ? 'h-20 w-20' : 'h-16 w-16'} disabled:cursor-not-allowed disabled:opacity-40 ${className}`}
>
<ControlIcon className={large ? 'text-xl' : 'text-base'} aria-hidden="true" />
{value != null ? <span className="text-xs font-semibold leading-none">{value}</span> : null}
<KeyPill label={keyLabel} />
</button>
);
}
@@ -1,40 +1,28 @@
// Top-left Corner Pod // Top-left Corner Pod
// Purpose: Shows the user's current-turn or queue-wait countdown and the rover identity expansion. // Purpose: Renders the caller's turn-display object without knowing its operating mode.
import { useEffect, useMemo, useRef, useState } from 'react'; import { useEffect, useMemo, useRef, useState } from 'react';
import { useControlSelector } from '../../../../controls/index.js'; import { useControlSelector } from '../../../../controls/index.js';
import { useSessionSelector } from '../../../../context/SessionContext.jsx';
import { useSharedClock } from '../../../../hooks/useSharedClock.js'; import { useSharedClock } from '../../../../hooks/useSharedClock.js';
import RoverLabel from '../../../RoverLabel/index.jsx'; import RoverLabel from '../../../RoverLabel/index.jsx';
import CornerPodToggle from './CornerPodToggle.jsx'; import CornerPodToggle from './CornerPodToggle.jsx';
import ExpansionPanel from './ExpansionPanel.jsx'; import ExpansionPanel from './ExpansionPanel.jsx';
import usePodVisibility from './usePodVisibility.js'; import usePodVisibility from './usePodVisibility.js';
export default function TopLeftPod({ roverId }) { export default function TopLeftPod({ turns, compact = false }) {
const [timerOpen, setTimerOpen] = usePodVisibility('turnTimer', true); const [timerOpen, setTimerOpen] = usePodVisibility('turnTimer', true);
const [nameOpen, setNameOpen] = usePodVisibility('roverName', true); const [nameOpen, setNameOpen] = usePodVisibility('roverName', true);
const mode = useSessionSelector((state) => state.session?.mode || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null);
const turnInfo = useSessionSelector((state) => state.session?.turnQueues?.[roverId] || null);
const activeDriverId = useSessionSelector((state) => state.session?.activeDrivers?.[roverId] || null);
const lastControlIntentAt = useControlSelector((control) => control.state.lastControlIntentAt); const lastControlIntentAt = useControlSelector((control) => control.state.lastControlIntentAt);
const deadline = turnInfo?.deadline || null; const { target, labels, enabled: turnActive, queueLength, turnsAhead, ownsControl: isCurrentTurn } = turns;
const idleDeadline = turnInfo?.idleDeadline || null; const deadline = turns.deadline;
const queue = turnInfo?.queue || []; const idleDeadline = turns.idleDeadline;
// Direct ownership is published independently from the detailed queue and is const durationSeconds = turns.durationMs / 1000;
// therefore the reliable initial-load/reconnect source for the current turn.
const currentDriverId = activeDriverId || turnInfo?.current || null;
const currentIndex = currentDriverId ? queue.indexOf(currentDriverId) : -1;
const userIndex = socketId ? queue.indexOf(socketId) : -1;
const turnActive = mode === 'turns' && queue.length > 1 && currentIndex >= 0 && userIndex >= 0;
const hasTurnDeadline = Boolean(turnActive && deadline); const hasTurnDeadline = Boolean(turnActive && deadline);
const now = useSharedClock(1000, hasTurnDeadline); const now = useSharedClock(1000, hasTurnDeadline);
const currentTurnSeconds = hasTurnDeadline ? Math.max(0, Math.ceil((deadline - now) / 1000)) : null; const currentTurnSeconds = hasTurnDeadline ? Math.max(0, Math.ceil((deadline - now) / 1000)) : null;
const idleSkipSeconds = idleDeadline ? Math.max(0, Math.ceil((idleDeadline - now) / 1000)) : null; const idleSkipSeconds = idleDeadline ? Math.max(0, Math.ceil((idleDeadline - now) / 1000)) : null;
const turnsAhead = turnActive ? (userIndex - currentIndex + queue.length) % queue.length : null;
const seconds = currentTurnSeconds == null || turnsAhead == null const seconds = currentTurnSeconds == null || turnsAhead == null
? null ? null
: currentTurnSeconds + Math.max(0, turnsAhead - 1) * 60; : currentTurnSeconds + Math.max(0, turnsAhead - 1) * durationSeconds;
const isCurrentTurn = turnsAhead === 0;
const isWaitingForTurn = turnActive && !isCurrentTurn; const isWaitingForTurn = turnActive && !isCurrentTurn;
const [showTurnCue, setShowTurnCue] = useState(false); const [showTurnCue, setShowTurnCue] = useState(false);
const [turnCueStartedAt, setTurnCueStartedAt] = useState(null); const [turnCueStartedAt, setTurnCueStartedAt] = useState(null);
@@ -81,25 +69,20 @@ export default function TopLeftPod({ roverId }) {
}, [lastControlIntentAt, showTurnCue, turnCueStartedAt]); }, [lastControlIntentAt, showTurnCue, turnCueStartedAt]);
const gaugePercent = useMemo(() => { const gaugePercent = useMemo(() => {
if (seconds == null) return 0; if (seconds == null) return 0;
/* // Scale waiting time by this target's actual turn duration. PTZ and rover
The server's rover turns are sixty seconds long. Waiting users need one complete // turns have different lengths, but use the same presentation.
turn added for each driver between the current driver and themselves. A complete const rotationSeconds = Math.max(durationSeconds, queueLength * durationSeconds);
queue rotation is a stable scale across handoffs, so the ring drains continuously
instead of jumping back to full when the current driver changes.
*/
const rotationSeconds = Math.max(60, queue.length * 60);
return Math.max(0, Math.min(1, seconds / rotationSeconds)); return Math.max(0, Math.min(1, seconds / rotationSeconds));
}, [queue.length, seconds]); }, [durationSeconds, queueLength, seconds]);
const visibleGaugePercent = showTurnCue && idleSkipSeconds != null const visibleGaugePercent = showTurnCue && idleSkipSeconds != null
// The server's initial inactivity grace is seven seconds. Mirroring that known lifecycle // The caller supplies its inactivity grace alongside the idle deadline.
// makes the enlarged ring itself reinforce the prominent skip countdown in the center. ? Math.max(0, Math.min(1, idleSkipSeconds / (turns.idleGraceMs / 1000)))
? Math.max(0, Math.min(1, idleSkipSeconds / 7))
: gaugePercent; : gaugePercent;
const showTimer = Boolean(turnActive && timerOpen); const showTimer = Boolean(turnActive && timerOpen);
const showLargeTimer = isWaitingForTurn || showTurnCue; const showLargeTimer = isWaitingForTurn || showTurnCue;
const timerLabel = seconds == null const timerLabel = seconds == null
// Ownership should remain visible while the detailed deadline is in flight. // Ownership should remain visible while the detailed deadline is in flight.
? isCurrentTurn ? 'Your turn' : 'Waiting' ? isCurrentTurn ? labels.activeTimer : labels.waitingTimer
: seconds >= 60 : seconds >= 60
? `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, '0')}` ? `${Math.floor(seconds / 60)}:${String(seconds % 60).padStart(2, '0')}`
: `${seconds}s`; : `${seconds}s`;
@@ -113,8 +96,8 @@ export default function TopLeftPod({ roverId }) {
// The waiting/handoff state replaces the old full-screen turn cue. // The waiting/handoff state replaces the old full-screen turn cue.
// It must be opaque and above every other in-video HUD surface so // It must be opaque and above every other in-video HUD surface so
// sensor graphics, chat, and docking controls cannot muddy the text. // sensor graphics, chat, and docking controls cannot muddy the text.
? 'h-102 w-102 rounded-br-[12.75rem] bg-black' ? compact ? 'h-[min(12rem,45vw)] w-[min(12rem,45vw)] rounded-br-[6rem] bg-black' : 'h-102 w-102 rounded-br-[12.75rem] bg-black'
: 'h-34 w-34 rounded-br-[4.25rem] bg-black/60' : compact ? 'h-24 w-24 rounded-br-[3rem] bg-black/60' : 'h-34 w-34 rounded-br-[4.25rem] bg-black/60'
}`} }`}
> >
{/* The SVG fills the shell. Its circle geometry supplies the same slim visible inset {/* The SVG fills the shell. Its circle geometry supplies the same slim visible inset
@@ -129,31 +112,31 @@ export default function TopLeftPod({ roverId }) {
fixed-size so it never becomes a giant obstruction over the video. */} fixed-size so it never becomes a giant obstruction over the video. */}
{showLargeTimer ? ( {showLargeTimer ? (
<> <>
<span className={`mb-2 text-[1.75rem] font-bold leading-tight transition-colors duration-300 ${showTurnCue ? 'text-amber-200' : 'text-sky-100'}`}> <span className={`${compact ? 'mb-1 text-sm' : 'mb-2 text-[1.75rem]'} font-bold leading-tight transition-colors duration-300 ${showTurnCue ? 'text-amber-200' : 'text-sky-100'}`}>
{showTurnCue ? 'It’s your turn!' : 'Someone else is driving'} {showTurnCue ? labels.handoff : labels.waiting}
</span> </span>
<strong className="text-[3.375rem] leading-none"> <strong className={`${compact ? "text-2xl" : "text-[3.375rem]"} leading-none`}>
{showTurnCue && idleSkipSeconds != null ? `${idleSkipSeconds}s` : timerLabel} {showTurnCue && idleSkipSeconds != null ? `${idleSkipSeconds}s` : timerLabel}
</strong> </strong>
<span className={`mt-2 text-[1rem] font-semibold leading-tight ${showTurnCue ? 'text-amber-200' : 'text-sky-200'}`}> <span className={`${compact ? 'mt-1 text-xs' : 'mt-2 text-[1rem]'} font-semibold leading-tight ${showTurnCue ? 'text-amber-200' : 'text-sky-200'}`}>
{showTurnCue {showTurnCue
? idleSkipSeconds != null ? idleSkipSeconds != null
? 'Start driving or your turn will be skipped' ? labels.idleWarning
: 'You’re driving' : labels.active
: 'until your turn'} : labels.untilTurn}
</span> </span>
</> </>
) : ( ) : (
<> <>
<strong className="text-lg leading-none">{timerLabel}</strong> <strong className="text-lg leading-none">{timerLabel}</strong>
{seconds != null ? <span className="mt-1 text-[0.6rem] font-semibold text-sky-200">left</span> : null} {seconds != null ? <span className="mt-1 text-[0.6rem] font-semibold text-sky-200">{labels.timeRemaining}</span> : null}
</> </>
)} )}
</span> </span>
<CornerPodToggle corner="top-left" expanded label="Hide turn timer" onClick={() => setTimerOpen(false)} /> <CornerPodToggle corner="top-left" expanded label={labels.hideTimer} onClick={() => setTimerOpen(false)} />
</div> </div>
) : turnActive ? ( ) : turnActive ? (
<CornerPodToggle corner="top-left" expanded={false} label="Show turn timer" onClick={() => setTimerOpen(true)} /> <CornerPodToggle corner="top-left" expanded={false} label={labels.showTimer} onClick={() => setTimerOpen(true)} />
) : null} ) : null}
{/* The rover name is an independent edge expansion. Its visibility control lives in {/* The rover name is an independent edge expansion. Its visibility control lives in
@@ -166,10 +149,10 @@ export default function TopLeftPod({ roverId }) {
panelClassName={`flex h-11 min-w-max items-center gap-2 bg-black/60 px-2 pt-3 ${showTimer ? '' : 'rounded-br-xl'}`} panelClassName={`flex h-11 min-w-max items-center gap-2 bg-black/60 px-2 pt-3 ${showTimer ? '' : 'rounded-br-xl'}`}
openDirection="down" openDirection="down"
closeDirection="up" closeDirection="up"
openLabel="Show rover name" openLabel={labels.showName}
closeLabel="Hide rover name" closeLabel={labels.hideName}
> >
<RoverLabel roverId={roverId} fallback={roverId} className="px-2 py-1 text-base" /> <RoverLabel name={target.name} color={target.color} fallback={target.fallback} className="px-2 py-1 text-base" />
</ExpansionPanel> </ExpansionPanel>
</div> </div>
); );
@@ -1,19 +1,22 @@
// New Drive Corner Pods // New Drive Corner Pods
// Purpose: Composes the four independently owned corner controls around the shared video stage. // Purpose: Composes the four independently owned corner controls around the shared video stage.
import TopLeftPod from './TopLeftPod.jsx'; import TopLeftPod from './TopLeftPod.jsx';
import { useSessionSelector } from '../../../../context/SessionContext.jsx';
import TopRightPod from './TopRightPod.jsx'; import TopRightPod from './TopRightPod.jsx';
import BottomLeftPod from './BottomLeftPod.jsx'; import BottomLeftPod from './BottomLeftPod.jsx';
import BottomRightPod from './BottomRightPod.jsx'; import BottomRightPod from './BottomRightPod.jsx';
import AccessoriesExpansion from './AccessoriesExpansion.jsx'; import AccessoriesExpansion from './AccessoriesExpansion.jsx';
import { useDriverLayout } from '../../../../layouts/driver/DriverLayoutContext.jsx'; import { useLayout } from '../../../../layouts/LayoutContext.jsx';
export default function CornerPods({ roverId }) { export default function CornerPods({ roverId }) {
const layout = useDriverLayout(); const layout = useLayout();
const turns = useSessionSelector((state) => state.session?.roster
?.find((rover) => String(rover.id) === String(roverId))?.turn);
const showPhysicalControlPods = layout === 'desktop'; const showPhysicalControlPods = layout === 'desktop';
return ( return (
<> <>
<TopLeftPod roverId={roverId} /> {turns ? <TopLeftPod turns={turns} /> : null}
<TopRightPod roverId={roverId} /> <TopRightPod roverId={roverId} />
{/* The mobile layouts already provide large touch controls around the video. {/* The mobile layouts already provide large touch controls around the video.
Omitting this pod avoids presenting duplicate horn, light, and laser actions. */} Omitting this pod avoids presenting duplicate horn, light, and laser actions. */}
@@ -11,7 +11,7 @@ import { dockTelemetryEqual, selectDockTelemetry } from '../../../../context/tel
import { useManualDockAssist } from '../../../../features/manualDockAssist/useManualDockAssist.js'; import { useManualDockAssist } from '../../../../features/manualDockAssist/useManualDockAssist.js';
import { useSettingsNamespace } from '../../../../settings/index.js'; import { useSettingsNamespace } from '../../../../settings/index.js';
import useCanControlRover from '../../../../hooks/useCanControlRover.js'; import useCanControlRover from '../../../../hooks/useCanControlRover.js';
import { useDriverLayout } from '../../../../layouts/driver/DriverLayoutContext.jsx'; import { useLayout } from '../../../../layouts/LayoutContext.jsx';
import { useSessionSelector } from '../../../../context/SessionContext.jsx'; import { useSessionSelector } from '../../../../context/SessionContext.jsx';
import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx'; import KeyPill from '../../../vip/VipAudioUploadCard/KeyPill.jsx';
import ExpansionPanel from '../CornerPods/ExpansionPanel.jsx'; import ExpansionPanel from '../CornerPods/ExpansionPanel.jsx';
@@ -250,7 +250,7 @@ function UndockTransitionGhost({ onFinish }) {
} }
export default function DockingHud({ roverId }) { export default function DockingHud({ roverId }) {
const layout = useDriverLayout(); const layout = useLayout();
const actions = useControlActions(); const actions = useControlActions();
const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual); const dockTelemetry = useTelemetrySelector(roverId, selectDockTelemetry, dockTelemetryEqual);
// This replaces ManualDockAssistOverlay as the current HUD's one lifecycle owner. It preserves the // This replaces ManualDockAssistOverlay as the current HUD's one lifecycle owner. It preserves the
@@ -170,8 +170,6 @@ export function ExternalInstancesCompact({ onBrowse = null }) {
key={remote.url} key={remote.url}
title={remote.instance?.name || remote.url} title={remote.instance?.name || remote.url}
roster={remote.roster} roster={remote.roster}
turnQueues={remote.turnQueues}
users={remote.users}
externalInstance={remote} externalInstance={remote}
disabledOverlay={getRemoteAvailability(remote).blocked ? getRemoteAvailability(remote).overlay : ''} disabledOverlay={getRemoteAvailability(remote).blocked ? getRemoteAvailability(remote).overlay : ''}
/> />
@@ -221,8 +219,6 @@ function InterInstanceCards({ instances, centered = false, singleColumn = false
<RemoteMediaStrip remote={remote} /> <RemoteMediaStrip remote={remote} />
<RoverQueuesPanel <RoverQueuesPanel
roster={remote.roster} roster={remote.roster}
turnQueues={remote.turnQueues}
users={remote.users}
externalInstance={remote} externalInstance={remote}
disabledOverlay={getRemoteAvailability(remote).blocked ? getRemoteAvailability(remote).overlay : ''} disabledOverlay={getRemoteAvailability(remote).blocked ? getRemoteAvailability(remote).overlay : ''}
/> />
+2 -2
View File
@@ -11,7 +11,7 @@ import BottomSensorHud from '../HudOverlays/newgen/BottomSensorHud/index.jsx';
import { useSessionSelector } from '../../context/SessionContext.jsx'; import { useSessionSelector } from '../../context/SessionContext.jsx';
import { useControlSelector } from '../../controls/index.js'; import { useControlSelector } from '../../controls/index.js';
import { useDriverVideoModePolicy } from '../../hooks/useDriverVideoModePolicy.js'; import { useDriverVideoModePolicy } from '../../hooks/useDriverVideoModePolicy.js';
import { useDriverLayout } from '../../layouts/driver/DriverLayoutContext.jsx'; import { useLayout } from '../../layouts/LayoutContext.jsx';
import EmptyDriverVideoNotice from '../DriverVideo/EmptyDriverVideoNotice.jsx'; import EmptyDriverVideoNotice from '../DriverVideo/EmptyDriverVideoNotice.jsx';
// Mobile keeps the same HUD composition and coordinate system as desktop, but its // Mobile keeps the same HUD composition and coordinate system as desktop, but its
@@ -21,7 +21,7 @@ import EmptyDriverVideoNotice from '../DriverVideo/EmptyDriverVideoNotice.jsx';
const MOBILE_HUD_SCALE = 0.68; const MOBILE_HUD_SCALE = 0.68;
export default function NewDriveVideo() { export default function NewDriveVideo() {
const layout = useDriverLayout(); const layout = useLayout();
const mobileHud = layout !== 'desktop'; const mobileHud = layout !== 'desktop';
const roverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null); const roverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
const videoMode = useDriverVideoModePolicy(roverId); const videoMode = useDriverVideoModePolicy(roverId);
@@ -0,0 +1,22 @@
import { useEffect, useRef } from 'react';
import { useLocation } from 'react-router-dom';
import { useSessionActions } from '../../context/SessionContext.jsx';
export default function OperatingModeRouteSync() {
const { pathname } = useLocation();
const previousPath = useRef(pathname);
const { setOperatingMode, pushAlert } = useSessionActions();
useEffect(() => {
const leavingPtz = previousPath.current === '/ptz' && pathname !== '/ptz';
previousPath.current = pathname;
if (!leavingPtz) return;
// Observe actual navigation rather than component cleanup, which also runs
// during Strict Mode remounts. Entry remains gated by the PTZ request action.
setOperatingMode('rover').catch((err) => {
pushAlert({ title: 'Operating mode', message: err.message, color: '#f59e0b' });
});
}, [pathname, pushAlert, setOperatingMode]);
return null;
}
+18 -878
View File
@@ -1,32 +1,13 @@
// PTZ Camera UI // Driver-page entry card; PTZ route lifecycle belongs to the PTZ page.
// Purpose: Integrates the single PTZ camera into the main rover UI flow as a import { useState } from 'react';
// queueable controllable target instead of a VIP-panel card.
// Scope: Owns the driver-page PTZ entry card and the dedicated PTZ route
// composition; PTZ command authority, queue ownership, and stream authorization
// remain server-owned.
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useNavigate } from 'react-router-dom'; import { useNavigate } from 'react-router-dom';
import CardFrame from '../CardFrame/index.jsx'; import CardFrame from '../CardFrame/index.jsx';
import ChatPanel from '../ChatPanel/index.jsx'; import QueueTargetRow from '../QueueTargetRow/index.jsx';
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx'; import { PTZ_CAMERA_ID } from '../PtzLiveVideo/index.jsx';
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
import HomeAssistantControls from '../HomeAssistantControls/index.jsx';
import PtzLiveVideo, { PTZ_CAMERA_ID } from '../PtzLiveVideo/index.jsx';
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
import QueueTargetRow, { QueueUserChips } from '../QueueTargetRow/index.jsx';
import TurnsOverlay from '../HudOverlays/TurnsOverlay/index.jsx';
import KeyPill from '../vip/VipAudioUploadCard/KeyPill.jsx';
import { useControlActions } from '../../controls/index.js';
import ControlHint from '../ControlHint/index.jsx';
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx'; import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
import { useSharedClock } from '../../hooks/useSharedClock.js'; import { useSharedClock } from '../../hooks/useSharedClock.js';
import { isFeatureEnabled } from '../../lib/features.js'; import { isFeatureEnabled } from '../../lib/features.js';
import { useSettingsNamespace } from '../../settings/index.js'; const PTZ_DEFAULT_COLOR = '#387bf8';
import { DEFAULT_PAGE_THEME_KEY, usePageThemeClass } from '../../themes/index.js';
import { triggerTouchHaptic } from '../../lib/touchHaptics.js';
const PTZ_DEFAULT_COLOR = '#38bdf8';
function formatRemaining(deadline, now) { function formatRemaining(deadline, now) {
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - now) / 1000)); const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - now) / 1000));
@@ -36,840 +17,17 @@ function formatRemaining(deadline, now) {
return `${minutes}:${String(seconds).padStart(2, '0')}`; return `${minutes}:${String(seconds).padStart(2, '0')}`;
} }
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeIrMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function nextIrMode(currentMode) {
const current = normalizeIrMode(currentMode);
if (current === 'Auto') return 'On';
if (current === 'On') return 'Off';
return 'Auto';
}
function normalizePtzQueue(ptz = null) {
/*
The PTZ service exposes the current operator separately from the waiting
queue, while the rover queue row expects one ordered queue plus a current id.
Normalizing once keeps every PTZ surface consistent with the shared queue
renderer without making that renderer understand PTZ service internals.
*/
const currentId = ptz?.operatorSocketId || null;
const waiting = Array.isArray(ptz?.queue)
? ptz.queue.map((entry) => entry?.socketId || entry).filter(Boolean)
: [];
const queue = currentId ? [currentId, ...waiting.filter((id) => id !== currentId)] : waiting;
const nextId = currentId ? waiting[0] || null : queue[0] || null;
return { queue, currentId, nextId };
}
function usePtzQueueLookup(ptz = null) {
const users = useSessionSelector((state) => state.session?.users ?? []);
return useCallback(
(socketId) => {
const fromUsers = users.find((entry) => entry.socketId === socketId);
if (fromUsers) return fromUsers;
if (ptz?.operatorSocketId === socketId) {
return { socketId, nickname: ptz?.operatorLabel || null, role: null };
}
const fromQueue = Array.isArray(ptz?.queue)
? ptz.queue.find((entry) => (entry?.socketId || entry) === socketId)
: null;
return {
socketId,
nickname: fromQueue?.label || null,
role: null,
};
},
[ptz, users],
);
}
function PtzSnapshotPreview({ feed, label = 'PTZ Camera', className = 'h-full w-full' }) {
return (
<div className={`relative overflow-hidden bg-black ${className}`}>
{feed?.objectUrl ? (
<img src={feed.objectUrl} alt={label} className="h-full w-full object-contain" />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
)}
{/*
Snapshot mode should look like the regular rover video player: the
camera name belongs to the surrounding card/menu, while the media pane
only exposes stream health in the small top-left diagnostic overlay.
*/}
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
</div>
</div>
</div>
);
}
function PtzQueueSummary({ ptz, title = 'PTZ queue' }) {
const selfId = useSessionSelector((state) => state.session?.socketId || null);
const lookupUser = usePtzQueueLookup(ptz);
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
return (
<CardFrame title={title} bodyClassName="space-y-0.5 p-1 text-sm">
<QueueUserChips
targetId={ptz?.id || PTZ_CAMERA_ID}
queue={queue}
currentId={currentId}
nextId={nextId}
selfId={selfId}
lookupUser={lookupUser}
/>
</CardFrame>
);
}
function PtzLightingControls({ ptz, disabled = false }) {
const { ptzSpotlight, ptzIr } = useSessionActions();
const [busy, setBusy] = useState('');
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const toggleSpotlight = async (nextOn) => {
if (disabled) return;
setBusy('spotlight');
try {
await ptzSpotlight({ state: nextOn ? 1 : 0 });
} finally {
setBusy('');
}
};
const cycleIr = async () => {
if (disabled) return;
triggerTouchHaptic('button');
setBusy('ir');
try {
await ptzIr({ state: nextIrMode(irMode) });
} finally {
setBusy('');
}
};
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<GPIOToggleControl
label="Spotlight"
on={spotlightOn}
disabled={disabled || busy === 'spotlight'}
onToggle={toggleSpotlight}
heightClass="min-h-14"
/>
<button
type="button"
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
disabled={disabled || busy === 'ir'}
onClick={cycleIr}
>
<span className="text-sm font-semibold">Infrared</span>
<span className="rounded-sm bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
</button>
</div>
);
}
function PtzMobileZoomButtons({ disabled = false }) {
const { setCameraAxisIntent } = useControlActions();
const stopZoom = useCallback(() => {
/*
Zero only releases the zoom axis. The PTZ adapter combines it with any
pan/tilt direction still held on the movement pad, so lifting one finger
cannot erase the other finger's intent.
*/
setCameraAxisIntent(0);
}, [setCameraAxisIntent]);
const startZoom = useCallback(
(direction) => (event) => {
/*
Publish held state once. The adapter owns the single motion heartbeat,
so this button no longer creates a second interval whose queued callback
could run after pointerup and restart zoom.
*/
event.preventDefault();
if (disabled) return;
triggerTouchHaptic('button');
event.currentTarget.setPointerCapture?.(event.pointerId);
setCameraAxisIntent(direction);
},
[disabled, setCameraAxisIntent],
);
const stopFromPointer = useCallback(
(event) => {
event?.preventDefault?.();
if (disabled) return;
stopZoom();
},
[disabled, stopZoom],
);
useEffect(
() => () => {
/*
A touch surface can unmount during orientation changes or fullscreen
close while a pointer is still down. Explicitly clear zoom here because
an unmounted DOM node cannot deliver its pointerup/pointercancel event.
*/
setCameraAxisIntent(0);
},
[setCameraAxisIntent],
);
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(-1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onLostPointerCapture={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom out
</button>
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onLostPointerCapture={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom in
</button>
</div>
);
}
function PtzMobileControlsPanel({ ptz, disabled = false }) {
return (
<div className="mobile-touch-control space-y-0.5">
<PtzMobileZoomButtons disabled={disabled} />
<div className="mobile-touch-control h-44 min-h-0">
{/*
Reuse the rover control pad so touch intent still enters the normal
control system. The PTZ adapter translates that same drive vector into
pan/tilt commands only while this user is the PTZ operator.
*/}
<ControlPadPanel compact disabled={disabled} />
</div>
<PtzLightingControls ptz={ptz} disabled={disabled} />
</div>
);
}
function PtzControlReference() {
const rows = [
['Tilt up', 'driveForward'],
['Tilt down', 'driveBackward'],
['Pan left', 'driveLeft'],
['Pan right', 'driveRight'],
['Zoom in', 'cameraUp'],
['Zoom out', 'cameraDown'],
['Spotlight', 'headlightToggle'],
['Infrared mode', 'laserToggle'],
];
return (
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
{rows.map(([label, actionId]) => (
<div key={label} className="surface flex items-center justify-between gap-1">
<span className="text-slate-400">{label}</span>
<KeyPill label={<ControlHint actionId={actionId} />} />
</div>
))}
</CardFrame>
);
}
function PtzPresetPanel({ ptz }) {
const role = useSessionSelector((state) => state.session?.role || null);
const {
ptzListPresets,
ptzGotoPreset,
ptzCreatePreset,
ptzRemovePreset,
} = useSessionActions();
const [name, setName] = useState('');
const [busy, setBusy] = useState('');
const presets = Array.isArray(ptz?.presets) ? ptz.presets : [];
const isPresetAdmin = role === 'admin' || role === 'lockdown';
const canCreatePreset = Boolean(ptz?.canUse);
const canMoveToPreset = Boolean(ptz?.isOperator);
const refreshPresets = async () => {
if (busy) return;
setBusy('refresh');
try {
/*
Presets live on the camera, not in browser state. A manual refresh gives
admins a simple recovery path if another admin or the camera's native
app changes preset storage while this UI is already open.
*/
await ptzListPresets();
} catch (err) {
alert(err.message || 'Failed to refresh PTZ presets.');
} finally {
setBusy('');
}
};
const goToPreset = async (preset) => {
if (!canMoveToPreset || busy || !preset?.token) return;
setBusy(`goto:${preset.token}`);
try {
/*
Moving to a preset is a physical camera move, so the server still checks
that this browser owns the active PTZ turn before accepting the command.
*/
await ptzGotoPreset({ token: preset.token });
} catch (err) {
alert(err.message || 'Failed to move to PTZ preset.');
} finally {
setBusy('');
}
};
const createPreset = async (event) => {
event.preventDefault();
const trimmed = name.trim();
if (!canCreatePreset || busy || !trimmed) return;
setBusy('create');
try {
/*
ONVIF setPreset stores the camera's current physical position. The UI
only sends the user's label; the server supplies the active profile
token so browser code does not need camera profile internals, and the
server still enforces the PTZ feature gate for raw socket callers.
*/
await ptzCreatePreset({ name: trimmed });
setName('');
} catch (err) {
alert(err.message || 'Failed to create PTZ preset.');
} finally {
setBusy('');
}
};
const removePreset = async (preset) => {
if (!isPresetAdmin || busy || !preset?.token) return;
const confirmed = window.confirm(`Remove preset "${preset.name}"?`);
if (!confirmed) return;
setBusy(`remove:${preset.token}`);
try {
/*
The token is the camera's durable preset identifier. Names are only UI
labels and may not be unique, so deletion always targets the token.
*/
await ptzRemovePreset({ token: preset.token });
} catch (err) {
alert(err.message || 'Failed to remove PTZ preset.');
} finally {
setBusy('');
}
};
return (
<CardFrame
title="Position presets"
fillHeight
actions={(
<button type="button" className="button-dark text-xs" disabled={Boolean(busy)} onClick={refreshPresets}>
Refresh
</button>
)}
bodyClassName="flex min-h-0 flex-col gap-1 p-1 text-xs"
>
{ptz?.presetsError ? (
<div className="rounded-sm border border-amber-500/50 bg-amber-950/40 p-1 text-amber-100">
{ptz.presetsError}
</div>
) : null}
<div className="flex min-h-0 flex-1 flex-wrap content-start items-start gap-0.5 overflow-y-auto">
{/*
Presets should behave like a compact pile of actions, not a table.
flex-wrap lets each preset keep its natural button width and only
starts a new visual line when the current line runs out of room.
*/}
{presets.length ? presets.map((preset) => {
const gotoBusy = busy === `goto:${preset.token}`;
const removeBusy = busy === `remove:${preset.token}`;
return (
<div key={preset.token} className="surface inline-flex max-w-full items-center gap-1">
<button
type="button"
className="button-dark min-w-0 max-w-40 truncate text-left text-xs disabled:opacity-50"
disabled={!canMoveToPreset || Boolean(busy)}
onClick={() => goToPreset(preset)}
title={canMoveToPreset ? `Move to ${preset.name}` : 'Your PTZ turn must be active'}
>
{gotoBusy ? 'Moving...' : preset.name}
</button>
{isPresetAdmin ? (
<button
type="button"
className="button-dark text-xs text-rose-200 disabled:opacity-50"
disabled={Boolean(busy)}
onClick={() => removePreset(preset)}
>
{removeBusy ? 'Removing...' : 'Remove'}
</button>
) : null}
</div>
);
}) : (
<div className="rounded-sm border border-slate-700 bg-black/30 p-2 text-center text-slate-400">
No presets saved.
</div>
)}
</div>
{canCreatePreset ? (
<form className="grid grid-cols-[minmax(0,1fr)_auto] gap-1" onSubmit={createPreset}>
<input
className="field-input min-w-0 text-xs"
value={name}
maxLength={60}
disabled={Boolean(busy)}
onChange={(event) => setName(event.target.value)}
placeholder="Preset name"
/>
<button type="submit" className="button-dark text-xs" disabled={Boolean(busy) || !name.trim()}>
{busy === 'create' ? 'Saving...' : 'Save'}
</button>
</form>
) : null}
</CardFrame>
);
}
function buildPtzTurnModel(ptz, selfId) {
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
const isActive = Boolean(ptz?.isOperator);
const isQueued = Boolean(ptz?.queuedPosition);
return {
enabled: Boolean(ptz && (isActive || isQueued || currentId)),
targetId: ptz?.id || PTZ_CAMERA_ID,
activeId: currentId,
nextId,
isActive,
isNext: Boolean(selfId && nextId === selfId),
deadline: ptz?.deadline || null,
idleDeadline: null,
showNotTurnNotice: Boolean(!isActive && (isQueued || currentId || queue.length)),
showPreviewReason: false,
};
}
function PtzMediaPane({ ptz, open, framed = true }) {
const isOperator = Boolean(ptz?.isOperator);
const isParticipant = Boolean(isOperator || ptz?.queuedPosition);
const nonTurnSnapshotsActive = useSessionSelector(
(state) => Boolean(state.session?.bandwidthSavings?.nonTurnVideo?.snapshotsActive),
);
const selfId = useSessionSelector((state) => state.session?.socketId || null);
/*
PTZ has its own turn queue, so non-operators are the camera equivalent of a
non-active rover driver. The server enforces the same policy in
canRequestLiveVideo(); this branch only chooses the expected browser render
path and never unlocks movement controls.
*/
/*
A direct /ptz load renders before its automatic queue claim is acknowledged.
Do not mount the live player during that short pre-claim window: its first
token request would correctly be rejected, and PtzLiveVideo intentionally
treats authorization rejection as a terminal snapshot fallback. Once the
session confirms queue/operator membership, mounting the player creates a
fresh authorized request without changing shared retry or server policy.
*/
const shouldUseLiveVideo = isParticipant && (isOperator || !nonTurnSnapshotsActive);
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !shouldUseLiveVideo });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const turnModel = useMemo(() => buildPtzTurnModel(ptz, selfId), [ptz, selfId]);
const media = (
<>
{shouldUseLiveVideo ? (
<PtzLiveVideo enabled={open} startMuted={false} />
) : (
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
)}
<TurnsOverlay turnModel={turnModel} />
</>
);
if (!framed) {
return <div className="relative h-full min-h-0 w-full overflow-hidden bg-black">{media}</div>;
}
return (
<div className="flex h-full min-h-0 w-full items-center justify-center overflow-hidden bg-black">
<div className="relative aspect-video max-h-full w-full max-w-full overflow-hidden bg-black">
{media}
</div>
</div>
);
}
function PtzDesktopFullscreen({ ptz, releasePending }) {
return (
<div className="grid h-full min-h-0 grid-rows-[minmax(0,1fr)_minmax(7rem,0.22fr)] gap-0.5 overflow-hidden p-0.5">
<div className="flex min-h-0 min-w-0 gap-0.5 overflow-hidden">
<main className="min-h-0 shrink-0 overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
<PtzMediaPane ptz={ptz} open framed />
</main>
{/* Keep the sidebar itself transparent. Its child cards still own their dark surfaces,
while the shared PTZ page theme can show through the same compact gaps as the driver
layout instead of being covered by one solid sidebar rectangle. */}
<aside className="flex min-h-0 min-w-56 flex-1 flex-col gap-0.5 overflow-y-auto text-sm">
<PtzQueueSummary ptz={ptz} />
{ptz?.isOperator ? (
<PtzLightingControls ptz={ptz} />
) : (
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
Live PTZ controls unlock when your camera turn is active.
</CardFrame>
)}
<PtzControlReference />
<ReplaySourcesPanel panelId="ptz-controller-replay" defaultSelectedKey={`ptz:${PTZ_CAMERA_ID}`} />
{/*
Desktop keeps room lights as the final sidebar tool so camera
turn controls and replay remain above the less-frequent room-wide
actions. HomeAssistantControls owns its own feature and policy gate.
*/}
<HomeAssistantControls />
</aside>
</div>
<div className="grid min-h-0 grid-cols-[minmax(0,1.6fr)_minmax(16rem,0.7fr)] gap-0.5 overflow-hidden">
<ChatPanel fillHeight title="Chat" allowSpectatorInput inputTarget="overlay" />
<PtzPresetPanel ptz={ptz} />
</div>
{releasePending ? (
<div className="pointer-events-none absolute bottom-1 right-1 rounded-sm bg-black/80 px-2 py-1 text-xs text-slate-200">
Closing...
</div>
) : null}
</div>
);
}
function PtzMobileLandscape({ ptz, onClose, releasePending = false }) {
return (
<div className="flex min-h-0 flex-1 flex-col gap-0.5 overflow-y-auto p-0.5">
{/*
Landscape intentionally retains one control column beside the video.
This is the established PTZ interaction and avoids forcing rover-style
left/right columns onto a camera that has a smaller control inventory.
*/}
<section className="mobile-touch-control grid min-h-[calc(100dvh-0.25rem)] shrink-0 grid-cols-[minmax(0,1fr)_13rem] items-start gap-0.5">
{/*
The video keeps one viewport of height, but the grid row is allowed to
grow when the control column is taller. That makes the sidebar's tail
extend below the video instead of forcing it into a nested scroller.
*/}
<div className="min-w-0 space-y-0.5">
<main className="relative h-[calc(100dvh-0.25rem)] min-h-0 overflow-hidden bg-black">
<button
type="button"
className="absolute left-1 top-1 z-50 rounded-sm border border-white/40 bg-black/80 px-2 py-1 text-xs font-semibold text-white shadow-sm disabled:opacity-50"
disabled={releasePending}
onClick={onClose}
>
Close
</button>
<PtzMediaPane ptz={ptz} open framed={false} />
</main>
{/*
The right control column is naturally taller than the viewport.
Placing room lights after the fixed-height video uses that left-
column space while the whole landscape page continues scrolling as
one surface.
*/}
<HomeAssistantControls />
</div>
{/*
Do not put overflow scrolling on this column. The surrounding PTZ
landscape content is the single page scroller, so a swipe over either
the video area or these controls advances the same document flow.
*/}
<aside className="min-h-0 space-y-0.5">
{/*
Landscape keeps all turn-critical controls in its one existing
sidebar. Queue position belongs first so the operator can confirm
control ownership before touching the camera, while replay follows
the lighting buttons because it is the next secondary action in
the same scroll column.
*/}
<PtzQueueSummary ptz={ptz} />
<PtzMobileControlsPanel ptz={ptz} disabled={!ptz?.isOperator} />
<ReplaySourcesPanel
panelId="ptz-controller-replay-mobile-landscape"
defaultSelectedKey={`ptz:${PTZ_CAMERA_ID}`}
/>
</aside>
</section>
<section className="grid gap-0.5 md:grid-cols-[minmax(0,1fr)_minmax(0,0.7fr)]">
<ChatPanel title="Chat" allowSpectatorInput inputTarget="overlay" />
<PtzPresetPanel ptz={ptz} />
</section>
</div>
);
}
function PtzMobilePortrait({ ptz, onClose, releasePending = false }) {
return (
<div className="flex min-h-0 flex-1 flex-col gap-0.5 overflow-y-auto p-0.5">
<main className="relative aspect-video min-h-0 shrink-0 overflow-hidden bg-black">
<button
type="button"
className="absolute left-1 top-1 z-50 rounded-sm border border-white/40 bg-black/80 px-2 py-1 text-xs font-semibold text-white shadow-sm disabled:opacity-50"
disabled={releasePending}
onClick={onClose}
>
Close
</button>
<PtzMediaPane ptz={ptz} open framed={false} />
</main>
{/*
Portrait gives the video its full available width and places controls
below it. Reusing the landscape sidebar width here was the source of the
cramped portrait presentation, while the controls themselves remain the
same shared PTZ controls used in landscape.
*/}
<section className="mobile-touch-control">
<PtzMobileControlsPanel ptz={ptz} disabled={!ptz?.isOperator} />
</section>
<section className="space-y-0.5">
{/*
Replay and presets are compact secondary actions, so portrait places
them in one equal-width row before the full-width queue and chat. The
explicit two-column grid keeps this arrangement local to portrait and
leaves the desktop and one-column landscape compositions unchanged.
*/}
<div className="grid grid-cols-2 items-start gap-0.5">
<ReplaySourcesPanel
panelId="ptz-controller-replay-mobile-portrait"
defaultSelectedKey={`ptz:${PTZ_CAMERA_ID}`}
/>
<PtzPresetPanel ptz={ptz} />
</div>
<PtzQueueSummary ptz={ptz} />
<ChatPanel title="Chat" allowSpectatorInput inputTarget="overlay" />
{/* Portrait keeps room lights immediately after chat as requested. */}
<HomeAssistantControls />
</section>
</div>
);
}
export function PtzControllerPage({ layout = 'desktop' }) {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
const role = useSessionSelector((state) => state.session?.role || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null);
const { ptzClaim, ptzRelease, pushAlert } = useSessionActions();
const { stopAllMotion } = useControlActions();
const navigate = useNavigate();
const [releasePending, setReleasePending] = useState(false);
const autoClaimSocketRef = useRef(null);
const routeExitReleaseTimerRef = useRef(null);
const participantRef = useRef(false);
const closingThroughButtonRef = useRef(false);
const { value: pageSettings } = useSettingsNamespace('page', {
backgroundTheme: DEFAULT_PAGE_THEME_KEY,
});
const isMobile = layout !== 'desktop';
const canUse = Boolean(ptz?.canUse || isVerified || role === 'admin' || role === 'lockdown');
const isParticipant = Boolean(ptz?.isOperator || ptz?.queuedPosition);
// PTZ is a separate route but shares the browser's page settings. Applying the catalog class to
// its body surface exposes the theme only through layout padding and card gaps; camera pixels,
// controls, and card interiors retain their purpose-built dark backgrounds.
const pageBackgroundClass = usePageThemeClass(pageSettings?.backgroundTheme);
useEffect(() => {
// Route-exit cleanup runs after the last render, so retain the latest
// server-confirmed membership without making the lifecycle effect resubscribe.
participantRef.current = isParticipant;
}, [isParticipant]);
useEffect(() => {
if (routeExitReleaseTimerRef.current) {
clearTimeout(routeExitReleaseTimerRef.current);
routeExitReleaseTimerRef.current = null;
}
return () => {
if (!participantRef.current || closingThroughButtonRef.current) return;
/*
Browser Back and route navigation unmount the PTZ page without invoking
its Close button. Defer release by one task so React Strict Mode's
development-only cleanup/remount cycle can cancel it in the next setup;
a real route exit has no replacement setup, so membership is released.
This is intentionally membership-gated. An admin release command can
revoke the current operator even when the admin is not that operator,
so an admin merely visiting/leaving a disabled or unjoined page must not
emit a release command.
*/
routeExitReleaseTimerRef.current = setTimeout(() => {
routeExitReleaseTimerRef.current = null;
ptzRelease().catch(() => {});
}, 0);
};
}, [ptzRelease]);
useEffect(() => {
if (!featureEnabled || !ptz || !socketId || !canUse) return undefined;
if (ptz.isOperator || ptz.queuedPosition) {
/*
Navigation from the driver queue normally arrives with membership
already established. Mark this socket complete so later session syncs
cannot turn that normal route transition into another claim request.
*/
autoClaimSocketRef.current = socketId;
return undefined;
}
if (autoClaimSocketRef.current === socketId) return undefined;
autoClaimSocketRef.current = socketId;
let active = true;
/*
A direct /ptz load still receives the ordinary user role first, which can
briefly assign a rover. Claiming through the existing server action is
deliberate: ptzCameraService releases that rover ownership before it
activates or queues this socket, keeping one authoritative transition.
The socket-keyed ref suppresses repeats caused by session updates and
React's development effect replay. The server claim is also idempotent for
an existing operator/queue member, which covers an acknowledgement racing
with a fresh public-state sync.
*/
ptzClaim().catch((err) => {
if (!active) return;
pushAlert({
id: `ptz-auto-claim-${socketId}`,
title: 'PTZ camera',
message: err?.message || 'Unable to join the PTZ queue.',
color: '#f59e0b',
lifetimeMs: 6000,
});
});
return () => {
// Do not emit or update UI from a rejected request after this route has
// unmounted; the server still owns completion of any request in flight.
active = false;
};
}, [canUse, featureEnabled, ptz, ptzClaim, pushAlert, socketId]);
const releaseAndClose = useCallback(async () => {
if (releasePending) return;
setReleasePending(true);
closingThroughButtonRef.current = true;
try {
/*
Stop first so a held key/pointer cannot leave ONVIF continuous movement
running while the server removes this socket from the PTZ queue.
*/
stopAllMotion?.();
if (ptz?.isOperator || ptz?.queuedPosition) {
await ptzRelease();
}
navigate('/');
} catch (err) {
// A rejected manual release leaves the route mounted, so route-exit
// cleanup must remain armed for a later Back/navigation attempt.
closingThroughButtonRef.current = false;
throw err;
} finally {
setReleasePending(false);
}
}, [navigate, ptz?.isOperator, ptz?.queuedPosition, ptzRelease, releasePending, stopAllMotion]);
if (!featureEnabled) {
return (
<main className={`flex min-h-dvh items-center justify-center p-2 text-slate-100 ${pageBackgroundClass}`}>
<CardFrame title="PTZ camera" bodyClassName="space-y-1 p-2 text-sm">
<p>The PTZ camera is not available.</p>
<button type="button" className="button-dark w-full" onClick={() => navigate('/')}>Return to driver page</button>
</CardFrame>
</main>
);
}
return (
<main className={`h-dvh w-full overflow-hidden text-slate-100 ${pageBackgroundClass}`}>
{/* The fullscreen CardFrame remains the structural shell. Painting its otherwise
transparent body is what lets every desktop and mobile PTZ composition share one
continuous pattern without threading theme props into each individual child panel. */}
<CardFrame
title={isMobile ? '' : ptz?.name || 'PTZ Camera'}
actions={isMobile ? null : (
<button type="button" className="button-dark text-xs" disabled={releasePending} onClick={releaseAndClose}>
Close
</button>
)}
hideHeader={isMobile}
fillHeight
clipOverflow={false}
className="h-dvh w-screen rounded-none border-0 bg-black!"
bodyClassName={`relative min-h-0 flex-1 ${pageBackgroundClass}`}
>
{isMobile ? (
layout === 'mobile-landscape' ? (
<PtzMobileLandscape ptz={ptz} onClose={releaseAndClose} releasePending={releasePending} />
) : (
<PtzMobilePortrait ptz={ptz} onClose={releaseAndClose} releasePending={releasePending} />
)
) : (
<PtzDesktopFullscreen ptz={ptz} releasePending={releasePending} />
)}
</CardFrame>
</main>
);
}
export default function PtzQueueCard() { export default function PtzQueueCard() {
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera')); const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null); const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
const role = useSessionSelector((state) => state.session?.role || null);
const selfId = useSessionSelector((state) => state.session?.socketId || null); const selfId = useSessionSelector((state) => state.session?.socketId || null);
const { ptzClaim, ptzRelease } = useSessionActions(); const { setOperatingMode } = useSessionActions();
const navigate = useNavigate(); const navigate = useNavigate();
const lookupUser = usePtzQueueLookup(ptz); const now = useSharedClock(1000, Boolean(ptz?.turn?.deadline));
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
const now = useSharedClock(1000, Boolean(ptz?.deadline));
const [pending, setPending] = useState(false); const [pending, setPending] = useState(false);
const canUse = Boolean(ptz?.canUse || isVerified || role === 'admin' || role === 'lockdown'); const canUse = Boolean(ptz?.permissions?.canEnter);
const isParticipant = Boolean(ptz?.isOperator || ptz?.queuedPosition); const isParticipant = Boolean(ptz?.turn?.ownsControl || ptz?.turn?.turnsAhead);
const timerLabel = ptz?.isOperator && ptz?.deadline ? `${formatRemaining(ptz.deadline, now)} left` : ''; const timerLabel = ptz?.turn?.ownsControl && ptz?.turn?.deadline ? `${formatRemaining(ptz.turn.deadline, now)} left` : '';
if (!featureEnabled) return null; if (!featureEnabled) return null;
@@ -881,13 +39,9 @@ export default function PtzQueueCard() {
} }
setPending(true); setPending(true);
try { try {
const response = await ptzClaim(); const response = await setOperatingMode('ptz');
/* // Enter first; the page requests its initial turn after navigation.
The server is authoritative for whether the click became an active turn if (response?.operatingMode === 'ptz') {
or a queued wait. Open only after it confirms one of those states so a
dock-guard rejection does not strand the user in fullscreen.
*/
if (response?.state?.isOperator || response?.state?.queuedPosition) {
navigate('/ptz'); navigate('/ptz');
} }
} catch (err) { } catch (err) {
@@ -901,7 +55,7 @@ export default function PtzQueueCard() {
if (pending) return; if (pending) return;
setPending(true); setPending(true);
try { try {
await ptzRelease(); await setOperatingMode('rover');
} catch (err) { } catch (err) {
alert(err.message || 'Failed to leave PTZ camera.'); alert(err.message || 'Failed to leave PTZ camera.');
} finally { } finally {
@@ -909,13 +63,7 @@ export default function PtzQueueCard() {
} }
}; };
const actionLabel = pending const actionLabel = pending ? '...' : isParticipant ? 'Open' : 'request';
? '...'
: ptz?.isOperator
? 'Open'
: ptz?.queuedPosition
? 'Open'
: 'request';
return ( return (
<CardFrame title={ptz?.name || 'PTZ camera'} bodyClassName="relative space-y-0.5 text-sm"> <CardFrame title={ptz?.name || 'PTZ camera'} bodyClassName="relative space-y-0.5 text-sm">
@@ -925,22 +73,14 @@ export default function PtzQueueCard() {
id: ptz?.id || PTZ_CAMERA_ID, id: ptz?.id || PTZ_CAMERA_ID,
name: ptz?.name || 'PTZ Camera', name: ptz?.name || 'PTZ Camera',
color: ptz?.color || PTZ_DEFAULT_COLOR, color: ptz?.color || PTZ_DEFAULT_COLOR,
// description: ptz?.isOperator
// ? 'Live camera turn active'
// : ptz?.queuedPosition
// ? `Queue position ${ptz.queuedPosition}`
// : 'Pan, tilt, and zoom camera',
}} }}
queue={queue} turn={ptz?.turn}
currentId={currentId}
nextId={nextId}
selfId={selfId} selfId={selfId}
lookupUser={lookupUser}
canClick={canUse && !pending} canClick={canUse && !pending}
pending={pending} pending={pending}
buttonLabel={actionLabel} buttonLabel={actionLabel}
batteryLabel={ptz?.isOperator ? 'LIVE' : ptz?.queuedPosition ? `#${ptz.queuedPosition}` : '--'} batteryLabel={ptz?.turn?.ownsControl ? 'LIVE' : ptz?.turn?.turnsAhead ? `#${ptz.turn.turnsAhead}` : '--'}
batteryClassName={ptz?.isOperator ? 'text-emerald-300' : ptz?.queuedPosition ? 'text-sky-300' : 'text-slate-400'} batteryClassName={ptz?.turn?.ownsControl ? 'text-emerald-300' : ptz?.turn?.turnsAhead ? 'text-sky-300' : 'text-slate-400'}
timerLabel={timerLabel} timerLabel={timerLabel}
onRequest={handleRequest} onRequest={handleRequest}
showAction={canUse} showAction={canUse}
+16 -87
View File
@@ -1,15 +1,14 @@
// PTZ Live Video // PTZ Live Video
// Purpose: Plays the single PTZ camera WHEP stream with the same fresh-session retry loop used by rover video. // Purpose: Plays the single PTZ camera WHEP stream with the same fresh-session retry loop used by rover video.
// Scope: Owns browser-side WHEP playback/retry only; server authorization and snapshot fallback policy stay outside. // Scope: Owns browser-side WHEP playback/retry only; server authorization and snapshot fallback policy stay outside.
import { useCallback, useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { useVideoRequests } from '../../hooks/useVideoRequests.js'; import { useVideoRequests } from '../../hooks/useVideoRequests.js';
import { WhepPlayer } from '../../lib/whepPlayer.js'; import { startWhepPlayback } from '../../lib/whepPlayback.js';
import { RESTART_DELAY_MS } from '../RoverMediaPlayer/constants.js'; import useWhepRestart from '../../hooks/useWhepRestart.js';
export const PTZ_CAMERA_ID = 'ptz-camera'; export const PTZ_CAMERA_ID = 'ptz-camera';
const PTZ_AUDIO_RETRY_MS = 1000; const PTZ_AUDIO_RETRY_MS = 1000;
const TERMINAL_WHEP_STATES = new Set(['error', 'failed', 'disconnected', 'closed']);
const AUTHORIZATION_ERROR_RE = /not authorized/i; const AUTHORIZATION_ERROR_RE = /not authorized/i;
function isAuthorizationError(error) { function isAuthorizationError(error) {
@@ -25,14 +24,10 @@ export default function PtzLiveVideo({
fallback = null, fallback = null,
}) { }) {
const videoRef = useRef(null); const videoRef = useRef(null);
const retryTimerRef = useRef(null);
const playTimerRef = useRef(null); const playTimerRef = useRef(null);
const enabledRef = useRef(enabled);
const fallbackRef = useRef(false);
const playerGenerationRef = useRef(0);
const [status, setStatus] = useState('idle'); const [status, setStatus] = useState('idle');
const [detail, setDetail] = useState(null); const [detail, setDetail] = useState(null);
const [restartToken, setRestartToken] = useState(0); const { restartToken, scheduleRestart, cancelRestart } = useWhepRestart(enabled);
const sources = useVideoRequests( const sources = useVideoRequests(
[{ type: 'ptz', id: PTZ_CAMERA_ID, key: PTZ_CAMERA_ID }], [{ type: 'ptz', id: PTZ_CAMERA_ID, key: PTZ_CAMERA_ID }],
{ enabled, version: restartToken }, { enabled, version: restartToken },
@@ -41,100 +36,34 @@ export default function PtzLiveVideo({
const shouldUseFallback = Boolean(source?.error && isAuthorizationError(source.error)); const shouldUseFallback = Boolean(source?.error && isAuthorizationError(source.error));
useEffect(() => { useEffect(() => {
enabledRef.current = enabled; // An authorization denial switches to snapshots and cancels any queued retry.
}, [enabled]); if (shouldUseFallback) cancelRestart();
}, [shouldUseFallback, cancelRestart]);
useEffect(() => {
fallbackRef.current = shouldUseFallback;
}, [shouldUseFallback]);
useEffect(() => {
if (enabled && !shouldUseFallback) return undefined;
/*
A retry that was scheduled before the server denied live access should not
keep firing in snapshot mode. Clear it when live playback is no longer the
active display policy.
*/
if (retryTimerRef.current) {
clearTimeout(retryTimerRef.current);
retryTimerRef.current = null;
}
return undefined;
}, [enabled, shouldUseFallback]);
const scheduleRestart = useCallback(() => {
if (!enabledRef.current || fallbackRef.current) return;
/*
WHEP sessions are one-shot browser/server negotiations. When the camera
reboots, the old PeerConnection and token can look alive enough to keep a
black element on screen, but they are not useful anymore. Bumping this
token forces useVideoRequests to ask the server for a new MediaMTX auth
session before creating the next WhepPlayer.
*/
clearTimeout(retryTimerRef.current);
retryTimerRef.current = setTimeout(() => {
retryTimerRef.current = null;
setRestartToken(Date.now());
}, RESTART_DELAY_MS);
}, []);
useEffect(() => { useEffect(() => {
if (!enabled || shouldUseFallback || !source?.url || !videoRef.current) return undefined; if (!enabled || shouldUseFallback || !source?.url || !videoRef.current) return undefined;
let active = true; const video = videoRef.current;
const generation = playerGenerationRef.current + 1; const stop = startWhepPlayback({
playerGenerationRef.current = generation;
const player = new WhepPlayer({
url: source.url, url: source.url,
token: source.token, token: source.token,
video: videoRef.current, video: videoRef.current,
startMuted, startMuted,
scheduleRestart,
onStatus: (nextStatus, info) => { onStatus: (nextStatus, info) => {
/*
Old PeerConnection callbacks can arrive after React has already
cleaned up this effect for a newer token. Only the currently-owned
generation is allowed to update status or schedule another restart.
*/
if (!active || playerGenerationRef.current !== generation) return;
const normalized = String(nextStatus || '').toLowerCase();
setStatus(nextStatus || 'unknown'); setStatus(nextStatus || 'unknown');
setDetail(info || null); setDetail(info || null);
if (TERMINAL_WHEP_STATES.has(normalized)) { },
scheduleRestart(); onError: (err) => {
} setStatus('error');
setDetail(err.message || 'WHEP start failed');
}, },
}); });
player.start().catch((err) => {
if (!active || playerGenerationRef.current !== generation) return;
setStatus('error');
setDetail(err.message || 'WHEP start failed');
scheduleRestart();
});
return () => {
/*
Mark inactive before stop() because WhepPlayer reports "stopped" during
normal cleanup. Cleanup-driven stops should not immediately schedule the
next retry; only the replacement effect should own the new connection.
*/
active = false;
player.stop();
};
}, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted]);
useEffect(() => {
const video = videoRef.current;
if (!enabled || shouldUseFallback || !source?.url || !video) return undefined;
const generation = playerGenerationRef.current;
const handleEnded = () => { const handleEnded = () => {
if (playerGenerationRef.current !== generation) return;
setStatus('stopped'); setStatus('stopped');
setDetail('ended'); setDetail('ended');
scheduleRestart(); scheduleRestart();
}; };
const handleError = () => { const handleError = () => {
if (playerGenerationRef.current !== generation) return;
setStatus('error'); setStatus('error');
setDetail(video.error?.message || 'video element error'); setDetail(video.error?.message || 'video element error');
scheduleRestart(); scheduleRestart();
@@ -145,8 +74,9 @@ export default function PtzLiveVideo({
return () => { return () => {
video.removeEventListener('ended', handleEnded); video.removeEventListener('ended', handleEnded);
video.removeEventListener('error', handleError); video.removeEventListener('error', handleError);
stop();
}; };
}, [enabled, scheduleRestart, shouldUseFallback, source?.url]); }, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted]);
useEffect(() => { useEffect(() => {
const video = videoRef.current; const video = videoRef.current;
@@ -178,7 +108,6 @@ export default function PtzLiveVideo({
}, [enabled, shouldUseFallback, source?.url, startMuted, status]); }, [enabled, shouldUseFallback, source?.url, startMuted, status]);
useEffect(() => () => { useEffect(() => () => {
if (retryTimerRef.current) clearTimeout(retryTimerRef.current);
if (playTimerRef.current) clearInterval(playTimerRef.current); if (playTimerRef.current) clearInterval(playTimerRef.current);
}, []); }, []);
+8 -30
View File
@@ -21,36 +21,20 @@ function roleColors(role) {
} }
} }
function formatQueueUserLabel(user, selfId) {
/*
Queue chips need to be readable even when a socket has no nickname yet.
Keeping the socket-prefix fallback here means rover queues and PTZ queues
degrade identically instead of each target inventing its own anonymous label.
*/
if (!user) return '';
const base = user.nickname || user.label || user.socketId?.slice(0, 6) || 'unknown';
if (user.socketId && user.socketId === selfId) {
return `${base} (you)`;
}
return base;
}
export function QueueUserChips({ export function QueueUserChips({
targetId, targetId,
queue = [], turn,
currentId = null,
nextId = null,
selfId = null, selfId = null,
lookupUser,
}) { }) {
if (!queue.length) { const { queue = [], currentId, nextId } = turn || {};
if (!queue?.length) {
return <p className="text-[0.7rem] text-slate-500">No queue.</p>; return <p className="text-[0.7rem] text-slate-500">No queue.</p>;
} }
return ( return (
<div className="flex flex-wrap items-center gap-0.5"> <div className="flex flex-wrap items-center gap-0.5">
{queue.map((socketId, idx) => { {queue.map((user, idx) => {
const user = lookupUser?.(socketId) || { socketId, nickname: null, role: null }; const { socketId } = user;
const isCurrent = socketId === currentId; const isCurrent = socketId === currentId;
const isNext = Boolean(nextId && socketId === nextId && !isCurrent); const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
/* /*
@@ -69,7 +53,7 @@ export function QueueUserChips({
className={`flex items-center gap-0.5 rounded-sm px-1 text-[0.7rem] ${highlightClass}`} className={`flex items-center gap-0.5 rounded-sm px-1 text-[0.7rem] ${highlightClass}`}
> >
<span className={`${roleColors(user.role)} font-semibold`}> <span className={`${roleColors(user.role)} font-semibold`}>
{formatQueueUserLabel(user, selfId)} {user.name}{socketId === selfId ? ' (you)' : ''}
</span> </span>
{isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>} {isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>}
{isNext && <span className="text-[0.65rem] text-emerald-100">next</span>} {isNext && <span className="text-[0.65rem] text-emerald-100">next</span>}
@@ -82,11 +66,8 @@ export function QueueUserChips({
export default function QueueTargetRow({ export default function QueueTargetRow({
target, target,
queue = [], turn,
currentId = null,
nextId = null,
selfId = null, selfId = null,
lookupUser,
canClick = false, canClick = false,
pending = false, pending = false,
locked = false, locked = false,
@@ -164,11 +145,8 @@ export default function QueueTargetRow({
</div> </div>
<QueueUserChips <QueueUserChips
targetId={targetId} targetId={targetId}
queue={queue} turn={turn}
currentId={currentId}
nextId={nextId}
selfId={selfId} selfId={selfId}
lookupUser={lookupUser}
/> />
</div> </div>
{showAction ? ( {showAction ? (
@@ -1,7 +1,6 @@
// constants // constants
// Purpose: Defines the constants module and the local helpers/components used in this file. // Purpose: Defines the constants module and the local helpers/components used in this file.
// Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit. // Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
export const RESTART_DELAY_MS = 2000;
export const UNMUTE_RETRY_MS = 3000; export const UNMUTE_RETRY_MS = 3000;
export const AUDIO_RETRY_MS = 3000; export const AUDIO_RETRY_MS = 3000;
export const BRUSH_CURRENT_THRESHOLD_MA = 40; export const BRUSH_CURRENT_THRESHOLD_MA = 40;
+30 -70
View File
@@ -1,5 +1,6 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { WhepPlayer } from '../../lib/whepPlayer.js'; import { startWhepPlayback } from '../../lib/whepPlayback.js';
import useWhepRestart from '../../hooks/useWhepRestart.js';
import { useTelemetrySelector } from '../../context/TelemetryContext.jsx'; import { useTelemetrySelector } from '../../context/TelemetryContext.jsx';
import { import {
mainBrushAudioTelemetryEqual, mainBrushAudioTelemetryEqual,
@@ -11,7 +12,6 @@ import { useRoverSnapshots } from '../../hooks/useRoverSnapshots.js';
import { useSettingsNamespace } from '../../settings/index.js'; import { useSettingsNamespace } from '../../settings/index.js';
import { AUDIO_SETTINGS_DEFAULTS, VIDEO_SETTINGS_DEFAULTS } from '../../settings/namespaces.js'; import { AUDIO_SETTINGS_DEFAULTS, VIDEO_SETTINGS_DEFAULTS } from '../../settings/namespaces.js';
import { import {
RESTART_DELAY_MS,
UNMUTE_RETRY_MS, UNMUTE_RETRY_MS,
AUDIO_RETRY_MS, AUDIO_RETRY_MS,
BRUSH_CURRENT_THRESHOLD_MA, BRUSH_CURRENT_THRESHOLD_MA,
@@ -212,10 +212,10 @@ export default function RoverMediaPlayer({
mainBrushOvercurrent: Boolean(sensors?.wheelOvercurrents?.mainBrush), mainBrushOvercurrent: Boolean(sensors?.wheelOvercurrents?.mainBrush),
} }
: mainBrushTelemetry; : mainBrushTelemetry;
const { restartToken, scheduleRestart } = useWhepRestart();
const { restartToken: audioRestartToken, scheduleRestart: scheduleAudioRestart } = useWhepRestart();
const videoRef = useRef(null); const videoRef = useRef(null);
const audioRef = useRef(null); const audioRef = useRef(null);
const restartTimer = useRef(null);
const audioRestartTimer = useRef(null);
const audioPlayInterval = useRef(null); const audioPlayInterval = useRef(null);
const unmuteTimer = useRef(null); const unmuteTimer = useRef(null);
const volumeFadeFrameRef = useRef(null); const volumeFadeFrameRef = useRef(null);
@@ -224,8 +224,6 @@ export default function RoverMediaPlayer({
const [detail, setDetail] = useState(null); const [detail, setDetail] = useState(null);
const [audioStatus, setAudioStatus] = useState('idle'); const [audioStatus, setAudioStatus] = useState('idle');
const [audioDetail, setAudioDetail] = useState(null); const [audioDetail, setAudioDetail] = useState(null);
const [restartToken, setRestartToken] = useState(0);
const [audioRestartToken, setAudioRestartToken] = useState(0);
const [muted, setMuted] = useState(true); const [muted, setMuted] = useState(true);
const hasDedicatedAudio = Boolean(resolvedAudioSessionInfo?.url); const hasDedicatedAudio = Boolean(resolvedAudioSessionInfo?.url);
const usingSnapshot = videoMode === 'snapshot' || (!videoMode && !resolvedSessionInfo?.url); const usingSnapshot = videoMode === 'snapshot' || (!videoMode && !resolvedSessionInfo?.url);
@@ -341,15 +339,6 @@ export default function RoverMediaPlayer({
logAudio('settings/update'); logAudio('settings/update');
}, [logAudio]); }, [logAudio]);
const scheduleRestart = useCallback(() => {
clearTimeout(restartTimer.current);
restartTimer.current = setTimeout(() => setRestartToken(Date.now()), RESTART_DELAY_MS);
}, []);
const scheduleAudioRestart = useCallback(() => {
clearTimeout(audioRestartTimer.current);
audioRestartTimer.current = setTimeout(() => setAudioRestartToken(Date.now()), RESTART_DELAY_MS);
}, []);
const ensurePlayback = useCallback(async () => { const ensurePlayback = useCallback(async () => {
const video = videoRef.current; const video = videoRef.current;
if (!video) return; if (!video) return;
@@ -397,8 +386,6 @@ export default function RoverMediaPlayer({
useEffect( useEffect(
() => () => { () => () => {
clearTimeout(restartTimer.current);
clearTimeout(audioRestartTimer.current);
clearTimeout(unmuteTimer.current); clearTimeout(unmuteTimer.current);
clearInterval(audioPlayInterval.current); clearInterval(audioPlayInterval.current);
if (volumeFadeFrameRef.current) { if (volumeFadeFrameRef.current) {
@@ -426,41 +413,27 @@ export default function RoverMediaPlayer({
if (!isVideoVisible || usingSnapshot || !resolvedSessionInfo?.url || !videoRef.current) { if (!isVideoVisible || usingSnapshot || !resolvedSessionInfo?.url || !videoRef.current) {
return undefined; return undefined;
} }
let active = true;
let player;
const resetMuteId = setTimeout(() => setMuted(true), 0); const resetMuteId = setTimeout(() => setMuted(true), 0);
const handleStatus = (nextStatus, info) => { const stop = startWhepPlayback({
if (!active) return;
logAudio('video/status', { nextStatus, info: info || null });
setStatus(nextStatus);
setDetail(info || null);
if (nextStatus === 'playing') {
ensurePlayback();
}
if (['error', 'failed', 'disconnected', 'closed'].includes(nextStatus)) {
scheduleRestart();
}
};
player = new WhepPlayer({
url: resolvedSessionInfo.url, url: resolvedSessionInfo.url,
token: resolvedSessionInfo.token, token: resolvedSessionInfo.token,
video: videoRef.current, video: videoRef.current,
receiveAudio: !hasDedicatedAudio, receiveAudio: !hasDedicatedAudio,
onStatus: handleStatus, scheduleRestart,
onStatus: (nextStatus, info) => {
logAudio('video/status', { nextStatus, info: info || null });
setStatus(nextStatus);
setDetail(info || null);
if (nextStatus === 'playing') ensurePlayback();
},
onError: (err) => {
setStatus('error');
setDetail(err.message);
},
}); });
player.start().catch((err) => {
if (!active) return;
setStatus('error');
setDetail(err.message);
scheduleRestart();
});
return () => { return () => {
active = false;
clearTimeout(resetMuteId); clearTimeout(resetMuteId);
player?.stop(); stop();
}; };
}, [ }, [
usingSnapshot, usingSnapshot,
@@ -536,37 +509,24 @@ export default function RoverMediaPlayer({
if (!resolvedAudioSessionInfo?.url || !audioRef.current) { if (!resolvedAudioSessionInfo?.url || !audioRef.current) {
return undefined; return undefined;
} }
let active = true; return startWhepPlayback({
let player;
const handleStatus = (nextStatus, info) => {
if (!active) return;
logAudio('audio/status', { nextStatus, info: info || null });
setAudioStatus(nextStatus);
setAudioDetail(info || null);
if (['error', 'failed'].includes(nextStatus)) {
scheduleAudioRestart();
}
};
player = new WhepPlayer({
url: resolvedAudioSessionInfo.url, url: resolvedAudioSessionInfo.url,
token: resolvedAudioSessionInfo.token, token: resolvedAudioSessionInfo.token,
video: audioRef.current, video: audioRef.current,
audioOnly: true, audioOnly: true,
onStatus: handleStatus, // Dedicated rover audio historically retries only errors and failures.
terminalStates: ['error', 'failed'],
scheduleRestart: scheduleAudioRestart,
onStatus: (nextStatus, info) => {
logAudio('audio/status', { nextStatus, info: info || null });
setAudioStatus(nextStatus);
setAudioDetail(info || null);
},
onError: (err) => {
setAudioStatus('error');
setAudioDetail(err.message);
},
}); });
player.start().catch((err) => {
if (!active) return;
setAudioStatus('error');
setAudioDetail(err.message);
scheduleAudioRestart();
});
return () => {
active = false;
player?.stop();
};
}, [ }, [
resolvedAudioSessionInfo?.url, resolvedAudioSessionInfo?.url,
resolvedAudioSessionInfo?.token, resolvedAudioSessionInfo?.token,
@@ -96,16 +96,12 @@ function ScrollableQueueContent({ enabled = false, children }) {
export default function RoverQueuesPanel({ export default function RoverQueuesPanel({
title = 'Rovers', title = 'Rovers',
roster: rosterOverride = null, roster: rosterOverride = null,
turnQueues: turnQueuesOverride = null,
users: usersOverride = null,
externalInstance = null, externalInstance = null,
disabledOverlay = '', disabledOverlay = '',
fillHeight = false, fillHeight = false,
}) { }) {
const role = useSessionSelector((state) => state.session?.role || null); const role = useSessionSelector((state) => state.session?.role || null);
const localRoster = useSessionSelector((state) => state.session?.roster ?? []); const localRoster = useSessionSelector((state) => state.session?.roster ?? []);
const localTurnQueues = useSessionSelector((state) => state.session?.turnQueues ?? {});
const localUsers = useSessionSelector((state) => state.session?.users ?? []);
const interInstanceEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'interInstance')); const interInstanceEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'interInstance'));
const { value: pageSettings } = useSettingsNamespace('page', { interInstanceTransferSettings: true }); const { value: pageSettings } = useSettingsNamespace('page', { interInstanceTransferSettings: true });
const selfId = useSessionSelector((state) => state.session?.socketId || null); const selfId = useSessionSelector((state) => state.session?.socketId || null);
@@ -124,8 +120,6 @@ export default function RoverQueuesPanel({
const externalBlocked = Boolean(externalMode && disabledOverlay); const externalBlocked = Boolean(externalMode && disabledOverlay);
const includeInterInstanceSettings = pageSettings?.interInstanceTransferSettings !== false; const includeInterInstanceSettings = pageSettings?.interInstanceTransferSettings !== false;
const roster = Array.isArray(rosterOverride) ? rosterOverride : localRoster; const roster = Array.isArray(rosterOverride) ? rosterOverride : localRoster;
const turnQueues = turnQueuesOverride && typeof turnQueuesOverride === 'object' ? turnQueuesOverride : localTurnQueues;
const users = Array.isArray(usersOverride) ? usersOverride : localUsers;
const canRequest = useMemo( const canRequest = useMemo(
() => (externalMode ? !externalBlocked : role && role !== 'spectator'), () => (externalMode ? !externalBlocked : role && role !== 'spectator'),
@@ -136,8 +130,8 @@ export default function RoverQueuesPanel({
[role], [role],
); );
const hasDeadlines = useMemo( const hasDeadlines = useMemo(
() => Object.values(turnQueues || {}).some((info) => info?.deadline || info?.idleDeadline), () => roster.some((rover) => rover.turn?.deadline || rover.turn?.idleDeadline),
[turnQueues], [roster],
); );
/* /*
Queue timers are shown in whole seconds, and several queue panels can be Queue timers are shown in whole seconds, and several queue panels can be
@@ -146,14 +140,6 @@ export default function RoverQueuesPanel({
*/ */
const now = useSharedClock(1000, hasDeadlines); const now = useSharedClock(1000, hasDeadlines);
const rosterItems = useMemo(() => {
const known = new Set(roster.map((rover) => String(rover.id)));
const extra = Object.keys(turnQueues || {})
.filter((id) => !known.has(String(id)))
.map((id) => ({ id, name: id, locked: false, batteryState: null }));
return [...roster, ...extra];
}, [roster, turnQueues]);
async function handleRequest(targetRoverId) { async function handleRequest(targetRoverId) {
if (!targetRoverId) return; if (!targetRoverId) return;
if (externalBlocked) return; if (externalBlocked) return;
@@ -181,8 +167,6 @@ export default function RoverQueuesPanel({
} }
} }
const lookupUser = (socketId) =>
users.find((u) => u.socketId === socketId) || { socketId, nickname: null, role: null };
async function handleRebootOwnRover() { async function handleRebootOwnRover() {
if (rebootPending) return; if (rebootPending) return;
@@ -224,14 +208,13 @@ export default function RoverQueuesPanel({
> >
<ScrollableQueueContent enabled={fillHeight}> <ScrollableQueueContent enabled={fillHeight}>
<div className="relative space-y-0.5"> <div className="relative space-y-0.5">
{rosterItems.length === 0 ? ( {roster.length === 0 ? (
<p className="text-sm text-slate-500">No rovers registered.</p> <p className="text-sm text-slate-500">No rovers registered.</p>
) : ( ) : (
<ul className="space-y-0.5 text-sm"> <ul className="space-y-0.5 text-sm">
{rosterItems.map((rover) => { {roster.map((rover) => {
const roverId = String(rover.id); const roverId = String(rover.id);
const info = turnQueues?.[roverId] || null; const info = rover.turn;
const queue = info?.queue || [];
/* /*
A turn handoff and an inactivity skip are separate server A turn handoff and an inactivity skip are separate server
deadlines. Keep them separate through formatting so an idle deadlines. Keep them separate through formatting so an idle
@@ -246,14 +229,8 @@ export default function RoverQueuesPanel({
const idleRemainingSeconds = idleDeadline const idleRemainingSeconds = idleDeadline
? Math.max(0, Math.ceil((idleDeadline - now) / 1000)) ? Math.max(0, Math.ceil((idleDeadline - now) / 1000))
: null; : null;
const currentId = info?.current || null; const currentId = info?.currentId || null;
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1; const nextId = info?.nextId || null;
const nextId =
queue.length > 1
? currentIdx >= 0
? queue[(currentIdx + 1) % queue.length]
: queue[0]
: null;
const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId); const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId);
const isSelfNext = Boolean(selfId && nextId && nextId === selfId); const isSelfNext = Boolean(selfId && nextId && nextId === selfId);
const timerLabel = isSelfCurrent const timerLabel = isSelfCurrent
@@ -281,11 +258,8 @@ export default function RoverQueuesPanel({
<QueueTargetRow <QueueTargetRow
key={rover.id} key={rover.id}
target={{ ...rover, rover, roverId, id: roverId }} target={{ ...rover, rover, roverId, id: roverId }}
queue={queue} turn={info}
currentId={currentId}
nextId={nextId}
selfId={selfId} selfId={selfId}
lookupUser={lookupUser}
canClick={canClickRow} canClick={canClickRow}
pending={Boolean(pending[roverId])} pending={Boolean(pending[roverId])}
locked={locked} locked={locked}
@@ -25,7 +25,6 @@ function useAudio(src, volume = 1) {
export default function TurnAlertListener() { export default function TurnAlertListener() {
const socketId = useSessionSelector((state) => state.session?.socketId || null); const socketId = useSessionSelector((state) => state.session?.socketId || null);
const assignments = useSessionSelector((state) => state.session?.turnQueues || {});
const roster = useSessionSelector((state) => state.session?.roster || []); const roster = useSessionSelector((state) => state.session?.roster || []);
const { pushAlert } = useSessionActions(); const { pushAlert } = useSessionActions();
const { value: audioSettings } = useSettingsNamespace('audio', AUDIO_SETTINGS_DEFAULTS); const { value: audioSettings } = useSettingsNamespace('audio', AUDIO_SETTINGS_DEFAULTS);
@@ -38,18 +37,19 @@ export default function TurnAlertListener() {
useEffect(() => { useEffect(() => {
if (!socketId) return; if (!socketId) return;
const newlyMine = []; const newlyMine = [];
Object.entries(assignments).forEach(([roverId, info]) => { roster.forEach((rover) => {
if (!info || !info.current || info.current !== socketId) return; const roverId = String(rover.id);
if (!rover.turn?.ownsControl) return;
if (!seenRoversRef.current.has(roverId)) { if (!seenRoversRef.current.has(roverId)) {
newlyMine.push(roverId); newlyMine.push(roverId);
seenRoversRef.current.add(roverId); seenRoversRef.current.add(roverId);
} }
}); });
Object.keys(assignments).forEach((roverId) => { for (const roverId of seenRoversRef.current) {
if (!assignments[roverId] || assignments[roverId].current !== socketId) { if (!roster.some((rover) => String(rover.id) === roverId && rover.turn?.ownsControl)) {
seenRoversRef.current.delete(roverId); seenRoversRef.current.delete(roverId);
} }
}); }
if (newlyMine.length === 0) return; if (newlyMine.length === 0) return;
newlyMine.forEach((roverId) => { newlyMine.forEach((roverId) => {
const roverName = roster.find((r) => String(r.id) === String(roverId))?.name || roverId; const roverName = roster.find((r) => String(r.id) === String(roverId))?.name || roverId;
@@ -60,7 +60,7 @@ export default function TurnAlertListener() {
}); });
}); });
playSound(); playSound();
}, [assignments, playSound, pushAlert, roster, socketId]); }, [playSound, pushAlert, roster, socketId]);
return null; return null;
} }
+11 -24
View File
@@ -73,8 +73,8 @@ export default function UserListPanel({
const users = useSessionSelector((state) => state.session?.users ?? []); const users = useSessionSelector((state) => state.session?.users ?? []);
const selfId = useSessionSelector((state) => state.session?.socketId || null); const selfId = useSessionSelector((state) => state.session?.socketId || null);
const mode = useSessionSelector((state) => state.session?.mode || null); const mode = useSessionSelector((state) => state.session?.mode || null);
const turnQueues = useSessionSelector((state) => state.session?.turnQueues || {});
const roster = useSessionSelector((state) => state.session?.roster || []); const roster = useSessionSelector((state) => state.session?.roster || []);
const queuedRovers = roster.filter((rover) => rover.turn?.queue != null);
const isTurnsMode = mode === 'turns'; const isTurnsMode = mode === 'turns';
const [turnView, setTurnView] = useState('queues'); const [turnView, setTurnView] = useState('queues');
@@ -98,11 +98,6 @@ export default function UserListPanel({
[roster], [roster],
); );
const lookupUser = useCallback(
(socketId) => users.find((u) => u.socketId === socketId) || { socketId, nickname: null, role: null },
[users],
);
const secondsRemaining = useCallback((deadline) => { const secondsRemaining = useCallback((deadline) => {
if (!deadline) return null; if (!deadline) return null;
const ms = deadline - Date.now(); const ms = deadline - Date.now();
@@ -141,10 +136,11 @@ export default function UserListPanel({
className={`flex items-center gap-0.5 ${compact ? 'py-0.25 text-[0.8rem]' : 'text-sm'}`} className={`flex items-center gap-0.5 ${compact ? 'py-0.25 text-[0.8rem]' : 'text-sm'}`}
> >
<p className={`font-semibold ${roleColors(user.role)}`}>{formatLabel(user, selfId)}</p> <p className={`font-semibold ${roleColors(user.role)}`}>{formatLabel(user, selfId)}</p>
{user.roverId ? ( {user.roverId || user.operatingModeDisplay ? (
<RoverLabel <RoverLabel
roverId={user.roverId} roverId={user.roverId}
color={rosterEntry(user.roverId)?.color} name={user.operatingModeDisplay?.name}
color={user.operatingModeDisplay?.color || rosterEntry(user.roverId)?.color}
fallback={user.roverId} fallback={user.roverId}
className="text-[0.7rem]" className="text-[0.7rem]"
/> />
@@ -164,7 +160,7 @@ export default function UserListPanel({
return ( return (
<CardFrame <CardFrame
title={!hideHeader ? (isTurnsMode ? (showQueuesSection ? 'Turn queues' : 'Users') : 'Users') : ''} title={!hideHeader ? (isTurnsMode ? (showQueuesSection ? 'Turn queues' : 'Users') : 'Users') : ''}
meta={!hideHeader ? (showQueuesSection && isTurnsMode ? Object.keys(turnQueues || {}).length : sorted.length) : null} meta={!hideHeader ? (showQueuesSection && isTurnsMode ? queuedRovers.length : sorted.length) : null}
hideHeader={hideHeader} hideHeader={hideHeader}
fillHeight={fillHeight} fillHeight={fillHeight}
className={className} className={className}
@@ -208,21 +204,14 @@ export default function UserListPanel({
) : null} ) : null}
{showQueuesSection ? ( {showQueuesSection ? (
<div className={`space-y-0.5 px-0 pb-0 ${turnsListClass} ${compact ? 'text-[0.8rem]' : ''}`}> <div className={`space-y-0.5 px-0 pb-0 ${turnsListClass} ${compact ? 'text-[0.8rem]' : ''}`}>
{Object.keys(turnQueues || {}).length === 0 ? ( {queuedRovers.length === 0 ? (
<p className="text-sm text-slate-500">No turn queues yet.</p> <p className="text-sm text-slate-500">No turn queues yet.</p>
) : ( ) : (
Object.entries(turnQueues).map(([roverId, info]) => { queuedRovers.map(({ id: roverId, turn: info }) => {
const queue = info?.queue || []; const queue = info?.queue || [];
const deadline = info?.idleDeadline || info?.deadline || null; const deadline = info?.idleDeadline || info?.deadline || null;
const remaining = secondsRemaining(deadline); const remaining = secondsRemaining(deadline);
const currentId = info?.current || null; const { currentId, nextId } = info;
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1;
const nextId =
queue.length > 1
? currentIdx >= 0
? queue[(currentIdx + 1) % queue.length]
: queue[0]
: null;
return ( return (
<div key={roverId} className={`flex flex-col gap-0.5 ${compact ? 'text-[0.8rem] py-0.25' : 'text-sm'}`}> <div key={roverId} className={`flex flex-col gap-0.5 ${compact ? 'text-[0.8rem] py-0.25' : 'text-sm'}`}>
<div className="flex items-center gap-0.5"> <div className="flex items-center gap-0.5">
@@ -239,11 +228,10 @@ export default function UserListPanel({
<p className="text-[0.75rem] text-slate-500">No drivers queued.</p> <p className="text-[0.75rem] text-slate-500">No drivers queued.</p>
) : ( ) : (
<div className="flex flex-wrap items-center gap-0.5"> <div className="flex flex-wrap items-center gap-0.5">
{queue.map((socketId, idx) => { {queue.map((user, idx) => {
const user = lookupUser(socketId); const { socketId } = user;
const isCurrent = socketId === currentId; const isCurrent = socketId === currentId;
const isNext = Boolean(nextId && socketId === nextId && !isCurrent); const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
const isSelf = Boolean(selfId && socketId === selfId);
const isAdmin = const isAdmin =
user.role === 'admin' || user.role === 'lockdown'; user.role === 'admin' || user.role === 'lockdown';
const highlightClass = isCurrent const highlightClass = isCurrent
@@ -257,10 +245,9 @@ export default function UserListPanel({
className={`flex items-center gap-0.5 rounded-sm px-1 ${compact ? 'text-[0.7rem]' : 'text-[0.8rem]'} ${highlightClass}`} className={`flex items-center gap-0.5 rounded-sm px-1 ${compact ? 'text-[0.7rem]' : 'text-[0.8rem]'} ${highlightClass}`}
> >
<span className={`${roleColors(user.role)} font-semibold`}> <span className={`${roleColors(user.role)} font-semibold`}>
{formatLabel(user, selfId)} {user.name}{socketId === selfId ? ' (you)' : ''}
</span> </span>
{isAdmin && <span className="text-[0.7rem] text-amber-200">★</span>} {isAdmin && <span className="text-[0.7rem] text-amber-200">★</span>}
{/* {isSelf && <span className="text-[0.7rem] text-white">YOU</span>} */}
{isCurrent && <span className="text-[0.7rem] text-slate-200">now</span>} {isCurrent && <span className="text-[0.7rem] text-slate-200">now</span>}
{isNext && <span className="text-[0.7rem] text-emerald-100">next</span>} {isNext && <span className="text-[0.7rem] text-emerald-100">next</span>}
</span> </span>
@@ -1,596 +0,0 @@
// Vip PTZ Camera Card
// Purpose: Provides the verified-user entry point and fullscreen controller for the single Reolink PTZ camera.
// Scope: Owns PTZ UI state only; server-side PTZ ownership, rover handoff, and command authorization remain authoritative.
import { useCallback, useEffect, useMemo, useState } from 'react';
import '../MobileControls/mobileControls.css';
import { createPortal } from 'react-dom';
import ChatPanel from '../ChatPanel/index.jsx';
import CardFrame from '../CardFrame/index.jsx';
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx';
import PtzLiveVideo from '../PtzLiveVideo/index.jsx';
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
import KeyPill from './VipAudioUploadCard/KeyPill.jsx';
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
import { useControlActions } from '../../controls/index.js';
import ControlHint from '../ControlHint/index.jsx';
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
import { isFeatureEnabled } from '../../lib/features.js';
import { triggerTouchHaptic } from '../../lib/touchHaptics.js';
const PTZ_CAMERA_ID = 'ptz-camera';
function formatRemaining(deadline) {
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - Date.now()) / 1000));
if (!remaining) return '--';
const minutes = Math.floor(remaining / 60);
const seconds = remaining % 60;
return `${minutes}:${String(seconds).padStart(2, '0')}`;
}
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeIrMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function nextIrMode(currentMode) {
const current = normalizeIrMode(currentMode);
if (current === 'Auto') return 'On';
if (current === 'On') return 'Off';
return 'Auto';
}
function PtzSnapshotPreview({ feed, label = 'PTZ Camera' }) {
return (
<div className="relative w-full overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
{feed?.objectUrl ? (
<img src={feed.objectUrl} alt={label} className="h-full w-full object-cover" />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
)}
{/*
Even on this legacy VIP surface, keep the video pane itself identical
to rover media panes: no camera title inside the frame, only the small
top-left stream status. Any PTZ name/context belongs to the card chrome
around the media, not the media player.
*/}
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
</div>
</div>
</div>
);
}
function StatusRow({ label, value, tone = '' }) {
return (
<div className="flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">{label}</span>
<span className={`min-w-0 truncate font-medium ${tone || 'text-slate-100'}`}>{value}</span>
</div>
);
}
function formatPublisherProgress(progress = null) {
/*
ffmpeg progress is intentionally shown as raw operational numbers instead
of translated prose. When the PTZ video feels delayed, fps/speed/drop/out
time make it obvious whether ffmpeg itself is keeping up or the delay is
somewhere before/after the transcoder.
*/
if (!progress) return null;
return [
progress.fps ? `fps ${progress.fps}` : null,
progress.speed ? `speed ${progress.speed}` : null,
progress.drop_frames ? `drop ${progress.drop_frames}` : null,
progress.out_time ? `out ${progress.out_time}` : null,
].filter(Boolean).join(' | ');
}
function PtzQueueList({ queue = [], operatorLabel = '' }) {
const hasQueue = Array.isArray(queue) && queue.length > 0;
return (
<div className="space-y-0.5">
<div className="panel-muted text-xs">Turn queue</div>
<div className="space-y-0.5">
{operatorLabel ? (
<div className="surface flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">Now</span>
<span className="min-w-0 truncate text-emerald-200">{operatorLabel}</span>
</div>
) : null}
{hasQueue ? queue.map((entry, index) => (
<div key={entry.socketId || `${entry.label}-${index}`} className="surface flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">{index + 1}</span>
<span className="min-w-0 truncate text-slate-100">{entry.label || entry.socketId || 'queued user'}</span>
</div>
)) : (
<div className="surface text-xs text-slate-400">No one waiting</div>
)}
</div>
</div>
);
}
function PtzStatePanel({ ptz, onClose, onRelease, releaseDisabled = false }) {
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const publisher = ptz?.publisher || {};
const publisherStatus = publisher.running
? `running${publisher.pid ? ` ${publisher.pid}` : ''}`
: publisher.restartAt
? 'restarting'
: publisher.lastEvent || 'stopped';
const publisherProgress = formatPublisherProgress(publisher.progress);
const statusTone = ptz?.error ? 'text-amber-300' : ptz?.isOperator ? 'text-emerald-300' : 'text-slate-100';
return (
<CardFrame
title="Camera state"
actions={onClose ? <button type="button" className="button-dark text-xs" onClick={onClose}>Close</button> : null}
bodyClassName="space-y-0.5 p-1 text-sm"
>
<StatusRow label="Mode" value={ptz?.isOperator ? 'operator' : ptz?.queuedPosition ? `queued ${ptz.queuedPosition}` : 'spectator'} tone={statusTone} />
<StatusRow label="Operator" value={ptz?.operatorLabel || 'none'} />
<StatusRow label="Remaining" value={formatRemaining(ptz?.deadline)} />
<StatusRow label="Spotlight" value={spotlightOn ? 'On' : 'Off'} tone={spotlightOn ? 'text-emerald-300' : 'text-slate-200'} />
<StatusRow label="Infrared mode" value={irMode} />
<StatusRow label="Stream" value={ptz?.status || ptz?.error || 'idle'} tone={ptz?.error ? 'text-amber-300' : ''} />
<StatusRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} />
{publisherProgress ? <StatusRow label="Progress" value={publisherProgress} /> : null}
{publisher.lastStderr ? (
<div className="surface max-h-24 overflow-y-auto whitespace-pre-wrap wrap-break-word font-mono text-[0.68rem] leading-tight text-slate-200">
{publisher.lastStderr}
</div>
) : null}
{ptz?.blocked?.message ? (
<div className="rounded-sm border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
{ptz.blocked.message}
</div>
) : null}
{onRelease ? (
<button type="button" className="button-dark w-full text-xs" disabled={releaseDisabled} onClick={onRelease}>
Release camera
</button>
) : null}
</CardFrame>
);
}
function PtzLightingControls({ ptz, disabled = false }) {
const { ptzSpotlight, ptzIr } = useSessionActions();
const [busy, setBusy] = useState('');
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const toggleSpotlight = async (nextOn) => {
if (disabled) return;
setBusy('spotlight');
try {
await ptzSpotlight({ state: nextOn ? 1 : 0 });
} finally {
setBusy('');
}
};
const cycleIr = async () => {
if (disabled) return;
triggerTouchHaptic('button');
setBusy('ir');
try {
await ptzIr({ state: nextIrMode(irMode) });
} finally {
setBusy('');
}
};
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<GPIOToggleControl
label="Spotlight"
on={spotlightOn}
disabled={disabled || busy === 'spotlight'}
onToggle={toggleSpotlight}
heightClass="min-h-14"
/>
<button
type="button"
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
disabled={disabled || busy === 'ir'}
onClick={cycleIr}
>
<span className="text-sm font-semibold">Infrared</span>
<span className="rounded-sm bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
</button>
</div>
);
}
function PtzMobileZoomButtons({ disabled = false }) {
const { setCameraAxisIntent } = useControlActions();
const stopZoom = useCallback(() => {
// This is a zoom-only release; the shared adapter retains any simultaneous
// pan/tilt intent from the movement pad in its next combined motion state.
setCameraAxisIntent(0);
}, [setCameraAxisIntent]);
const startZoom = useCallback(
(direction) => (event) => {
/*
The adapter owns renewal for held PTZ state. Publishing the direction
once avoids a component-local repeat timer and keeps this legacy card on
the exact same motion path as the dedicated PTZ route.
*/
event.preventDefault();
if (disabled) return;
triggerTouchHaptic('button');
event.currentTarget.setPointerCapture?.(event.pointerId);
setCameraAxisIntent(direction);
},
[disabled, setCameraAxisIntent],
);
const stopFromPointer = useCallback(
(event) => {
event?.preventDefault?.();
if (disabled) return;
stopZoom();
},
[disabled, stopZoom],
);
useEffect(
() => () => {
// Orientation changes can unmount the button before pointerup. Clear the
// held zoom state explicitly instead of waiting for the server watchdog.
setCameraAxisIntent(0);
},
[setCameraAxisIntent],
);
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(-1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onLostPointerCapture={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom out
</button>
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onLostPointerCapture={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom in
</button>
</div>
);
}
function PtzMobileControlsPanel({ ptz, disabled = false }) {
return (
<div className="mobile-touch-control space-y-0.5">
<PtzMobileZoomButtons disabled={disabled} />
<div className="mobile-touch-control h-44 min-h-0">
{/*
Reuse the rover mobile movement card instead of building a second PTZ
joystick. Its drive vector goes through the shared internal control
layer, where the PTZ adapter already converts movement plus speed mode
into camera pan/tilt commands.
*/}
<ControlPadPanel compact disabled={disabled} />
</div>
<PtzLightingControls ptz={ptz} disabled={disabled} />
</div>
);
}
function PtzControlReference() {
const rows = [
['Tilt up', 'driveForward'],
['Tilt down', 'driveBackward'],
['Pan left', 'driveLeft'],
['Pan right', 'driveRight'],
['Zoom in', 'cameraUp'],
['Zoom out', 'cameraDown'],
['Spotlight', 'headlightToggle'],
['Infrared mode', 'laserToggle'],
];
return (
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
{rows.map(([label, actionId]) => (
<div key={label} className="surface flex items-center justify-between gap-1">
<span className="text-slate-400">{label}</span>
{/* Use the same key display component as the rest of the UI so PTZ
controls read as normal mapped controls instead of custom labels. */}
<KeyPill label={<ControlHint actionId={actionId} />} />
</div>
))}
</CardFrame>
);
}
function PtzController({ open, onClose, layout = 'desktop' }) {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isOperator = Boolean(ptz?.isOperator);
const nonTurnSnapshotsActive = useSessionSelector(
(state) => Boolean(state.session?.bandwidthSavings?.nonTurnVideo?.snapshotsActive),
);
const isMobile = layout === 'mobile-portrait' || layout === 'mobile-landscape';
const { ptzRelease } = useSessionActions();
/*
The VIP fullscreen surface is available to queued PTZ users too. Let queued
users see live video only when the central non-turn video policy allows it;
all movement and light controls still remain guarded by isOperator.
*/
const shouldUseLiveVideo = isOperator || !nonTurnSnapshotsActive;
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !shouldUseLiveVideo });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const [releasePending, setReleasePending] = useState(false);
if (!open) return null;
const releaseAndClose = async () => {
setReleasePending(true);
try {
await ptzRelease();
onClose();
} finally {
setReleasePending(false);
}
};
const desktopSidebar = (
<>
<div className="shrink-0">
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
</div>
{isOperator ? (
<div className="shrink-0">
<PtzLightingControls ptz={ptz} />
</div>
) : (
<div className="shrink-0">
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
Live PTZ controls unlock when your camera turn is active.
</CardFrame>
</div>
)}
<div className="shrink-0">
<PtzControlReference />
</div>
<div className="min-h-48 flex-1">
{/*
The global chat key focuses the input registered by ChatPanel through
ChatContext. Keeping a real ChatPanel mounted inside the PTZ fullscreen
sidebar lets the normal keyboard path focus chat and lets Enter submit
through the existing chat composer form instead of adding PTZ-specific
chat handling.
*/}
<ChatPanel fillHeight title="Chat" />
</div>
<div className="shrink-0">
<ReplaySourcesPanel panelId="ptz-controller-replay" defaultSelectedKey={`ptz:${PTZ_CAMERA_ID}`} />
</div>
<div className="shrink-0">
<PtzStatePanel
ptz={ptz}
onClose={onClose}
onRelease={isOperator ? releaseAndClose : null}
releaseDisabled={releasePending}
/>
</div>
</>
);
const mobileSidebar = (
<>
<div className="shrink-0">
<PtzMobileControlsPanel ptz={ptz} disabled={!isOperator} />
</div>
<div className="min-h-40 flex-1">
{/*
Mobile uses the same ChatPanel registration as desktop so the mapped
chat key and the on-screen input stay on one shared chat implementation.
*/}
<ChatPanel fillHeight title="Chat" />
</div>
<div className="shrink-0">
<ReplaySourcesPanel
panelId="ptz-controller-replay-mobile"
defaultSelectedKey={`ptz:${PTZ_CAMERA_ID}`}
/>
</div>
<div className="shrink-0">
<PtzStatePanel
ptz={ptz}
onClose={onClose}
onRelease={isOperator ? releaseAndClose : null}
releaseDisabled={releasePending}
/>
</div>
<div className="shrink-0">
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
</div>
</>
);
const sidebarWidthClass = isMobile
? 'grid-cols-[minmax(0,1fr)_14rem]'
: 'grid-cols-[minmax(0,1fr)_20rem]';
const controller = (
<div className="fixed inset-0 z-110 h-dvh w-screen overflow-hidden bg-black text-slate-100">
<CardFrame
hideHeader
fillHeight
clipOverflow={false}
className="h-dvh w-screen rounded-none border-0 bg-black!"
bodyClassName={`grid h-full min-h-0 overflow-hidden ${sidebarWidthClass}`}
>
<main className="relative min-h-0 min-w-0 bg-black">
{shouldUseLiveVideo ? (
<PtzLiveVideo enabled startMuted={false} />
) : (
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
)}
</main>
<aside className="flex h-full min-h-0 items-stretch overflow-hidden border-l border-neutral-600 bg-neutral-950 text-sm">
<div className="flex min-h-0 w-full flex-col gap-0.5 overflow-y-auto p-0.5">
{isMobile ? mobileSidebar : desktopSidebar}
</div>
</aside>
</CardFrame>
</div>
);
/*
The controller is a true fullscreen surface, so mount it directly under
document.body instead of inside the VIP tab/card tree. That keeps tab panel
spacing, mobile banners, and parent overflow rules from creating visible
gaps around a fixed-position camera interface.
*/
return createPortal(controller, document.body);
}
export default function VipPtzCameraCard({ onMessage, fullWidth = false, layout = 'desktop' }) {
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
const { ptzClaim, ptzRelease } = useSessionActions();
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: Boolean(featureEnabled) });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const [controllerOpen, setControllerOpen] = useState(false);
const [pending, setPending] = useState(false);
const wrapClass = fullWidth ? 'w-full' : 'mx-auto w-full max-w-xl';
const queueText = useMemo(() => {
if (ptz?.isOperator) return 'Your turn';
if (ptz?.queuedPosition) return `Queue position ${ptz.queuedPosition}`;
if (ptz?.operatorLabel) return `${ptz.operatorLabel} operating`;
return 'Available';
}, [ptz?.isOperator, ptz?.operatorLabel, ptz?.queuedPosition]);
if (!featureEnabled) return null;
const handleClaim = async () => {
setPending(true);
onMessage?.('');
try {
const response = await ptzClaim();
/*
Requesting the PTZ camera is only enough to open fullscreen after the
server confirms the user actually became the operator or entered the
queue. Dock-required failures reject before queue entry, so they must
leave the user on this card with the dock/charge message visible.
*/
if (response?.state?.isOperator || response?.state?.queuedPosition) setControllerOpen(true);
if (response?.state?.isOperator) onMessage?.('PTZ camera turn active.');
else if (response?.state?.queuedPosition) onMessage?.(`Joined PTZ queue at position ${response.state.queuedPosition}.`);
} catch (err) {
/*
Rover-to-rover switch denial uses the browser alert popup for the
dock-and-charge message. PTZ claim denial should feel identical because
it is enforcing the same "do not abandon an undocked rover" rule.
*/
alert(err.message || 'PTZ request failed.');
onMessage?.(err.message || 'PTZ request failed.');
} finally {
setPending(false);
}
};
const handleRelease = async () => {
setPending(true);
try {
await ptzRelease();
onMessage?.('Left PTZ camera.');
} catch (err) {
onMessage?.(err.message || 'Failed to leave PTZ camera.');
} finally {
setPending(false);
}
};
return (
<>
<CardFrame title="PTZ camera" className={wrapClass} bodyClassName="text-sm text-slate-300">
<div className="grid w-full gap-0.5 md:grid-cols-[minmax(16rem,0.8fr)_minmax(0,1.2fr)] md:items-start">
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
<div className="flex w-full min-w-0 flex-col gap-0.5">
<div className="grid w-full grid-cols-2 gap-0.5 text-left text-xs">
<div className="surface-muted p-1">
<p className="text-slate-500">State</p>
<p className="truncate text-slate-100">{queueText}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Remaining</p>
<p className="text-slate-100">{formatRemaining(ptz?.deadline)}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Spotlight</p>
<p className="truncate text-slate-100">{isSpotlightOn(ptz?.light) ? 'On' : 'Off'}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Infrared</p>
<p className="truncate text-slate-100">{normalizeIrMode(ptz?.ir?.state)}</p>
</div>
</div>
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
{ptz?.blocked?.message ? (
<p className="w-full rounded-sm border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
{ptz.blocked.message}
</p>
) : null}
<div className="grid w-full grid-cols-2 gap-0.5">
{ptz?.isOperator ? (
<>
<button type="button" className="button-dark text-xs" onClick={() => setControllerOpen(true)}>
Open controller
</button>
<button type="button" className="button-dark text-xs" disabled={pending} onClick={handleRelease}>
Release
</button>
</>
) : (
<button
type="button"
className="button-dark col-span-2 text-xs"
disabled={pending || !isVerified}
onClick={ptz?.queuedPosition ? handleRelease : handleClaim}
>
{ptz?.queuedPosition ? 'Leave queue' : pending ? 'Requesting...' : 'Claim camera'}
</button>
)}
</div>
</div>
</div>
</CardFrame>
<PtzController open={controllerOpen} onClose={() => setControllerOpen(false)} layout={layout} />
</>
);
}
@@ -25,22 +25,20 @@ function Group({ title, children }) {
</section>; </section>;
} }
const actionNames = { motors: 'Brushes / vacuum', servo: 'Camera tilt', headlight: 'Headlight', const actionNames = { drive: 'Drive', motors: 'Brushes / vacuum', servo: 'Camera tilt', headlight: 'Headlight',
laser: 'Laser', horn: 'Horn', peripheral: 'Accessory', song: 'Beeper', raw: 'Recorded control' }; laser: 'Laser', horn: 'Horn', peripheral: 'Accessory', song: 'Beeper', raw: 'Recorded control' };
function WheelMeter({ label, speed }) { function commandParameters(command) {
const known = Number.isFinite(speed); const fields = [];
const value = known ? Math.max(-500, Math.min(500, speed)) : 0; const visit = (value, path) => {
return <div className="surface-muted min-w-0 space-y-1"> if (value && typeof value === 'object' && !Array.isArray(value)) {
<div className="flex justify-between gap-2 text-xs"><span>{label}</span><span>{known ? `${value} mm/s` : 'Waiting'}</span></div> for (const [key, item] of Object.entries(value)) visit(item, path ? `${path}.${key}` : key);
<div role="meter" aria-label={label} aria-valuemin={-500} aria-valuemax={500} aria-valuenow={value} } else {
aria-valuetext={known ? `${value} millimeters per second` : 'Unavailable'} className="relative h-3 overflow-hidden rounded bg-slate-800"> fields.push(`${path}: ${typeof value === 'number' ? Number(value.toFixed(2)) : JSON.stringify(value)}`);
<div className={`absolute inset-y-0 rounded ${value < 0 ? 'bg-amber-400' : 'bg-sky-400'}`} }
style={{ left: `${value < 0 ? 50 + value / 10 : 50}%`, width: `${Math.abs(value) / 10}%` }} /> };
<div className="absolute inset-y-0 left-1/2 w-px bg-slate-400" /> for (const [key, value] of Object.entries(command || {})) if (key !== 'type') visit(value, key);
</div> return fields.join(' · ');
<div className="flex justify-between text-xs text-slate-500"><span>Reverse</span><span>Forward</span></div>
</div>;
} }
export default function VipRoverLearningCard() { export default function VipRoverLearningCard() {
@@ -130,12 +128,12 @@ function LearningCard({ roverId }) {
return <CardFrame title="Rover learning" className="w-full" bodyClassName="text-sm text-slate-300"> return <CardFrame title="Rover learning" className="w-full" bodyClassName="text-sm text-slate-300">
<div className="space-y-2"> <div className="space-y-2">
<p className="text-center text-xs text-slate-400"> <p className="text-center text-xs text-slate-400">
{session ? `${session.modelName} · ${session.status}` : state ? 'Choose a model to control your rover.' : 'Waiting for rover learning status…'} {session ? `${session.modelName} · ${session.status}` : state ? 'Stopped' : 'Waiting for rover learning status…'}
</p> </p>
<div className="flex flex-wrap items-center justify-center gap-1"> <div className="flex flex-wrap items-center justify-center gap-1">
<select aria-label="Rover model" className={fieldClass} value={chosen?.id || ''} onChange={(event) => setSelectedId(event.target.value)} disabled={working || !models.length}> <select aria-label="Rover model" className={fieldClass} value={chosen?.id || ''} onChange={(event) => setSelectedId(event.target.value)} disabled={working || !models.length}>
{!models.length && <option value="">No published models yet</option>} {!models.length && <option value="">No published models yet</option>}
{models.map((model) => <option key={model.id} value={model.id}>{favorites.includes(model.id) ? '★ ' : ''}{model.name}{model.experimental ? ' (experimental)' : ''}{model.compatible ? '' : model.formatVersion !== 2 ? ' (older model format)' : ' (incompatible)'}</option>)} {models.map((model) => <option key={model.id} value={model.id}>{favorites.includes(model.id) ? '★ ' : ''}{model.name}{model.experimental ? ' (experimental)' : ''}{model.compatible ? '' : model.formatVersion !== 4 ? ' (older model format)' : ' (incompatible)'}</option>)}
</select> </select>
<button type="button" className="button-dark text-sm disabled:opacity-50" aria-pressed={Boolean(chosen && favorites.includes(chosen.id))} disabled={!chosen} onClick={favorite}> <button type="button" className="button-dark text-sm disabled:opacity-50" aria-pressed={Boolean(chosen && favorites.includes(chosen.id))} disabled={!chosen} onClick={favorite}>
{chosen && favorites.includes(chosen.id) ? '★ Favorite' : '☆ Favorite'} {chosen && favorites.includes(chosen.id) ? '★ Favorite' : '☆ Favorite'}
@@ -147,38 +145,37 @@ function LearningCard({ roverId }) {
</div> </div>
{state?.control?.canSnapshot && <div className="text-center"> {state?.control?.canSnapshot && <div className="text-center">
<button type="button" className="button-dark text-sm disabled:opacity-50" disabled={working} onClick={() => act('snapshot')}>Save learner snapshot</button> <button type="button" className="button-dark text-sm disabled:opacity-50" disabled={working} onClick={() => act('snapshot')}>Save learner snapshot</button>
<p className="mt-1 text-xs text-slate-400">Preserves the latest completed training checkpoint, including experimental results.</p>
</div>} </div>}
{session && <section className="space-y-2" aria-label="Live model actions"> {session && <section className="space-y-2" aria-label="Live model actions">
<p className="text-center text-xs text-sky-300">{session.predictionStatus || 'Waiting for fresh video and sensors'}</p> <p className="text-center text-xs text-sky-300">{session.predictionStatus || 'Waiting for fresh video and sensors'}</p>
<div className="grid grid-cols-2 gap-2">
<WheelMeter label="Predicted left wheel" speed={session.wheelSpeeds?.[0]} />
<WheelMeter label="Predicted right wheel" speed={session.wheelSpeeds?.[1]} />
</div>
<label className="surface-muted block space-y-1 text-xs"> <label className="surface-muted block space-y-1 text-xs">
<span className="flex justify-between gap-2"><span>Accessory action threshold</span><span>{(thresholdDraft ?? session.threshold ?? .7).toFixed(2)}</span></span> <span className="flex justify-between gap-2"><span>Command threshold</span><span>{(thresholdDraft ?? session.threshold ?? .7).toFixed(2)}</span></span>
<input type="range" min="0.05" max="0.99" step="0.01" className="w-full accent-sky-400" <input type="range" min="0.05" max="0.99" step="0.01" className="w-full accent-sky-400"
value={thresholdDraft ?? session.threshold ?? .7} disabled={working || !state?.control?.canAdjust} value={thresholdDraft ?? session.threshold ?? .7} disabled={working || !state?.control?.canAdjust}
onChange={(event) => setThresholdDraft(Number(event.target.value))} onChange={(event) => setThresholdDraft(Number(event.target.value))}
onPointerUp={commitThreshold} onKeyUp={commitThreshold} onBlur={commitThreshold} /> onPointerUp={commitThreshold} onKeyUp={commitThreshold} onBlur={commitThreshold} />
<span className="block text-slate-400">Lower allows more accessory actions. Wheel speeds are independent.</span>
</label> </label>
<div className="grid max-h-52 grid-cols-1 gap-1 overflow-y-auto @[28rem]:grid-cols-2"> <div className="grid max-h-52 grid-cols-1 gap-1 overflow-y-auto @[28rem]:grid-cols-2">
{session.proposals?.map((proposal) => { {session.proposals?.map((proposal) => {
const submission = session.submissions?.find((item) => item.command.type === proposal.command.type const submission = proposal.reason === 'Proposed' && session.submissions?.find((item) => item.command.type === proposal.command.type
&& (proposal.command.type !== 'peripheral' || (item.command.peripheral?.id === proposal.command.peripheral?.id && (proposal.command.type !== 'peripheral' || (item.command.peripheral?.id === proposal.command.peripheral?.id
&& item.command.peripheral?.control === proposal.command.peripheral?.control))); && item.command.peripheral?.control === proposal.command.peripheral?.control)));
const status = proposal.reason === 'Proposed' && submission ? submission.result : proposal.reason; const status = proposal.reason === 'Proposed' && submission ? submission.result : proposal.reason;
return <div key={proposal.slot} className="surface-muted space-y-1 text-xs"> return <div key={proposal.slot} className="surface-muted space-y-1 text-xs">
<div className="flex justify-between gap-2"><span>{actionNames[proposal.command.type] || proposal.command.type}{proposal.command.type === 'peripheral' ? ` · ${proposal.command.peripheral?.id} / ${proposal.command.peripheral?.control}` : ''}</span><span>{proposal.score.toFixed(2)}</span></div> <div className="flex justify-between gap-2"><span>{actionNames[proposal.command.type] || proposal.command.type}{proposal.command.type === 'peripheral' ? ` · ${proposal.command.peripheral?.id} / ${proposal.command.peripheral?.control}` : ''}</span><span>{proposal.score.toFixed(2)}</span></div>
<div className="h-1.5 overflow-hidden rounded bg-slate-800"><div className={status === 'issued' ? 'h-full bg-emerald-400' : 'h-full bg-slate-500'} style={{ width: `${proposal.score * 100}%` }} /></div> <p className="break-words text-slate-300">{commandParameters(proposal.command)}</p>
<div role="meter" aria-label={`${actionNames[proposal.command.type] || proposal.command.type} score`}
aria-valuemin={0} aria-valuemax={1} aria-valuenow={proposal.score} className="relative h-2 overflow-hidden rounded bg-slate-800">
<div className={status === 'issued' ? 'h-full bg-emerald-400' : 'h-full bg-sky-500'} style={{ width: `${proposal.score * 100}%` }} />
<div className="absolute inset-y-0 w-px bg-white" style={{ left: `${(session.appliedThreshold ?? session.threshold ?? .7) * 100}%` }} />
</div>
<p className={status === 'issued' ? 'text-emerald-300' : 'text-slate-400'}>{status === 'issued' ? 'Submitted' : status}</p> <p className={status === 'issued' ? 'text-emerald-300' : 'text-slate-400'}>{status === 'issued' ? 'Submitted' : status}</p>
{submission && <p className="break-words text-slate-400">Sent: {commandParameters(submission.command)}</p>}
</div>; </div>;
})} })}
</div> </div>
</section>} </section>}
{state?.control?.reason && <p className="text-center text-xs text-slate-400">{state.control.reason}</p>} {state?.control?.reason && <p className="text-center text-xs text-slate-400">{state.control.reason}</p>}
{/* <p className="text-center text-xs text-slate-500">Manual controls stay available and do not stop the model.</p> */}
<p className="text-center text-xs text-slate-400">Recording: {recording ? recording.paused ? 'paused for disk space' : `${recording.sessions.filter((item) => item.video === 'Recording human driving').length} driving · ${recording.sessions.filter((item) => item.video === 'Dock lead-in buffer').length} dock buffer(s)` : 'unavailable'} · Training: {training?.status || 'unavailable'}</p> <p className="text-center text-xs text-slate-400">Recording: {recording ? recording.paused ? 'paused for disk space' : `${recording.sessions.filter((item) => item.video === 'Recording human driving').length} driving · ${recording.sessions.filter((item) => item.video === 'Dock lead-in buffer').length} dock buffer(s)` : 'unavailable'} · Training: {training?.status || 'unavailable'}</p>
{(error || state?.catalogError) && <p role="alert" className="break-words text-xs text-amber-300">{error || state.catalogError}</p>} {(error || state?.catalogError) && <p role="alert" className="break-words text-xs text-amber-300">{error || state.catalogError}</p>}
<details className="surface-muted text-xs" onToggle={(event) => setDetailsOpen(event.currentTarget.open)}> <details className="surface-muted text-xs" onToggle={(event) => setDetailsOpen(event.currentTarget.open)}>
@@ -205,8 +202,11 @@ function LearningCard({ roverId }) {
<Row label="Status">{training?.status || 'Unavailable'}</Row> <Row label="Status">{training?.status || 'Unavailable'}</Row>
<Row label="Steps">{display(training?.steps)}</Row> <Row label="Steps">{display(training?.steps)}</Row>
<Row label="Examples processed">{display(training?.examples)}</Row> <Row label="Examples processed">{display(training?.examples)}</Row>
<Row label="Wheel / event / accessory losses">{training?.losses ? `${training.losses.wheels.toFixed(4)} / ${training.losses.events.toFixed(4)} / ${training.losses.accessoryValues.toFixed(4)}` : 'Unavailable'}</Row> <Row label="Event / parameter losses">{training?.losses ? `${training.losses.events?.toFixed(4) ?? 'Unavailable'} / ${training.losses.commandValues?.toFixed(4) ?? 'Unavailable'}` : 'Unavailable'}</Row>
<Row label="Wheel error / stopped baseline / previous speed baseline">{training?.evaluation?.wheelMaeMmPerSecond == null ? 'Unavailable' : `${training.evaluation.wheelMaeMmPerSecond.toFixed(1)} / ${training.evaluation.stoppedBaselineMaeMmPerSecond.toFixed(1)} / ${training.evaluation.previousSpeedBaselineMaeMmPerSecond.toFixed(1)} mm/s`}</Row> <Row label="Parameter error">{training?.evaluation?.numericMse?.toFixed(4) ?? 'Unavailable'}</Row>
<Row label="Missed / total stop events">{training?.evaluation?.stopEvents == null ? 'Unavailable' : `${training.evaluation.missedStops} / ${training.evaluation.stopEvents}`}</Row>
<Row label="Unwanted / no-command windows">{training?.evaluation?.noCommandWindows == null ? 'Unavailable' : `${training.evaluation.unwantedCommandWindows} / ${training.evaluation.noCommandWindows}`}</Row>
<Row label="Usable held-out recordings">{display(training?.evaluation?.heldOutSessions)}</Row>
<Row label="Training loss">{training?.loss == null ? 'Unavailable' : training.loss.toFixed(4)}</Row> <Row label="Training loss">{training?.loss == null ? 'Unavailable' : training.loss.toFixed(4)}</Row>
<Row label="Current job age">{training?.status === 'training' ? age(now, training.startedAt) : 'Not running'}</Row> <Row label="Current job age">{training?.status === 'training' ? age(now, training.startedAt) : 'Not running'}</Row>
<Row label="Next attempt">{time(training?.nextAttemptAt)}</Row> <Row label="Next attempt">{time(training?.nextAttemptAt)}</Row>
@@ -223,6 +223,7 @@ function LearningCard({ roverId }) {
<Row label="Steps between Latest updates">{display(limits?.training.checkpointEverySteps)}</Row> <Row label="Steps between Latest updates">{display(limits?.training.checkpointEverySteps)}</Row>
</dl>{training?.error && <p className="break-words text-amber-300">{training.error}</p>} </dl>{training?.error && <p className="break-words text-amber-300">{training.error}</p>}
{training?.reason && <p>{training.reason}</p>} {training?.reason && <p>{training.reason}</p>}
{training?.evaluation?.gateReasons?.map((reason) => <p key={reason}>{reason}</p>)}
{training?.rejectionReasons?.map((reason) => <p className="break-words text-amber-300" key={reason}>{reason}</p>)} {training?.rejectionReasons?.map((reason) => <p className="break-words text-amber-300" key={reason}>{reason}</p>)}
</Group> </Group>
<Group title="Selected model"><dl> <Group title="Selected model"><dl>
+3 -1
View File
@@ -411,8 +411,10 @@ export function SessionProvider({ children }) {
emitWithAck('audioLevels:setPersonalAdjustmentRange', { maxAdjustmentPercent }), emitWithAck('audioLevels:setPersonalAdjustmentRange', { maxAdjustmentPercent }),
setPrivateSafety: (roverId, safety = {}) => setPrivateSafety: (roverId, safety = {}) =>
emitWithAck('session:privateSafety:set', { roverId, safety }), emitWithAck('session:privateSafety:set', { roverId, safety }),
ptzClaim: () => emitWithAck('ptzCamera:claim'), setOperatingMode: (operatingMode) => emitWithAck('session:setOperatingMode', { operatingMode }),
ptzRequestTurn: () => emitWithAck('ptzCamera:requestTurn'),
ptzRelease: () => emitWithAck('ptzCamera:release'), ptzRelease: () => emitWithAck('ptzCamera:release'),
ptzRevokeOperator: () => emitWithAck('ptzCamera:revokeOperator'),
ptzSpotlight: (payload = {}) => emitWithAck('ptzCamera:spotlight', payload), ptzSpotlight: (payload = {}) => emitWithAck('ptzCamera:spotlight', payload),
ptzIr: (payload = {}) => emitWithAck('ptzCamera:ir', payload), ptzIr: (payload = {}) => emitWithAck('ptzCamera:ir', payload),
ptzListPresets: () => emitWithAck('ptzCamera:presets:list'), ptzListPresets: () => emitWithAck('ptzCamera:presets:list'),
+29 -10
View File
@@ -2,7 +2,7 @@
// Purpose: Hooks the single PTZ camera into the internal control action layer. // Purpose: Hooks the single PTZ camera into the internal control action layer.
// Scope: Owns PTZ-specific control mixing and socket commands so keyboard, // Scope: Owns PTZ-specific control mixing and socket commands so keyboard,
// mobile, desktop, and gamepad inputs do not each learn camera-specific rules. // mobile, desktop, and gamepad inputs do not each learn camera-specific rules.
import { useCallback, useEffect, useMemo, useRef } from 'react'; import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useSocket } from '../context/SocketContext.jsx'; import { useSocket } from '../context/SocketContext.jsx';
import { useSessionActions, useSessionSelector } from '../context/SessionContext.jsx'; import { useSessionActions, useSessionSelector } from '../context/SessionContext.jsx';
@@ -102,9 +102,27 @@ function nextIrMode(currentMode) {
export function usePtzControlAdapter() { export function usePtzControlAdapter() {
const socket = useSocket(); const socket = useSocket();
const { ptzSpotlight, ptzIr } = useSessionActions(); const { ptzSpotlight, ptzIr, pushAlert } = useSessionActions();
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null); const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isActive = Boolean(ptz?.isOperator); const isActive = Boolean(ptz?.permissions?.canControl);
const [lightingPending, setLightingPending] = useState({});
const lightingPendingRef = useRef({});
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const sendLighting = useCallback((key, send) => {
// Keyboard repeats and simultaneous touch/key input share one in-flight
// command per light, so neither can race a pending state change.
if (lightingPendingRef.current[key]) return;
lightingPendingRef.current[key] = true;
setLightingPending((current) => ({ ...current, [key]: true }));
send().catch((err) => {
pushAlert({ title: 'PTZ camera', message: err.message, color: '#f59e0b' });
}).finally(() => {
lightingPendingRef.current[key] = false;
setLightingPending((current) => ({ ...current, [key]: false }));
});
}, [pushAlert]);
const lastMotionSignatureRef = useRef(payloadSignature(PTZ_STOP)); const lastMotionSignatureRef = useRef(payloadSignature(PTZ_STOP));
const desiredMotionRef = useRef(PTZ_STOP); const desiredMotionRef = useRef(PTZ_STOP);
const panTiltIntentRef = useRef({ pan: 0, tilt: 0 }); const panTiltIntentRef = useRef({ pan: 0, tilt: 0 });
@@ -227,7 +245,7 @@ export function usePtzControlAdapter() {
const setSpotlight = useCallback( const setSpotlight = useCallback(
(nextOn) => { (nextOn) => {
if (!isActive) return false; if (!isActive) return false;
const desiredOn = typeof nextOn === 'boolean' ? nextOn : !isSpotlightOn(ptz?.light); const desiredOn = typeof nextOn === 'boolean' ? nextOn : !spotlightOn;
/* /*
Lighting keybinds should use the exact acknowledged command action as Lighting keybinds should use the exact acknowledged command action as
the visible PTZ buttons. Movement remains fire-and-forget because it is the visible PTZ buttons. Movement remains fire-and-forget because it is
@@ -235,10 +253,10 @@ export function usePtzControlAdapter() {
from the existing authorization/error contract and must not maintain a from the existing authorization/error contract and must not maintain a
second socket-only behavior merely because its source is a keybind. second socket-only behavior merely because its source is a keybind.
*/ */
ptzSpotlight({ state: desiredOn ? 1 : 0 }).catch(() => {}); sendLighting('spotlight', () => ptzSpotlight({ state: desiredOn ? 1 : 0 }));
return true; return true;
}, },
[isActive, ptz?.light, ptzSpotlight], [isActive, spotlightOn, ptzSpotlight, sendLighting],
); );
const setIr = useCallback( const setIr = useCallback(
@@ -246,13 +264,13 @@ export function usePtzControlAdapter() {
if (!isActive) return false; if (!isActive) return false;
const desiredState = typeof nextOn === 'boolean' const desiredState = typeof nextOn === 'boolean'
? (nextOn ? 'On' : 'Off') ? (nextOn ? 'On' : 'Off')
: nextIrMode(ptz?.ir?.state); : nextIrMode(irMode);
// Match the button path for the same reason as spotlight above. The // Match the button path for the same reason as spotlight above. The
// shared laser key continues to select IR; only its transport is unified. // shared laser key continues to select IR; only its transport is unified.
ptzIr({ state: desiredState }).catch(() => {}); sendLighting('ir', () => ptzIr({ state: desiredState }));
return true; return true;
}, },
[isActive, ptz?.ir?.state, ptzIr], [isActive, irMode, ptzIr, sendLighting],
); );
useEffect(() => { useEffect(() => {
@@ -300,12 +318,13 @@ export function usePtzControlAdapter() {
() => ({ () => ({
isActive, isActive,
state: ptz, state: ptz,
lighting: { spotlightOn, irMode, pending: lightingPending },
applyDriveVector, applyDriveVector,
setZoomIntent, setZoomIntent,
setSpotlight, setSpotlight,
setIr, setIr,
stopMotion, stopMotion,
}), }),
[applyDriveVector, isActive, ptz, setIr, setSpotlight, setZoomIntent, stopMotion], [applyDriveVector, isActive, ptz, spotlightOn, irMode, lightingPending, setIr, setSpotlight, setZoomIntent, stopMotion],
); );
} }
@@ -7,7 +7,7 @@ import { useSessionSelector } from '../../../context/SessionContext.jsx';
export default function DisplayPtzOperatorBadge() { export default function DisplayPtzOperatorBadge() {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null); const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const operatorLabel = String(ptz?.operatorLabel || '').trim(); const operatorLabel = String(ptz?.turn?.queue?.find((entry) => entry.socketId === ptz.turn.currentId)?.name || '').trim();
const visible = Boolean(ptz?.enabled && operatorLabel); const visible = Boolean(ptz?.enabled && operatorLabel);
return ( return (
+3 -7
View File
@@ -13,13 +13,9 @@ export function formatUserName(user) {
} }
export function findDriverForRover({ roverId, session }) { export function findDriverForRover({ roverId, session }) {
const activeDriverId = session?.activeDrivers?.[roverId] || null; const turn = session?.roster?.find((rover) => String(rover.id) === String(roverId))?.turn;
if (!activeDriverId) return null; const driver = turn?.queue?.find((entry) => entry.socketId === turn.currentId);
const user = (session?.users || []).find((entry) => entry.socketId === activeDriverId); return driver ? { socketId: driver.socketId, label: driver.name } : null;
return {
socketId: activeDriverId,
label: formatUserName(user || { socketId: activeDriverId }),
};
} }
export function getDisplayBatteryVisual({ rover, frame }) { export function getDisplayBatteryVisual({ rover, frame }) {

Some files were not shown because too many files have changed in this diff Show More