Merge pull request #14 from legop3/ptz

Ptz
This commit is contained in:
legop3
2026-07-13 14:55:09 -04:00
committed by GitHub
69 changed files with 5639 additions and 456 deletions
+2
View File
@@ -5,6 +5,8 @@ create_2_Open_Interface_Spec.txt
logs
node_modules/
__pycache__/
*.py[cod]
.pio
.vscode/
config.h
Binary file not shown.
+130 -1
View File
@@ -39,4 +39,133 @@
- uhhh idk
- obviously, camera on the screen
- probably add a variant of the mobile controls just to retitle the things from the rover controls to the camera controls
- and just use the same control columns
- and just use the same control columns
-- slop generated below --
## clarified implementation direction
This is not intended to become a generic ONVIF camera framework. The camera integration is for one specific Reolink PTZ camera. Once the camera arrives, we will run a one-time ONVIF capability discovery against that exact camera, record what it exposes, and then build the integration around those known capabilities.
The one-time discovery should capture:
- ONVIF services exposed by the camera
- media profiles and stream URIs
- snapshot URI support
- PTZ support and movement modes
- pan/tilt/zoom ranges and speed ranges
- preset/home support
- imaging controls
- any ONVIF-exposed spotlight, IR, or night-vision controls
- whether PTZ status reporting is reliable
After that, runtime code should assume this known camera profile instead of trying to dynamically support every possible ONVIF camera.
## claiming and operator rules
The PTZ camera is a single scarce controllable resource.
Only verified/VIP users can claim it during normal operation. Only one user can operate it at a time. The active operator gets live WebRTC video and PTZ control for a limited turn, probably around five minutes. Other remote users should only receive slow snapshots. Local spectators may be allowed live video because LAN traffic is not the bandwidth problem.
A user operating the PTZ camera must not also be operating a rover. When a user tries to move from a rover to PTZ, the existing rover-switch safety rule should be reused: switching is allowed if another driver remains on that rover, or if the current rover is docked and charging. Otherwise, the server should block the PTZ handoff and tell the user to dock and charge their rover first.
This should be implemented by refactoring the existing rover switching check into a shared helper, such as `canLeaveCurrentRover(socket)`, then using that helper from both rover switching and PTZ claiming.
## streaming model
Camera video should come from the Reolink camera over the local network, likely RTSP into MediaMTX. Browser playback should use the existing MediaMTX WHEP/WebRTC pipeline.
The existing video session and MediaMTX auth system should be extended with a `ptz` source type. Remote live WHEP access should be allowed for the current PTZ operator, local spectators, and authorized admins according to normal server rules. Remote non-operators should not get live video.
Slow snapshots should use a PTZ-specific snapshot path or socket gateway, modeled after the existing room camera snapshot system, but with PTZ-specific authorization rules.
## lockdown behavior
No extra UI work is needed for lockdown because the app already visually blocks things in lockdown mode.
Server-side lockdown enforcement is still required everywhere. In lockdown mode, only lockdown admins/users may claim, queue, operate, subscribe to snapshots, request live PTZ video, or use PTZ replay sources. If lockdown starts while a normal user is operating PTZ, the server should immediately revoke their operator state, remove them from the PTZ queue if needed, revoke PTZ video sessions, and stop accepting PTZ commands from them.
## reusable existing systems
Strong reuse targets:
- rover switch safety logic from `roverManager/roverLifecycle.js`
- `videoSessions`
- `videoSocketService`
- `videoAuthService`
- `WhepPlayer`
- `sessionService` session sync
- VIP panel/card structure
- alert system
- replay source validation and replay worker architecture
Adapted reuse targets:
- turn queue/timer structure from `turnService`
- turn alert listener behavior
- room camera snapshot socket/feed pattern
- `RoomCameraFeed` for slow preview display
- replay source catalog and ffmpeg workers
- existing control input concepts, but with a PTZ-specific command pipeline
Do not directly merge PTZ into `roomCameraService` or `commandService`. PTZ should have its own service boundary because it has ownership, queueing, ONVIF control, video authorization, and camera-specific state.
## operator UI and controls
The VIP tab should get a PTZ camera card. The card should be technical/utilitarian and match the existing site style. It should show:
- current camera operator
- queue/turn state
- turn time remaining when relevant
- whether the current user can claim or must wait
- whether the current user must dock and charge before switching
- a slow snapshot preview
- a button to open the fullscreen PTZ controller when the user is the active operator
The fullscreen PTZ controller should take over the whole app surface while open. It should not feel like a normal side panel. When active, the user is in camera-operation mode, not rover-driving mode.
Desktop controls:
- movement input pans and tilts the camera
- camera up/down or equivalent camera tilt controls zoom in/out
- available special controls expose only what the one-time ONVIF probe proved exists
- if ONVIF exposes presets/home, provide those controls
- if ONVIF exposes spotlight, IR, or night mode, provide those controls
- if those features are not exposed through ONVIF, leave them out until a Reolink-specific fallback is intentionally added
- include a compact right-side status/control panel with operator, queue, camera state, and available controls
Mobile controls:
- reuse the existing mobile control layout concept where practical
- relabel/re-map rover movement controls for PTZ movement
- keep the camera view as the main screen
- use the existing mobile control columns/pads as inspiration, but send PTZ commands instead of rover commands
Input/control implementation:
- do not send PTZ through the existing rover `commandService`
- create PTZ-specific socket events/handlers owned by the PTZ camera service
- use a PTZ-specific client command pipeline that maps existing input intent into PTZ commands
- server must enforce that only the active PTZ operator can send movement/zoom/control commands
- client-side input interception is only for UX; server-side operator checks are the real authority
- all movement controls should send stop commands on key/button release, blur, disconnect, controller close, or turn loss
## NEW UI STUFF
- desktop:
- sidebar like there is now
- replay panel in sidebar
- better indicators of light and OR modes
- list of controls using keybind things
- mobile:
- one sidebar on the right
- reuse rover drive control panel for camera movement
- reuse gpio toggle buttons for spotlight and IR
- reuse camera tilt slider for zoom
- relabeled variants where needed for reused mobile controls
- scroll sidebar down to see replay panel
- both:
- the VIP panel
- sucks.
- wasted space
- put snapshot and everything else side by side
- add a display of ptz's turn queue
- camera should open when you request control over it. no need to have it be another button to press
- should show state of camera
- the fullscreen interface
- should be inside a cardframe, with no title bar
- reuse anything whereever possible
- needs to be ACTUALLY FULLSCREEN, not with space around the edges anywhere
- needs to match the global styling, and use cardframes internally for stuff.
+33
View File
@@ -0,0 +1,33 @@
- make ptz camera better integrated
- keep current fullscreen interface, its good.
- but remove the card from the vip panel
- clean up the fullscreen interface to match the rest of the page better
- have a clear close button
- on desktop, have some stuff in sidebar and some stuff below the video
- video should keep the rest of the space
- reuse rover HUD elements like chat input and not your turn indicator
- probably make it so that the rest of the page unmounts or unloads or whatever when youre in it
- make it feel more like youre switching to a different rover instead of switching to a completely different thing
- simplify and reuse components wherever possible, frontend and backend
- right now, it feels tacked on, badly integrated, and incomplete
- needs a much better UI flow
- still needs to be a VIP feature
- for users on ptz, make their chats have a rover badge that has the ptz name and a color
- make the cam show up as a room camera in the room camera panel
- snapshot mode only
- make the ptz queue and join button show up as one roverqueuespanel style rover row below the links panel
- only show it open for verified users, for non-verified users overlay it with a message and dont let them click on it
- for mobile layouts, show it below the roverqueuepanel.
- dont worry about not being invasive, just dont break anything
## ui flow should be:
1. you are verified
2. you see the ptz camera queue in the ui, it has 2 people in it
3. you click on it, the fullscreen UI opens
- other people will see you in the queue in the little panel
4. its not your turn yet. the fullscreen UI replaces the page.
- you see snapshots, you see the "not your turn" hud, same as driving a rover
5. its now your turn. the overlay shows up just as it does in rover hud
6. you control the camera like usual, you want to close it
7. you hit the close button, you get removed from the queue
8. the page returns to normal
+16
View File
@@ -0,0 +1,16 @@
# make all bandwidth saving options toggleable in one centralized server config
- multitab protection mode
- allowed
- verified only
- not allowed
- snapshots
- non-turn snapshots
- on (you see snapshots when its not your turn)
- off (everyone gets full video all the time)
- non-local spectator snapshots
- on (external spectators are only allowed snapshots)
- off (all spectators get full video)
- external spectator access (new)
- off (no one can access the spectate page externally)
- on (everyone can access the spectate page externally)
- anything else related to bandwidth savings should also get config
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""
Chrome Google TTS WAV renderer.
Purpose: Converts the same local ChromeOS Google TTS assets used by rovers into
server-side WAV files that can be handed to another playback transport.
Scope: This script only renders one utterance to a file; device playback and
camera delivery stay owned by Node services.
"""
import argparse
import ctypes
import os
import struct
import sys
import wave
ASSET_ROOT = "/opt/roverd/googletts"
LIB_PATH = os.path.join(ASSET_ROOT, "libchrometts.so")
VOICE_DIR = os.path.join(ASSET_ROOT, "en-us-x-multi-r30")
PIPELINE = "pipeline.pb"
SAMPLE_RATE = 24000
MAX_TEXT_CHARS = 512
VOICES = {
"sfg": "female",
"iob": "female",
"iog": "female",
"iol": "male",
"iom": "male",
"tpc": "female",
"tpd": "male",
"tpf": "female",
}
DEFAULT_VOICE = "tpf"
DEFAULT_PITCH = 1.0
DEFAULT_SPEED = 1.0
MIN_PITCH = 0.5
MAX_PITCH = 2.0
MIN_SPEED = 0.5
MAX_SPEED = 2.0
def varint(value):
out = bytearray()
while value >= 0x80:
out.append((value & 0x7F) | 0x80)
value >>= 7
out.append(value)
return bytes(out)
def field_bytes(number, payload):
return varint((number << 3) | 2) + varint(len(payload)) + payload
def field_float(number, value):
return varint((number << 3) | 5) + struct.pack("<f", float(value))
def build_utterance(text, pitch=1.0, speed=1.0):
params = field_float(2, pitch) + field_float(3, speed)
msg_b = field_bytes(1, text.encode("utf-8")) + field_bytes(20, params)
msg_a = field_bytes(1, msg_b)
return field_bytes(1, msg_a)
def build_speaker(name, gender):
return field_bytes(1, name.encode("utf-8")) + field_bytes(2, gender.encode("utf-8"))
def clamp_float(value, minimum, maximum, fallback):
try:
value = float(value)
except (TypeError, ValueError):
return fallback
if value <= 0:
return fallback
if value < minimum:
return minimum
if value > maximum:
return maximum
return value
def float_to_s16le(samples):
pcm = bytearray()
for sample in samples:
clipped = max(-1.0, min(1.0, float(sample)))
pcm.extend(struct.pack("<h", int(clipped * 32767)))
return bytes(pcm)
class ChromeTTS:
def __init__(self):
self.lib = ctypes.CDLL(LIB_PATH)
self.lib.GoogleTtsInit.argtypes = [ctypes.c_char_p, ctypes.c_char_p]
self.lib.GoogleTtsInit.restype = ctypes.c_bool
self.lib.GoogleTtsInitBuffered.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int, ctypes.c_int]
self.lib.GoogleTtsInitBuffered.restype = ctypes.c_bool
self.lib.GoogleTtsGetFramesInAudioBuffer.argtypes = []
self.lib.GoogleTtsGetFramesInAudioBuffer.restype = ctypes.c_size_t
self.lib.GoogleTtsReadBuffered.argtypes = [
ctypes.POINTER(ctypes.c_float),
ctypes.POINTER(ctypes.c_size_t),
]
self.lib.GoogleTtsReadBuffered.restype = ctypes.c_int
self.lib.GoogleTtsShutdown.argtypes = []
self.lib.GoogleTtsShutdown.restype = None
voice_dir = os.path.abspath(VOICE_DIR) + os.sep
pipeline = os.path.join(voice_dir, PIPELINE)
if not self.lib.GoogleTtsInit(pipeline.encode("utf-8"), voice_dir.encode("utf-8")):
raise RuntimeError("GoogleTtsInit failed")
self.frames = int(self.lib.GoogleTtsGetFramesInAudioBuffer())
if self.frames <= 0:
raise RuntimeError("invalid Google TTS audio buffer size")
self.buffer = (ctypes.c_float * self.frames)()
def render_wav(self, text, output_path, voice, pitch=DEFAULT_PITCH, speed=DEFAULT_SPEED):
voice = voice if voice in VOICES else DEFAULT_VOICE
pitch = clamp_float(pitch, MIN_PITCH, MAX_PITCH, DEFAULT_PITCH)
speed = clamp_float(speed, MIN_SPEED, MAX_SPEED, DEFAULT_SPEED)
text = text.strip()
if not text:
raise ValueError("text required")
text = text[:MAX_TEXT_CHARS]
utterance = build_utterance(text, pitch=pitch, speed=speed)
speaker = build_speaker(voice, VOICES[voice])
if not self.lib.GoogleTtsInitBuffered(utterance, speaker, len(utterance), len(speaker)):
raise RuntimeError("GoogleTtsInitBuffered failed")
os.makedirs(os.path.dirname(os.path.abspath(output_path)), exist_ok=True)
with wave.open(output_path, "wb") as wav:
wav.setnchannels(1)
wav.setsampwidth(2)
wav.setframerate(SAMPLE_RATE)
frames_written = ctypes.c_size_t(0)
while self.lib.GoogleTtsReadBuffered(self.buffer, ctypes.byref(frames_written)) > 0:
count = int(frames_written.value)
if count > 0:
wav.writeframes(float_to_s16le(self.buffer[:count]))
def shutdown(self):
self.lib.GoogleTtsShutdown()
def main():
parser = argparse.ArgumentParser(description="Render Chrome Google TTS to a WAV file.")
parser.add_argument("--text", required=True)
parser.add_argument("--voice", default=DEFAULT_VOICE)
parser.add_argument("--pitch", type=float, default=DEFAULT_PITCH)
parser.add_argument("--speed", type=float, default=DEFAULT_SPEED)
parser.add_argument("--output", required=True)
args = parser.parse_args()
tts = ChromeTTS()
try:
tts.render_wav(args.text, args.output, args.voice, args.pitch, args.speed)
finally:
tts.shutdown()
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as exc:
sys.stderr.write(f"chromegtts-wav failed: {exc}\n")
raise SystemExit(1)
+16
View File
@@ -128,6 +128,22 @@ roomCameras:
url: "http://192.168.0.51/snapshot.jpg"
streamUrl: "http://192.168.0.51/stream.mjpg"
ptzCamera:
enabled: false
name: "PTZ Camera"
host: "192.168.0.8"
onvifPort: 8000
username: "admin"
password: "REPLACE_WITH_CAMERA_PASSWORD"
# The Reolink TrackMix autotrack profile was token 003 during commissioning.
# Keeping this configurable lets firmware/profile resets be fixed without code
# changes while the integration still remains a single-camera feature.
profileToken: "003"
turnDurationMs: 300000
# PTZ replay capture needs a known-good replay encoder on the server. Keep it
# off by default so adding live PTZ does not start a broken replay worker loop.
replayEnabled: false
kinect:
enabled: false
# Capture requests are global across 3d/color so one person cannot spam room
+1
View File
@@ -26,6 +26,7 @@ require('./src/services/overseerControlService');
require('./src/services/globalObjectiveService');
require('./src/services/serverControlService');
require('./src/services/videoSessions');
require('./src/services/ptzCameraService');
require('./src/services/videoAuthService');
require('./src/services/videoSocketService');
require('./src/services/roomCameraService');
+119
View File
@@ -2,8 +2,12 @@
set -euo pipefail
MEDIAMTX_VERSION="1.15.3"
NEOLINK_VERSION="0.6.2"
MEDIAMTX_BASE_URL="https://github.com/bluenviron/mediamtx/releases/download/v${MEDIAMTX_VERSION}"
NEOLINK_BASE_URL="https://github.com/QuantumEntangledAndy/neolink/releases/download/v${NEOLINK_VERSION}"
MEDIAMTX_BIN="/usr/local/bin/mediamtx"
NEOLINK_BIN="/usr/local/bin/neolink"
CHROMEGTTS_WAV_BIN="/usr/local/bin/chromegtts-wav"
MEDIAMTX_CONF_DIR="/etc/mediamtx"
MEDIAMTX_CONFIG="$MEDIAMTX_CONF_DIR/mediamtx.yml"
ROVER_SNAPSHOT_WRITER_BIN="/usr/local/bin/rover-snapshot-writer.sh"
@@ -29,6 +33,79 @@ SERVER_DIR="$SCRIPT_DIR"
CONFIG_PATH="$SERVER_DIR/config.yaml"
MEDIAMTX_TEMPLATE="$SERVER_DIR/mediamtx/mediamtx.yml"
ROVER_SNAPSHOT_WRITER_TEMPLATE="$SERVER_DIR/mediamtx/rover-snapshot-writer.sh"
CHROMEGTTS_WAV_TEMPLATE="$SERVER_DIR/bin/chromegtts-wav.py"
install_google_tts_assets() {
local asset_dir="/opt/roverd/googletts"
local voice_dir="${asset_dir}/en-us-x-multi-r30"
local dist_url="https://storage.googleapis.com/chromeos-localmirror/distfiles/googletts-26.5.tar.xz"
local lib_member=""
local arch_name
arch_name=$(uname -m)
# The PTZ camera is not a rover, so Google speech must be synthesized on the
# server before neolink sends a WAV to the camera. These assets are the same
# offline ChromeOS local TTS assets that rover installers already use; keeping
# the layout identical lets the server helper and rover daemon share loader
# assumptions.
if [[ -f "${asset_dir}/libchrometts.so" && -f "${voice_dir}/pipeline.pb" ]]; then
echo " Google TTS assets already installed"
return
fi
case "$arch_name" in
x86_64|amd64)
lib_member="libchrometts_x86_64.so"
;;
aarch64)
lib_member="libchrometts_arm64.so"
;;
armv7l|armv6l)
lib_member="libchrometts_armv7.so"
;;
*)
echo "Unsupported Google TTS architecture: $arch_name" >&2
exit 1
;;
esac
echo " Installing Google TTS assets -> $asset_dir"
curl -L -o "$tmpdir/googletts-26.5.tar.xz" "$dist_url"
tar -xf "$tmpdir/googletts-26.5.tar.xz" -C "$tmpdir" en-us-x-multi.zvoice "$lib_member"
install -d -o root -g root -m 0755 "$asset_dir"
install -o root -g root -m 0644 "$tmpdir/$lib_member" "${asset_dir}/libchrometts.so"
rm -rf "$voice_dir"
install -d -o root -g root -m 0755 "$voice_dir"
# The .zvoice member is a zip archive inside the outer tar.xz. Match the
# rover installers here; trying to untar it fails after the large download.
unzip -q "$tmpdir/en-us-x-multi.zvoice" -d "$voice_dir"
chown -R root:root "$asset_dir"
find "$asset_dir" -type d -exec chmod 0755 {} +
find "$asset_dir" -type f -exec chmod 0644 {} +
}
verify_google_tts_helper() {
local smoke_wav="$tmpdir/chromegtts-smoke.wav"
echo " Verifying Chrome Google TTS helper"
# libchrometts is a native ChromeOS library. Rendering one tiny WAV during
# install catches missing shared-library dependencies, bad asset extraction,
# and helper path mistakes before multirover.service starts accepting PTZ TTS
# requests that would fail later in logs.
if ! "$CHROMEGTTS_WAV_BIN" \
--text "test" \
--voice tpf \
--pitch 1 \
--speed 1 \
--output "$smoke_wav"; then
echo "Chrome Google TTS helper smoke render failed." >&2
return 1
fi
if [[ ! -s "$smoke_wav" ]]; then
echo "Chrome Google TTS helper did not create a WAV file." >&2
return 1
fi
}
echo "[1/6] Installing dependencies..."
# The Kinect tooling uses a native libfreenect worker/probe rather than a
@@ -40,9 +117,21 @@ dnf install -y \
npm \
curl \
tar \
unzip \
xz \
gcc-c++ \
make \
pkgconf-pkg-config \
flite \
espeak \
python3 \
libcxx \
libcxxabi \
gstreamer1 \
gstreamer1-plugins-base \
gstreamer1-plugins-good \
gstreamer1-plugins-bad-free \
gstreamer1-rtsp-server \
libfreenect \
libfreenect-devel \
libusb1-devel >/dev/null
@@ -62,6 +151,13 @@ EOF
chmod 644 "$KINECT_UDEV_RULE"
udevadm control --reload-rules
if [[ ! -f "$CHROMEGTTS_WAV_TEMPLATE" ]]; then
echo "Chrome Google TTS WAV helper missing at $CHROMEGTTS_WAV_TEMPLATE" >&2
exit 1
fi
echo " Installing Chrome Google TTS WAV helper -> $CHROMEGTTS_WAV_BIN"
install -m 0755 "$CHROMEGTTS_WAV_TEMPLATE" "$CHROMEGTTS_WAV_BIN"
echo "[2/6] Installing Node production deps..."
runuser -u "$TARGET_USER" -- bash -c "cd '$SERVER_DIR' && npm install --production"
@@ -83,12 +179,15 @@ arch=$(uname -m)
case "$arch" in
x86_64|amd64)
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_amd64.tar.gz"
neolink_pkg="neolink_linux_x86_64_ubuntu.zip"
;;
aarch64)
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_arm64.tar.gz"
neolink_pkg="neolink_linux_arm64.zip"
;;
armv7l)
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_armv7.tar.gz"
neolink_pkg="neolink_linux_armhf.zip"
;;
*)
echo "Unsupported architecture: $arch" >&2
@@ -101,6 +200,26 @@ curl -L "$MEDIAMTX_BASE_URL/$mediamtx_pkg" -o "$tmpdir/mediamtx.tgz"
tar -xzf "$tmpdir/mediamtx.tgz" -C "$tmpdir" mediamtx
install -m 0755 "$tmpdir/mediamtx" "$MEDIAMTX_BIN"
echo " Installing neolink ${NEOLINK_VERSION} -> $NEOLINK_BIN"
curl -L "$NEOLINK_BASE_URL/$neolink_pkg" -o "$tmpdir/neolink.zip"
unzip -q "$tmpdir/neolink.zip" -d "$tmpdir/neolink"
neolink_extracted=$(find "$tmpdir/neolink" -type f -name neolink -perm /111 | head -n 1)
if [[ -z "$neolink_extracted" ]]; then
neolink_extracted=$(find "$tmpdir/neolink" -type f -name neolink | head -n 1)
fi
if [[ -z "$neolink_extracted" ]]; then
echo "neolink binary missing from $neolink_pkg" >&2
exit 1
fi
install -m 0755 "$neolink_extracted" "$NEOLINK_BIN"
install_google_tts_assets
if ! verify_google_tts_helper; then
echo " Reinstalling Google TTS assets after failed verification"
rm -rf /opt/roverd/googletts
install_google_tts_assets
verify_google_tts_helper
fi
mkdir -p "$MEDIAMTX_CONF_DIR"
if [[ ! -f "$MEDIAMTX_TEMPLATE" ]]; then
echo "mediaMTX template missing at $MEDIAMTX_TEMPLATE" >&2
+16 -2
View File
@@ -16,10 +16,24 @@ esac
mkdir -p "$SNAP_DIR"
FILTER="fps=1"
QUALITY="6"
case "$PATH_NAME" in
ptz-camera)
# PTZ snapshots are shown to non-operators specifically to avoid sending the
# full live video stream. The PTZ publisher is full-resolution 16:9 video,
# so resize the JPEGs at the snapshot writer boundary before Node ever reads
# and fans them out over Socket.IO.
FILTER="fps=1,scale=480:-2"
QUALITY="10"
;;
esac
exec ffmpeg -hide_banner -loglevel warning -nostdin -y \
-i "srt://127.0.0.1:9000?streamid=read:${PATH_NAME}" \
-an \
-vf fps=1 \
-q:v 6 \
-vf "$FILTER" \
-q:v "$QUALITY" \
-update 1 \
"${SNAP_DIR}/${PATH_NAME}.jpg"
+2
View File
@@ -19,6 +19,8 @@
"morgan": "^1.10.0",
"obscenity": "^0.4.6",
"ollama": "^0.6.3",
"onvif": "^0.8.1",
"reolink-nvr-api": "^0.3.0",
"sharp": "^0.33.5",
"socket.io": "^4.7.5",
"uuid": "^9.0.1",
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -78,8 +78,8 @@
<script defer src="https://analytics.otter.land/script.js" data-website-id="82dd56a5-db44-4279-bd1e-a4d9fee39af7" data-domains="rover.otter.land"></script>
<script defer src="https://analytics.otter.land/recorder.js" data-website-id="82dd56a5-db44-4279-bd1e-a4d9fee39af7" data-domains="rover.otter.land" data-sample-rate="0.15" data-mask-level="moderate" data-max-duration="300000"></script>
<title>Roomba Rover</title>
<script type="module" crossorigin src="/assets/index-htN8tzuK.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-BfZ_TWsb.css">
<script type="module" crossorigin src="/assets/index-D5vPzVhj.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-BN3kEVFL.css">
</head>
<body>
<div id="root"></div>
+7
View File
@@ -49,6 +49,7 @@ function buildFeatureFlags(config = loadConfig()) {
const barcodeGamesConfig = config.barcodeGames || {};
const socialsConfig = config.socials || {};
const interInstanceConfig = config.interInstance || {};
const ptzCameraConfig = config.ptzCamera || {};
const homeAssistant = Boolean(
asBoolean(homeAssistantConfig.enabled) &&
asTrimmedString(homeAssistantConfig.url) &&
@@ -80,6 +81,12 @@ function buildFeatureFlags(config = loadConfig()) {
),
socials: Boolean(asBoolean(socialsConfig.enabled) && getConfiguredSocials(config).length > 0),
interInstance: asBoolean(interInstanceConfig.enabled),
ptzCamera: Boolean(
asBoolean(ptzCameraConfig.enabled) &&
asTrimmedString(ptzCameraConfig.host) &&
asTrimmedString(ptzCameraConfig.username) &&
asTrimmedString(ptzCameraConfig.password),
),
};
}
@@ -7,8 +7,15 @@ const roverManager = require('../roverManager');
const { getRole } = require('../roleService');
const { describeAssignment } = require('../assignmentService');
const { getNickname } = require('../nicknameService');
const ptzCameraService = require('../ptzCameraService');
function resolvePtzChatTarget(socketId) {
return ptzCameraService.getChatTargetForSocket(socketId) || null;
}
function resolveRoverId(socketId) {
const ptzTarget = resolvePtzChatTarget(socketId);
if (ptzTarget?.roverId) return ptzTarget.roverId;
const primary = roverManager.getPrimaryRoverForSocket(socketId);
if (primary) return primary;
const assignment = describeAssignment(socketId);
@@ -17,13 +24,54 @@ function resolveRoverId(socketId) {
function resolveRoverColor(roverId) {
if (!roverId) return null;
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
return ptzCameraService.getPublicState()?.color || null;
}
const record = roverManager.rovers.get(String(roverId));
return record?.meta?.color || null;
}
function resolveRoverName(roverId) {
if (!roverId) return null;
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
return ptzCameraService.getPublicState()?.name || null;
}
const record = roverManager.rovers.get(String(roverId));
return record?.meta?.name || null;
}
function isPtzChatTargetId(roverId) {
/*
PTZ is intentionally treated as a virtual rover for chat identity only. It
does not live in roverManager.rovers because movement, video authorization,
and queue ownership are PTZ-service concerns, but chat needs one stable
"rover-like" id so the existing web UI, Discord bridge, and AI transcript
code can all render the same badge without learning PTZ internals.
*/
return Boolean(roverId) && String(roverId) === ptzCameraService.PTZ_CAMERA_ID;
}
function isPublicChatTargetId(roverId, socket = null) {
if (!roverId) return false;
/*
Normal rovers remain governed by the existing replay visibility rule, which
is also the rule chat historically used to avoid exposing closed private
rover activity. PTZ gets an explicit allow-list entry here because it is a
public chat target that deliberately pretends to be a rover, even though it
is not a roverManager record.
*/
if (isPtzChatTargetId(roverId)) return true;
return roverManager.canReplayRoverId(roverId, socket) === true;
}
function isPrivateClosedRoverId(roverId) {
if (!roverId) return false;
return roverManager.canReplayRoverId(roverId) !== true;
/*
PTZ uses the existing rover badge fields so chat rows can reuse RoverLabel,
but it is not a private rover. Let PTZ-badged messages broadcast normally
instead of falling into the closed-private rover path for unknown ids.
*/
return !isPublicChatTargetId(roverId);
}
function normalizeProfileImageUrl(value) {
@@ -100,6 +148,7 @@ function buildRoverCtxSnapshot(roverId) {
function buildMessage(socket, text, meta = {}) {
const roverId = meta.roverId || resolveRoverId(socket?.id);
const roverColor = meta.roverColor ?? resolveRoverColor(roverId);
const roverName = meta.roverName ?? resolveRoverName(roverId);
const toolCalls = Array.isArray(meta.toolCalls)
? meta.toolCalls
.map((entry) => {
@@ -122,6 +171,7 @@ function buildMessage(socket, text, meta = {}) {
nickname: meta.nickname || getNickname(socket) || null,
role: meta.role || getRole(socket),
roverId,
roverName,
roverColor,
fromDiscord: Boolean(meta.fromDiscord),
discordGuildId: meta.discordGuildId || null,
@@ -143,6 +193,7 @@ function buildMessage(socket, text, meta = {}) {
function buildTypingPayload(socket, meta = {}) {
const roverId = meta.roverId || resolveRoverId(socket?.id);
const roverColor = meta.roverColor ?? resolveRoverColor(roverId);
const roverName = meta.roverName ?? resolveRoverName(roverId);
const socketId = socket?.id || null;
const fromDiscord = Boolean(meta.fromDiscord);
let typingId = meta.typingId || null;
@@ -165,6 +216,7 @@ function buildTypingPayload(socket, meta = {}) {
nickname: meta.nickname || getNickname(socket) || null,
role: meta.role || getRole(socket),
roverId,
roverName,
roverColor,
fromDiscord,
discordGuildId: meta.discordGuildId || null,
@@ -179,6 +231,8 @@ function buildTypingPayload(socket, meta = {}) {
module.exports = {
resolveRoverId,
isPtzChatTargetId,
isPublicChatTargetId,
isPrivateClosedRoverId,
buildRoverCtxSnapshot,
buildMessage,
@@ -7,6 +7,7 @@ const { getRole } = require('../roleService');
const roverManager = require('../roverManager');
const { issueCommand } = require('../commandService');
const { getAdminReason } = require('../adminReasonService');
const ptzCameraService = require('../ptzCameraService');
const {
TYPING_NOTE_DURATION,
ACCESS_NOTICE_COOLDOWN_MS,
@@ -18,6 +19,13 @@ const { getLastAccessNoticeAt, setLastAccessNoticeAt } = require('./state');
function playTypingNote(roverId, note, socketId) {
if (!roverId) return;
/*
PTZ borrows the roverId field for chat badges, but it has no rover command
channel. Skipping the song command here keeps PTZ chat from producing noisy
"unknown rover" command attempts while still allowing the message itself to
behave like rover chat everywhere else.
*/
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) return;
try {
issueCommand(roverId, {
type: 'song',
@@ -83,6 +91,22 @@ function maybeSendAccessNotice(message, sendSystemMessage) {
function maybeSpeak(socket, message, ttsOptions) {
if (!ttsOptions || !message?.roverId) return;
if (String(message.roverId) === ptzCameraService.PTZ_CAMERA_ID) {
/*
PTZ has no rover websocket, but it does have a real speaker behind the
Reolink/neolink path. Keep PTZ routing here so chat remains the single
place that decides whether a user's message should produce speech, while
ptzCameraService owns camera-specific permissions and playback details.
*/
ptzCameraService.speakText(message.text, ttsOptions, socket)
.then(() => {
logger.info('PTZ TTS sent', { engine: ttsOptions.engine, socket: socket.id });
})
.catch((err) => {
logger.warn('PTZ TTS send failed', { error: err.message, socket: socket.id });
});
return;
}
const record = roverManager.rovers.get(message.roverId);
const ttsEnabled = Boolean(record?.meta?.audio?.ttsEnabled);
if (!ttsEnabled) return;
@@ -2,6 +2,7 @@
// Purpose: Bridges chat and typing between Discord and site sockets.
// Scope: Handles inbound Discord messages plus outbound webhook and typing relay.
const { WebhookClient } = require('discord.js');
const { isPublicChatTargetId } = require('../../chatService/contextBuilders');
function summarizeToolCall(entry = {}) {
const tool = String(entry?.tool || 'unknown');
@@ -23,7 +24,6 @@ function createChatBridgeHandlers(deps) {
const {
logger,
client,
roverManager,
getGuildConfig,
listGuildConfigs,
sendExternalMessage,
@@ -59,7 +59,13 @@ function createChatBridgeHandlers(deps) {
function handleChatBridgeOutbound(event) {
const payload = event?.payload;
if (!payload) return;
if (payload?.roverId && !roverManager.canReplayRoverId(payload.roverId)) return;
/*
Outbound bridge filtering must use chat visibility, not rover replay
visibility. PTZ deliberately uses roverId: "ptz-camera" so the existing
chat badge path can be reused, but that id is not a roverManager rover and
would be dropped by canReplayRoverId().
*/
if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return;
const guildConfigs = listGuildConfigs();
if (!guildConfigs.length) return;
@@ -96,7 +102,11 @@ function createChatBridgeHandlers(deps) {
function handleChatTypingOutbound(event) {
const payload = event?.payload;
if (!payload || payload.fromDiscord) return;
if (payload?.roverId && !roverManager.canReplayRoverId(payload.roverId)) return;
/*
Typing indicators follow the same public-chat-target rule as messages so
PTZ users do not look present in web chat while disappearing from Discord.
*/
if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return;
const guildConfigs = listGuildConfigs();
if (!guildConfigs.length) return;
@@ -24,7 +24,14 @@ function formatWebhookUsername(payload) {
const origin = payload.discordGuildName ? ` (From: ${payload.discordGuildName})` : '';
return `${name}${origin}${botTag}${spectatorTag}${adminTag}`;
}
const roverTag = payload.roverId ? ` [${payload.roverId}]` : '';
/*
The chat payload already carries the resolved display name for rover-like
targets. Prefer that name so PTZ, which is intentionally pretending to be a
rover in chat, shows up as "PTZ Camera" instead of the internal id
"ptz-camera"; fall back to the id for older payloads or missing metadata.
*/
const roverTagLabel = payload.roverName || payload.roverId;
const roverTag = payload.roverId ? ` [${roverTagLabel}]` : '';
return `${name}${botTag}${spectatorTag}${adminTag}${roverTag}`;
}
@@ -7,6 +7,7 @@ const { issueCommand } = require('../commandService');
const homeAssistantService = require('../homeAssistantService');
const neatoService = require('../neatoService');
const liftService = require('../liftService');
const ptzCameraService = require('../ptzCameraService');
const {
HEADLIGHT_DISABLE_ACTION,
LASER_DISABLE_ACTION,
@@ -103,6 +104,17 @@ async function disableAllRoverLasers() {
return { action: 'disableRoverLasers', attempted, failed };
}
async function disablePtzEmitters() {
/*
The PTZ camera has its own light APIs and ownership rules, so the idle
service delegates the actual Reolink calls to ptzCameraService instead of
pretending they are rover commands. This keeps idleService responsible only
for "idle fired; run cleanup actions" and keeps camera-specific payload
details beside the rest of the PTZ integration.
*/
return ptzCameraService.disableEmittersForIdle();
}
async function sendNeatoHome() {
try {
await neatoService.sendHome();
@@ -126,6 +138,7 @@ const idleActions = [
// dockAllRovers,
disableAllRoverHeadlights,
disableAllRoverLasers,
disablePtzEmitters,
sendNeatoHome,
raiseLift,
];
+62 -22
View File
@@ -1,33 +1,63 @@
// Idle Service
// Purpose: Triggers a modular idle action pipeline after a sustained no-driver period.
// Scope: Observes driver activity events and coordinates timer-based idle automation execution.
// Purpose: Triggers a modular idle action pipeline after a sustained no-operator-online period.
// Scope: Observes user/admin socket presence and coordinates timer-based idle automation execution.
const logger = require('../../globals/logger').child('idleService');
const { getActiveDrivers, turnEvents } = require('../turnService');
const roverManager = require('../roverManager');
const { getRecentDriveActivity } = require('../commandService');
const io = require('../../globals/io');
const { getRole, roleEvents } = require('../roleService');
const { IDLE_TIMEOUT_MS } = require('./constants');
const { runtime } = require('./state');
const { runIdleActions } = require('./actions');
function getActivitySnapshot() {
const active = getActiveDrivers();
const turnCount = active && typeof active === 'object' ? Object.keys(active).length : 0;
const activeByTurn = turnCount;
let onlineUsers = 0;
let onlineAdmins = 0;
let onlineSpectators = 0;
let onlineIgnored = 0;
let liveCount = 0;
roverManager.rovers.forEach((record) => {
if (record?.drivers?.size > 0) liveCount += 1;
io.sockets.sockets.forEach((socket) => {
const role = getRole(socket);
/*
Idle automation is about whether a real operator is present, not whether
a browser tab is merely watching. Spectators can leave the room lights,
PTZ emitters, and rovers in their automated idle state because they are
intentionally read-only and cannot be the person still using the setup.
*/
if (role === 'spectator') {
onlineSpectators += 1;
return;
}
/*
Lockdown admins are counted with regular admins because both represent a
person with operator-level access who may be supervising the room without
actively driving a rover. Plain users also count even before they request
control, which is the behavior this service now needs.
*/
if (role === 'admin' || role === 'lockdown') {
onlineAdmins += 1;
return;
}
if (role === 'user') {
onlineUsers += 1;
return;
}
/*
Unknown future roles should not accidentally keep automation disabled.
If a new role should count as an operator, it should be added explicitly
above so this policy remains easy to audit.
*/
onlineIgnored += 1;
});
const activeByRoverDrivers = liveCount;
const recentDriveEvents = getRecentDriveActivity(IDLE_TIMEOUT_MS, { excludeAdmins: false });
const activeByRecentDrive = recentDriveEvents.length;
const totalActive = Math.max(activeByTurn, activeByRoverDrivers, activeByRecentDrive);
const totalActive = onlineUsers + onlineAdmins;
return {
activeByTurn,
activeByRoverDrivers,
activeByRecentDrive,
onlineUsers,
onlineAdmins,
onlineSpectators,
onlineIgnored,
totalActive,
};
}
@@ -53,7 +83,7 @@ function scheduleIdleTimer() {
runtime.deadlineAt = null;
const activity = getActivitySnapshot();
if (activity.totalActive > 0) {
logger.info('Idle automation skipped; active control detected', activity);
logger.info('Idle automation skipped; user or admin online', activity);
return;
}
runtime.lastTriggeredAt = Date.now();
@@ -77,8 +107,18 @@ function refreshIdleState() {
scheduleIdleTimer();
}
turnEvents.on('activeDriver', refreshIdleState);
turnEvents.on('queue', refreshIdleState);
io.on('connection', (socket) => {
/*
A user/admin can be online without ever touching rover controls, so socket
presence has to be a first-class idle signal. The disconnect hook is just as
important: it is what starts the idle timeout after the last non-spectator
leaves, even if no driving event happens around that departure.
*/
refreshIdleState();
socket.on('disconnect', refreshIdleState);
});
roleEvents.on('change', refreshIdleState);
refreshIdleState();
@@ -1,6 +1,8 @@
// llm Commentary Service snapshot engine
// Purpose: Tracks rover activity/history and builds model snapshot payloads from live rover/chat state.
// Scope: Keeps runtime behavior unchanged while isolating sensor aggregation and snapshot assembly logic.
const { isPublicChatTargetId } = require('../chatService/contextBuilders');
function createSnapshotEngine(deps) {
const {
io,
@@ -368,7 +370,12 @@ function createSnapshotEngine(deps) {
.filter((entry) => {
const roverId = entry?.roverId ? String(entry.roverId) : null;
if (!roverId) return true;
return roverManager.canReplayRoverId(roverId);
/*
This is a chat transcript filter, not a physical-rover filter. PTZ
chat intentionally carries a rover-like id so transcript consumers can
render it consistently, even though roverManager cannot replay that id.
*/
return isPublicChatTargetId(roverId);
});
const chatRecent = allRecentMessages
.filter((entry) => !entry?.bot)
@@ -15,6 +15,7 @@ const { getState: getNeatoState, neatoEvents } = neatoService;
const { getState: getLiftState, liftEvents } = liftService;
const roverManager = require('../roverManager');
const { getRecentMessages, sendSystemMessage } = require('../chatService');
const { isPublicChatTargetId } = require('../chatService/contextBuilders');
const { subscribe } = require('../eventBus');
const {
PROMPT_PATH,
@@ -378,7 +379,12 @@ async function runDecision(triggerReason) {
.filter((entry) => Number(entry?.ts || 0) >= runtime.contextResetAt)
.filter((entry) => {
if (!entry?.roverId) return true;
return roverManager.canReplayRoverId(entry.roverId);
/*
Chat context should preserve every public chat target, including the
PTZ virtual rover. Rover replay visibility alone would drop PTZ because
it is owned by ptzCameraService instead of roverManager.
*/
return isPublicChatTargetId(entry.roverId);
})
.slice(-(MAX_CHAT_CONTEXT + MAX_BOT_CONTEXT));
const conversationMessages = buildConversation({ recentMessages: recentConversation, name });
@@ -0,0 +1,324 @@
// PTZ Camera Audio Playback
// Purpose: Generates server-side TTS files and sends them to the Reolink TrackMix speaker through neolink.
// Scope: Owns file/cache/process details for PTZ speech only; PTZ ownership, chat identity, and camera motion stay in index.js.
const crypto = require('crypto');
const fs = require('fs');
const fsp = require('fs/promises');
const path = require('path');
const { spawn } = require('child_process');
const { resolveDataDir } = require('../../helpers/dataPaths');
const DEFAULT_CAMERA_NAME = 'trackmix';
const DEFAULT_MEDIA_PORT = 9000;
const DEFAULT_NEOLINK_BIN = '/usr/local/bin/neolink';
const DEFAULT_CHROMEGTTS_WAV_BIN = '/usr/local/bin/chromegtts-wav';
const DEFAULT_ESPEAK_BIN = 'espeak';
const DEFAULT_FLITE_BIN = 'flite';
const DEFAULT_VOLUME = 0.7;
const MAX_TEXT_CHARS = 512;
const PLAYBACK_TIMEOUT_MS = 45000;
function clampNumber(value, fallback, min, max) {
const number = Number(value);
if (!Number.isFinite(number)) return fallback;
return Math.max(min, Math.min(max, number));
}
function normalizeText(text) {
return String(text || '').replace(/\s+/g, ' ').trim().slice(0, MAX_TEXT_CHARS);
}
function normalizeEngine(engine) {
const value = String(engine || '').trim().toLowerCase();
if (value === 'espeak' || value === 'e') return 'espeak';
if (value === 'flite' || value === 'f') return 'flite';
if (['chromegtts', 'googletts', 'gtts', 'google'].includes(value)) return 'chromegtts';
return 'chromegtts';
}
function tomlString(value) {
/*
The generated neolink config is intentionally tiny, so JSON string escaping
is enough for TOML basic strings and avoids pulling in a TOML writer just to
persist four operator-configured values.
*/
return JSON.stringify(String(value || ''));
}
function cacheKeyFor(text, ttsOptions) {
return crypto
.createHash('sha256')
.update(JSON.stringify({ text, ttsOptions }))
.digest('hex')
.slice(0, 32);
}
function createPtzAudioPlayback(deps) {
const {
logger,
cameraConfig,
enabled,
getSocketLabel,
} = deps;
const audioConfig = cameraConfig.audio || {};
const audioEnabled = audioConfig.enabled === undefined ? Boolean(enabled) : Boolean(audioConfig.enabled);
const dataRoot = path.join(resolveDataDir(), 'ptz-camera-audio');
const cacheDir = path.join(dataRoot, 'tts-cache');
const configPath = path.join(dataRoot, 'neolink-trackmix.toml');
const cameraName = String(audioConfig.neolinkCameraName || DEFAULT_CAMERA_NAME).trim() || DEFAULT_CAMERA_NAME;
const mediaPort = Number(audioConfig.mediaPort) || DEFAULT_MEDIA_PORT;
const neolinkBin = String(audioConfig.neolinkBin || process.env.NEOLINK_BIN || DEFAULT_NEOLINK_BIN).trim();
const chromegttsWavBin = String(
audioConfig.chromegttsWavBin || process.env.CHROMEGTTS_WAV_BIN || DEFAULT_CHROMEGTTS_WAV_BIN,
).trim();
const espeakBin = String(audioConfig.espeakBin || process.env.ESPEAK_BIN || DEFAULT_ESPEAK_BIN).trim();
const fliteBin = String(audioConfig.fliteBin || process.env.FLITE_BIN || DEFAULT_FLITE_BIN).trim();
const volume = clampNumber(audioConfig.volume, DEFAULT_VOLUME, 0, 4);
const fliteDefaultVoice = String(audioConfig.fliteDefaultVoice || 'kal').trim();
let playbackProc = null;
let playbackSeq = 0;
function getState() {
return {
enabled: audioEnabled,
state: playbackProc ? 'playing' : 'idle',
/*
This state is sent to browser sessions through ptzCamera public state.
Keep it operationally useful without leaking server filesystem layout or
binary paths that are only meaningful to the Node process.
*/
cameraName,
volume,
};
}
async function ensureNeolinkConfig() {
await fsp.mkdir(dataRoot, { recursive: true });
const host = String(cameraConfig.host || '').trim();
const username = String(cameraConfig.username || '').trim();
const password = String(cameraConfig.password || '');
if (!host || !username || !password) {
throw new Error('PTZ camera host/username/password required for audio playback');
}
const body = [
'bind = "127.0.0.1"',
'',
'[[cameras]]',
`name = ${tomlString(cameraName)}`,
`username = ${tomlString(username)}`,
`password = ${tomlString(password)}`,
`address = ${tomlString(`${host}:${mediaPort}`)}`,
'stream = "subStream"',
'',
].join('\n');
/*
Write on every playback instead of trying to detect config drift. The file
is small, and this guarantees a camera password/host change in config.yaml
is reflected without an extra migration path or manual cleanup.
*/
await fsp.writeFile(configPath, body, { mode: 0o600 });
return configPath;
}
function spawnChecked(label, command, args, options = {}) {
return new Promise((resolve, reject) => {
const proc = spawn(command, args, {
stdio: ['ignore', 'ignore', 'pipe'],
...options,
});
let stderr = '';
const timer = setTimeout(() => {
try {
proc.kill('SIGKILL');
} catch {
// noop
}
}, options.timeoutMs || PLAYBACK_TIMEOUT_MS);
proc.stderr?.on('data', (chunk) => {
stderr = `${stderr}${String(chunk || '')}`.slice(-4000);
});
proc.on('error', (err) => {
clearTimeout(timer);
reject(new Error(`${label} failed to start: ${err.message}`));
});
proc.on('exit', (code, signal) => {
clearTimeout(timer);
if (code === 0) {
resolve();
return;
}
reject(new Error(`${label} exited code=${code} signal=${signal || 'none'} ${stderr.trim()}`.trim()));
});
});
}
async function renderEspeak(text, ttsOptions, filePath) {
const args = ['-w', filePath];
const pitch = clampNumber(ttsOptions.pitch, 50, 0, 99);
if (pitch > 0) args.push('-p', String(Math.round(pitch)));
args.push(text);
await spawnChecked('espeak', espeakBin, args, { timeoutMs: 20000 });
}
async function renderFlite(text, ttsOptions, filePath) {
const args = ['-o', filePath];
const voice = String(ttsOptions.voice || fliteDefaultVoice || '').trim();
if (voice) args.push('-voice', voice);
args.push('-t', text);
await spawnChecked('flite', fliteBin, args, { timeoutMs: 20000 });
}
async function renderChromeGoogleTts(text, ttsOptions, filePath) {
const args = [
'--text',
text,
'--voice',
String(ttsOptions.voice || 'tpf'),
'--pitch',
String(clampNumber(ttsOptions.pitch, 1, 0.5, 2)),
'--speed',
String(clampNumber(ttsOptions.speed, 1, 0.5, 2)),
'--output',
filePath,
];
await spawnChecked('chromegtts-wav', chromegttsWavBin, args, { timeoutMs: 30000 });
}
async function ensureTtsFile(text, rawOptions = {}) {
const cleanText = normalizeText(text);
if (!cleanText) throw new Error('PTZ TTS text required');
const engine = normalizeEngine(rawOptions.engine);
const ttsOptions = {
engine,
voice: typeof rawOptions.voice === 'string' ? rawOptions.voice.trim() : '',
pitch: Number.isFinite(rawOptions.pitch) ? rawOptions.pitch : undefined,
speed: Number.isFinite(rawOptions.speed) ? rawOptions.speed : undefined,
};
await fsp.mkdir(cacheDir, { recursive: true });
const filePath = path.join(cacheDir, `${cacheKeyFor(cleanText, ttsOptions)}.wav`);
try {
const stat = await fsp.stat(filePath);
if (stat.isFile() && stat.size > 44) return { filePath, engine, cached: true };
} catch (err) {
if (err.code !== 'ENOENT') throw err;
}
const tmpPath = `${filePath}.${process.pid}.${Date.now()}.tmp.wav`;
/*
Every renderer writes a normal WAV file. Neolink/GStreamer handles the
final ADPCM talkback encoding that the Reolink camera expects, so the TTS
renderer stays concerned only with faithfully matching the selected rover
TTS engine's voice options.
*/
if (engine === 'espeak') await renderEspeak(cleanText, ttsOptions, tmpPath);
else if (engine === 'flite') await renderFlite(cleanText, ttsOptions, tmpPath);
else await renderChromeGoogleTts(cleanText, ttsOptions, tmpPath);
await fsp.rename(tmpPath, filePath);
return { filePath, engine, cached: false };
}
function stopActivePlayback(reason = 'replace') {
if (!playbackProc) return;
const proc = playbackProc;
playbackProc = null;
logger.info('Stopping PTZ TTS playback', { reason, pid: proc.pid || null });
try {
proc.kill('SIGTERM');
} catch {
// noop
}
setTimeout(() => {
if (proc.exitCode == null && proc.signalCode == null) {
try {
proc.kill('SIGKILL');
} catch {
// noop
}
}
}, 1200);
}
async function playFile(filePath, context = {}) {
if (!audioEnabled) throw new Error('PTZ audio disabled');
const neolinkConfigPath = await ensureNeolinkConfig();
const stat = await fsp.stat(filePath);
if (!stat.isFile()) throw new Error(`PTZ TTS file is not a regular file: ${filePath}`);
stopActivePlayback('new-playback');
const seq = ++playbackSeq;
const args = [
'talk',
cameraName,
'-c',
neolinkConfigPath,
'--volume',
String(volume),
'--file-path',
filePath,
];
const proc = spawn(neolinkBin, args, { stdio: ['ignore', 'ignore', 'pipe'] });
playbackProc = proc;
let stderr = '';
const timer = setTimeout(() => {
if (playbackProc === proc) stopActivePlayback('timeout');
}, PLAYBACK_TIMEOUT_MS);
proc.stderr?.on('data', (chunk) => {
stderr = `${stderr}${String(chunk || '')}`.slice(-4000);
});
proc.on('error', (err) => {
clearTimeout(timer);
if (playbackProc === proc) playbackProc = null;
logger.warn('PTZ TTS neolink spawn failed', { error: err.message, context });
});
proc.on('exit', (code, signal) => {
clearTimeout(timer);
if (playbackProc === proc) playbackProc = null;
if (code === 0 || signal === 'SIGTERM') {
logger.info('PTZ TTS playback finished', { code, signal, context });
return;
}
logger.warn('PTZ TTS playback failed', {
code,
signal,
stderr: stderr.trim().slice(-1000),
context,
});
});
logger.info('PTZ TTS playback started', {
pid: proc.pid || null,
filePath,
engine: context.engine || null,
actor: context.socketId ? getSocketLabel(context.socketId) : null,
seq,
});
return { pid: proc.pid || null, seq };
}
async function speakText(text, ttsOptions = {}, context = {}) {
const rendered = await ensureTtsFile(text, ttsOptions);
await playFile(rendered.filePath, {
...context,
engine: rendered.engine,
cached: rendered.cached,
});
return rendered;
}
return {
getState,
speakText,
stopActivePlayback,
};
}
module.exports = {
createPtzAudioPlayback,
};
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -49,7 +49,7 @@ const replayBuilder = createReplayBuilder({
ensureDir,
renderSidebarVideo: sidebarRenderer.renderSidebarVideo,
getVideoEntriesForSource: segmentStore.getVideoEntriesForSource,
getAudioEntriesForRover: segmentStore.getAudioEntriesForRover,
getAudioEntriesForSource: segmentStore.getAudioEntriesForSource,
overlapping: segmentStore.overlapping,
});
registerReplaySocketHooks({ tryTriggerReplay, validateSources, getDefaultWebSources });
@@ -51,7 +51,7 @@ function buildChatEventsForWindow(startMs, endMs, limit = 22, preWindowCount = 1
return [...beforeWindow, ...inWindow].sort((a, b) => a.ts - b.ts);
}
function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForRover, overlapping }) {
function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForSource, overlapping }) {
function resolveReplayWindow({ sources = [], nowMs, guardMs, durationMs }) {
const tentativeEnd = nowMs - guardMs;
const sourceEnds = [];
@@ -75,6 +75,33 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-f','concat','-safe','0','-i',listPath,'-c','copy',outPath]);
}
async function pinSegmentFiles(entries, tmpDir, prefix) {
/*
Replay segment files live in a rolling buffer, so cleanup can unlink one
while a slower replay build is still working. Pinning selected files into
the per-build temp directory gives ffmpeg stable paths for the whole build.
A hard link is preferred because it is cheap and keeps the inode alive even
when cleanup removes the original directory entry; copyFile is the fallback
for filesystems that do not support linking across the involved paths.
*/
const pinned = [];
for (let i = 0; i < entries.length; i += 1) {
const entry = entries[i];
const pinnedPath = path.join(tmpDir, `${prefix}-${String(i).padStart(4, '0')}.mp4`);
try {
await fsp.link(entry.filePath, pinnedPath);
} catch (linkErr) {
try {
await fsp.copyFile(entry.filePath, pinnedPath);
} catch (copyErr) {
continue;
}
}
pinned.push({ ...entry, filePath: pinnedPath });
}
return pinned;
}
async function probeMaxFrameSize(paths) {
let maxWidth = 0, maxHeight = 0;
for (const filePath of paths) {
@@ -108,7 +135,11 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
for (let i = 0; i < sources.length; i += 1) {
const source = sources[i];
const sourceId = String(source.id);
const videoEntries = overlapping(getVideoEntriesForSource({ type: String(source.type), id: sourceId }), tStart, tEnd);
const videoEntries = await pinSegmentFiles(
overlapping(getVideoEntriesForSource({ type: String(source.type), id: sourceId }), tStart, tEnd),
tmpDir,
`video-${i}-seg`,
);
if (!videoEntries.length) { missingSources.push({ ...source, reason: 'no video coverage in replay window' }); continue; }
const videoConcat = path.join(tmpDir, `video-${i}.mp4`);
@@ -122,18 +153,26 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
normalizedVideos.push({ path: videoTrimmed, source });
usedSources.push(source);
if (source.type === 'rover') {
const audioEntries = overlapping(getAudioEntriesForRover(sourceId), tStart, tEnd);
if (audioEntries.length) {
const audioConcat = path.join(tmpDir, `audio-${i}.m4a`);
await concatFiles(audioEntries.map((entry) => entry.filePath), audioConcat);
const audioTrimmed = path.join(tmpDir, `audio-${i}.trim.m4a`);
const firstAudioStartMs = audioEntries[0].startMs;
const ass = Math.max(0, (tStart - firstAudioStartMs) / 1000);
const ato = Math.max(ass + 0.1, (tEnd - firstAudioStartMs) / 1000);
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-ss',ass.toFixed(3),'-to',ato.toFixed(3),'-i',audioConcat,'-vn','-ac','1','-ar','48000','-c:a','aac','-b:a','96k',audioTrimmed]);
normalizedAudios.push(audioTrimmed);
}
const audioEntries = await pinSegmentFiles(
overlapping(getAudioEntriesForSource(source), tStart, tEnd),
tmpDir,
`audio-${i}-seg`,
);
if (audioEntries.length) {
/*
Audio workers are separate from selected video sources, even for PTZ
where the live camera path carries inline Opus. Trim the matching
source-owned audio window here and let the final graph mix every
selected source's audio together.
*/
const audioConcat = path.join(tmpDir, `audio-${i}.m4a`);
await concatFiles(audioEntries.map((entry) => entry.filePath), audioConcat);
const audioTrimmed = path.join(tmpDir, `audio-${i}.trim.m4a`);
const firstAudioStartMs = audioEntries[0].startMs;
const ass = Math.max(0, (tStart - firstAudioStartMs) / 1000);
const ato = Math.max(ass + 0.1, (tEnd - firstAudioStartMs) / 1000);
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-ss',ass.toFixed(3),'-to',ato.toFixed(3),'-i',audioConcat,'-vn','-ac','1','-ar','48000','-c:a','aac','-b:a','96k',audioTrimmed]);
normalizedAudios.push(audioTrimmed);
}
}
@@ -3,6 +3,7 @@
// Scope: Handles user-visible replay source catalogs and default source selection rules.
const roverManager = require('../roverManager');
const { getRoomCameras } = require('../roomCameraService');
const ptzCameraService = require('../ptzCameraService');
function getReplaySources(socket = null) {
const roster = socket ? roverManager.getRosterForSocket(socket) : roverManager.getRoster();
@@ -21,7 +22,10 @@ function getReplaySources(socket = null) {
label: camera.name || camera.id,
}));
return [...roverSources, ...roomSources];
const ptzSource = ptzCameraService.getReplaySource();
const ptzSources = ptzSource ? [ptzSource] : [];
return [...roverSources, ...roomSources, ...ptzSources];
}
function normalizeSource(entry) {
@@ -67,11 +71,14 @@ function getDefaultWebSources(assignment = {}, socket = null) {
}
function getDefaultDiscordSources() {
return getRoomCameras().map((camera) => ({
const sources = getRoomCameras().map((camera) => ({
type: 'room',
id: String(camera.id),
label: camera.name || camera.id,
}));
const ptzSource = ptzCameraService.getReplaySource();
if (ptzSource) sources.push(ptzSource);
return sources;
}
module.exports = {
@@ -8,6 +8,7 @@ const logger = require('../../globals/logger').child('replayEngineV2');
const { BUFFER_SECONDS, SEGMENT_SECONDS } = require('./constants');
const { workers, segmentIndex } = require('./state');
const { sourceKey, sourceDirForKey } = require('./sources');
const ptzCameraService = require('../ptzCameraService');
async function ensureDir(dir) {
await fsp.mkdir(dir, { recursive: true });
@@ -96,6 +97,24 @@ function createSegmentStore({ getActiveSegmentRoot }) {
return segmentIndex.get(key) || [];
}
function getAudioEntriesForSource(source) {
/*
Rover audio is published as a separate "<rover>-audio" stream, while PTZ
replay audio is split into an internal "ptz-camera-audio" worker from the
same live camera stream. Keep this mapping close to the segment index so
replayBuilder can ask for "audio that belongs to this selected source"
without knowing every worker naming convention.
*/
const type = String(source?.type || '');
const id = String(source?.id || '');
if (type === 'rover') return getAudioEntriesForRover(id);
if (type === 'ptz') {
const key = sourceKey({ sourceType: 'ptz', kind: 'audio', id: `${id}-audio` });
return segmentIndex.get(key) || [];
}
return [];
}
function overlapping(entries, startMs, endMs) {
return entries.filter((entry) => entry.endMs > startMs && entry.startMs < endMs);
}
@@ -123,6 +142,8 @@ function createSegmentStore({ getActiveSegmentRoot }) {
for (const camera of getRoomCameras()) {
replaySources.push({ type: 'room', id: String(camera.id), label: camera.name || camera.id });
}
const ptzSource = ptzCameraService.getReplaySource();
if (ptzSource) replaySources.push(ptzSource);
for (const source of replaySources) {
const key = sourceKey({ sourceType: source.type, kind: 'video', id: String(source.id) });
@@ -154,6 +175,7 @@ function createSegmentStore({ getActiveSegmentRoot }) {
cleanupOldFiles,
getVideoEntriesForSource,
getAudioEntriesForRover,
getAudioEntriesForSource,
overlapping,
bootstrapIndexFromDisk,
getReplayHealthSnapshot,
@@ -4,6 +4,7 @@
const path = require('path');
const roverManager = require('../roverManager');
const { getRoomCameras } = require('../roomCameraService');
const ptzCameraService = require('../ptzCameraService');
const { FFMPEG_BIN, SEGMENT_SECONDS, TARGET_FPS } = require('./constants');
function sourceKey(source) {
@@ -46,6 +47,10 @@ function listDesiredSources() {
if (!streamUrl) continue;
sources.push({ id: String(camera.id), sourceType: 'room', kind: 'video', label: camera.name || camera.id, inputUrl: streamUrl });
}
// The PTZ service owns its own worker list because video and microphone audio
// both come from the same live MediaMTX path, unlike rovers where audio is a
// separate published stream.
sources.push(...ptzCameraService.getReplayWorkerSources());
return sources;
}
@@ -89,6 +89,7 @@ const {
canDrive,
getRoversForSocket,
getPrimaryRoverForSocket,
canLeaveCurrentRover,
canSwitchRover,
} = roverLifecycle;
@@ -277,6 +278,7 @@ module.exports = {
managerEvents,
getRoversForSocket,
getPrimaryRoverForSocket,
canLeaveCurrentRover,
canSeeRover,
canRequestControl,
applyPrivateDriveSafety,
@@ -137,6 +137,22 @@ function createRoverLifecycle(deps) {
return false;
}
function canLeaveCurrentRover(socket, options = {}) {
const currentId = getPrimaryRoverForSocket(socket.id);
/*
Leaving a rover is only risky when this socket is the last person attached
to an undocked rover. The same rule is used for rover-to-rover switching
and PTZ camera claiming so there is exactly one definition of "do not
abandon a rover in the room".
*/
if (!currentId || currentId === options.targetRoverId) return { ok: true, currentId };
const currentRecord = rovers.get(currentId);
if (!currentRecord) return { ok: true, currentId };
if (hasOtherDrivers(currentRecord, socket.id)) return { ok: true, currentId };
if (isDockedAndCharging(currentRecord)) return { ok: true, currentId };
return { ok: false, currentId, message: 'Dock and charge your current rover before switching.' };
}
function canSwitchRover(socket, targetRoverId, options = {}) {
const target = rovers.get(targetRoverId);
if (!target) return { ok: false, message: 'Unknown rover' };
@@ -145,13 +161,7 @@ function createRoverLifecycle(deps) {
allowClosedPrivateGrantInLockdown: Boolean(options.allowClosedPrivateGrantInLockdown),
});
if (denied) return { ok: false, message: denied };
const currentId = getPrimaryRoverForSocket(socket.id);
if (!currentId || currentId === targetRoverId) return { ok: true, currentId };
const currentRecord = rovers.get(currentId);
if (!currentRecord) return { ok: true, currentId };
if (hasOtherDrivers(currentRecord, socket.id)) return { ok: true, currentId };
if (isDockedAndCharging(currentRecord)) return { ok: true, currentId };
return { ok: false, currentId, message: 'Dock and charge your current rover before switching.' };
return canLeaveCurrentRover(socket, { targetRoverId });
}
function canReplayRoverId(roverId, isPrivateRecord, isPrivateOpen) {
@@ -169,6 +179,7 @@ function createRoverLifecycle(deps) {
canDrive,
getRoversForSocket,
getPrimaryRoverForSocket,
canLeaveCurrentRover,
canSwitchRover,
canReplayRoverId,
};
+27 -2
View File
@@ -10,6 +10,12 @@ const { managerEvents } = roverManager;
const assignmentService = require('../assignmentService');
const { getActiveDrivers, getTurnQueues, turnEvents } = require('../turnService');
const { getRoomCameras, roomCameraEvents } = require('../roomCameraService');
const {
getPublicState: getPtzCameraState,
getChatTargetForSocket: getPtzChatTargetForSocket,
PTZ_CAMERA_ID,
ptzCameraEvents,
} = require('../ptzCameraService');
const { getState: getHomeAssistantState, homeAssistantEvents } = require('../homeAssistantService');
const { getState: getNeatoState, neatoEvents } = require('../neatoService');
const { getState: getLiftState, liftEvents } = require('../liftService');
@@ -61,12 +67,19 @@ function buildUserEntry(socket) {
const role = getRole(socket);
const assignment = assignmentService.describeAssignment(socket.id);
const primaryRover = roverManager.getPrimaryRoverForSocket(socket.id);
const ptzChatTarget = getPtzChatTargetForSocket(socket.id);
return {
socketId: socket.id,
userId: socket?.data?.userId || null,
nickname: getNickname(socket) || null,
role,
roverId: primaryRover || assignment?.roverId || null,
/*
PTZ is not inserted into the physical rover roster, but for chat and user
presence it should read like the user moved to a rover-like target. Prefer
the PTZ chat target while the socket is queued or operating so presence,
queue lookup, and chat identity all agree.
*/
roverId: ptzChatTarget?.roverId || primaryRover || assignment?.roverId || null,
};
}
@@ -78,7 +91,13 @@ function buildSession(socket) {
.filter(Boolean)
.map((entry) => ({
...entry,
roverId: filterVisibleRoverId(socket, entry.roverId),
/*
PTZ is intentionally not a roverManager record, so the normal physical
rover visibility filter would erase the user's PTZ chat target. Preserve
it here because getPtzChatTargetForSocket already applied the PTZ access
and queue/operator rules before buildUserEntry returned it.
*/
roverId: entry.roverId === PTZ_CAMERA_ID ? entry.roverId : filterVisibleRoverId(socket, entry.roverId),
}));
const roster = roverManager.getRosterForSocket(socket);
const assignment = assignmentService.describeAssignment(socket?.id || '');
@@ -107,6 +126,7 @@ function buildSession(socket) {
activeDrivers,
turnQueues,
roomCameras: getRoomCameras(),
ptzCamera: getPtzCameraState(socket),
homeAssistant: getHomeAssistantState(),
neato: getNeatoState(),
lift: getLiftState(),
@@ -281,6 +301,11 @@ roomCameraEvents.on('update', () => {
syncAll();
});
ptzCameraEvents.on('change', () => {
logger.info('PTZ camera state change; syncing all clients');
syncAll();
});
homeAssistantEvents.on('update', () => {
logger.info('Home Assistant state change; syncing all clients');
syncAll();
@@ -10,6 +10,7 @@ const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
const { isVerified } = require('../verificationService');
const turnService = require('../turnService');
const roverManager = require('../roverManager');
const ptzCameraService = require('../ptzCameraService');
const { getRequestIp, getSocketIp, isLocalNetwork } = require('../../helpers/ipResolver');
const { logAdminEvent } = require('../adminLogService');
@@ -26,6 +27,7 @@ const { canAccessStream } = createVideoAuthPolicy({
isVerified,
turnService,
roverManager,
ptzCameraService,
getSocketIp,
isLocalNetwork,
});
@@ -11,6 +11,7 @@ function createVideoAuthPolicy(deps) {
isVerified,
turnService,
roverManager,
ptzCameraService,
getSocketIp,
isLocalNetwork,
} = deps;
@@ -40,6 +41,10 @@ function createVideoAuthPolicy(deps) {
}
}
if (streamInfo.type === 'ptz') {
return ptzCameraService.canRequestLiveVideo(socket);
}
if (sourceType === 'roverMic' && action === 'publish') {
const roverId = streamInfo.baseId || streamInfo.id;
if (!isVerified(socket)) {
@@ -5,6 +5,7 @@ const { loadConfig } = require('../../helpers/configLoader');
const config = loadConfig();
const mediaConfig = config.media || {};
const PTZ_STREAM_PATH = 'ptz-camera';
function getPathPrefix() {
const base = mediaConfig.whepBaseUrl;
@@ -37,6 +38,14 @@ function extractStreamInfo(path) {
const remaining = segments.slice(start, end);
if (remaining.length === 1) {
const rawId = remaining[0] || '';
/*
PTZ is intentionally published as a flat MediaMTX path so WHEP requests
line up with the real stream name. Treat that one reserved path as PTZ
before falling back to the normal one-segment rover parsing rules.
*/
if (rawId === PTZ_STREAM_PATH) {
return { type: 'ptz', id: rawId };
}
if (rawId.endsWith('-fwd')) {
return { type: 'rover', id: rawId, baseId: rawId.slice(0, -4) };
}
@@ -48,6 +57,10 @@ function extractStreamInfo(path) {
return { type: 'room', id: remaining[1] || '' };
}
if (remaining.length === 2 && remaining[0] === 'ptz') {
return { type: 'ptz', id: remaining[1] || '' };
}
return null;
}
@@ -77,6 +90,10 @@ function extractStreamInfoFromBody(body = {}) {
extractSrtStreamId(body.query);
if (!srtId) return null;
if (srtId === PTZ_STREAM_PATH) {
return { type: 'ptz', id: srtId };
}
if (srtId.endsWith('-fwd')) {
return { type: 'rover', id: srtId, baseId: srtId.slice(0, -4) };
}
@@ -7,6 +7,7 @@ const { getMode, MODES } = require('../modeManager');
const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
const videoSessions = require('../videoSessions');
const roverManager = require('../roverManager');
const ptzCameraService = require('../ptzCameraService');
const { loadConfig } = require('../../helpers/configLoader');
const { getSocketIp, isLocalNetwork } = require('../../helpers/ipResolver');
@@ -34,6 +35,15 @@ function buildWhepUrlForSource(source) {
const segments = [];
if (source.type === 'room') {
segments.push('room', encodeURIComponent(source.id));
} else if (source.type === 'ptz') {
/*
MediaMTX exposes WHEP by the exact path name that is being published.
The PTZ ffmpeg publisher registers the single camera as "ptz-camera",
so the browser must request "/video/ptz-camera/whep" instead of a
namespace-like "/video/ptz/ptz-camera/whep" path that MediaMTX has never
seen and correctly returns as 404.
*/
segments.push(encodeURIComponent(source.id));
} else {
segments.push(encodeURIComponent(source.id));
}
@@ -81,6 +91,9 @@ function normalizeRequest(payload = {}) {
if (payload.roomCameraId) {
return { type: 'room', id: String(payload.roomCameraId) };
}
if (payload.ptzCameraId || payload.type === 'ptz') {
return { type: 'ptz', id: String(payload.ptzCameraId || payload.id || ptzCameraService.PTZ_CAMERA_ID) };
}
return null;
}
@@ -109,6 +122,13 @@ io.on('connection', (socket) => {
}
} else if (target.type === 'room') {
throw new Error('Room cameras now use the snapshot feed');
} else if (target.type === 'ptz') {
if (target.id !== ptzCameraService.PTZ_CAMERA_ID) {
throw new Error('Unknown PTZ camera');
}
if (!ptzCameraService.canRequestLiveVideo(socket)) {
throw new Error('Not authorized for PTZ video');
}
} else {
throw new Error('Unsupported video source');
}
+6 -6
View File
@@ -1,12 +1,12 @@
1. improve spectator page, options on what to see and what not to see
2. assign rovers based on battery percentage, give people highest one
3. add prefix config for discord bot to replace rs with something else
4. add config to disable client snapshot forcing, disable bandwidth saving
5. add admin ui for VIP and private requests instead of only through discord
6. make discord bots that can be fine with multiple in one server
7. pull page / tab title from session, use the name from profile of interinstance if enabled, if not just default to old one
3. add config to disable client snapshot forcing, disable bandwidth saving
4. add admin ui for VIP and private requests instead of only through discord
5. pull page / tab title from session, use the name from profile of interinstance if enabled, if not just default to old one
6. make overcurrent limiter speed sensitive, slower fill at lower speeds
7. add feature chat commands, like /neato start, /lift up, etc
8. add more background gap themes
8. fix this:
9. fix this:
`Jun 18 15:14:18 roombaserver.local node[216731]: /home/daniel/MultiRoombaRover/server/src/services/roverManager/socketHandlers.js:92
Jun 18 15:14:18 roombaserver.local node[216731]: cb({ error: err.message });
Jun 18 15:14:18 roombaserver.local node[216731]: ^
+4 -1
View File
@@ -38,6 +38,7 @@ import useUserIdentitySync from './hooks/useUserIdentitySync.js';
import useIncomingInterInstanceTransfer from './hooks/useIncomingInterInstanceTransfer.js';
import GlobalObjectiveBanner from './components/GlobalObjectiveBanner/index.jsx';
import RoverQueuesPanel from './components/RoverQueuesPanel/index.jsx';
import PtzQueueCard from './components/PtzCamera/index.jsx';
import VipPanel from './components/VipPanel/index.jsx';
import { useSessionSelector } from './context/SessionContext.jsx';
import { useTelemetryVisualPolicy } from './context/TelemetryContext.jsx';
@@ -206,7 +207,7 @@ function MobileFeatureTabs({
</div>
</TabPanel>
<TabPanel id="vip" keepMounted>
<VipPanel isActive={activeTab === 'vip'} />
<VipPanel isActive={activeTab === 'vip'} layout={layout} />
</TabPanel>
<TabPanel id="roomcontrols">
<div className={themeStackClass}>
@@ -253,6 +254,7 @@ function MobilePortraitLayout({ onOpenHelpOverlay, swapMobileControlColumns = fa
<ReplaySourcesPanel panelId="replay-sources-mobile-portrait" />
<div className="space-y-0.5">
<RoverQueuesPanel />
<PtzQueueCard layout="mobile-portrait" />
</div>
</div>
{/* <ControlSummary /> */}
@@ -283,6 +285,7 @@ function MobileLandscapeLayout({ onOpenHelpOverlay, swapMobileControlColumns = f
<ReplaySourcesPanel panelId="replay-sources-mobile-landscape" />
<div className="space-y-0.5">
<RoverQueuesPanel />
<PtzQueueCard layout="mobile-landscape" />
</div>
</div>
{/* <TelemetryPanel /> */}
@@ -127,6 +127,7 @@ export function ChatIdentity({ message, toolsToggle = null }) {
{message.roverId && (
<RoverLabel
roverId={message.roverId}
name={message.roverName}
color={message.roverColor}
fallback={message.roverId}
className="shrink-0 text-[0.7rem]"
+28 -4
View File
@@ -49,17 +49,38 @@ function resolveTtsSettings(settings) {
function useChatComposerSessionState(allowSpectatorInput) {
const role = useSessionSelector((state) => state.session?.role || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null);
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
const users = useSessionSelector((state) => state.session?.users ?? []);
const roster = useSessionSelector((state) => state.session?.roster ?? []);
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const chatTargetId = useMemo(() => {
/*
PTZ is intentionally not a roster entry, but session users expose the
current chat target for each socket. Prefer the self user entry so the TTS
control follows PTZ queue/operation state instead of only physical rover
assignment state.
*/
const self = users.find((entry) => entry?.socketId === socketId);
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
return self?.roverId || currentRoverId || null;
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
const rover = useMemo(
() => roster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
[currentRoverId, roster],
() => roster.find((entry) => String(entry.id) === String(chatTargetId)) || null,
[chatTargetId, roster],
);
const ptzTtsSupported = Boolean(
ptz?.id &&
String(chatTargetId) === String(ptz.id) &&
ptz?.audio?.enabled &&
(ptz?.isOperator || ptz?.queuedPosition),
);
return {
canChat: role !== 'spectator' || allowSpectatorInput,
ttsSupported: Boolean(rover?.audio?.ttsEnabled),
ttsSupported: Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported),
};
}
@@ -214,6 +235,7 @@ function TtsControls({
function ChatComposer({
allowSpectatorInput = false,
hideSpectatorNotice = false,
inputTarget = 'panel',
}) {
const {
sendMessage,
@@ -310,7 +332,7 @@ function ChatComposer({
setTypingActive(false);
}
}}
ref={(el) => registerInputRef(el, { target: 'panel' })}
ref={(el) => registerInputRef(el, { target: inputTarget })}
placeholder={canChat ? 'Type a message…' : hideSpectatorNotice ? '' : 'Spectators cannot chat'}
disabled={!canChat}
/>
@@ -346,6 +368,7 @@ export default function ChatPanel({
allowSpectatorInput = false,
title = 'Chat and speech',
minimal = false,
inputTarget = 'panel',
}) {
const effectiveHideInput = minimal || hideInput;
const effectiveTitle = minimal ? '' : title;
@@ -362,6 +385,7 @@ export default function ChatPanel({
<MemoizedChatComposer
allowSpectatorInput={allowSpectatorInput}
hideSpectatorNotice={hideSpectatorNotice}
inputTarget={inputTarget}
/>
)}
</CardFrame>
@@ -24,8 +24,11 @@ function detectSafari() {
function HudChatInput({ compact = false }) {
const role = useSessionSelector((state) => state.session?.role || null);
const socketId = useSessionSelector((state) => state.session?.socketId || null);
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
const users = useSessionSelector((state) => state.session?.users || []);
const roverRoster = useSessionSelector((state) => state.session?.roster || []);
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const { sendMessage, onInputFocus, onInputBlur, blurChat, registerInputRef, setTypingActive } = useChatActions();
const { value: ttsSettings } = useSettingsNamespace('tts', {
// HUD chat shares the normal browser TTS defaults, but it has no visible
@@ -41,11 +44,28 @@ function HudChatInput({ compact = false }) {
const [sending, setSending] = useState(false);
const canChat = role !== 'spectator';
const hideHudChat = role === 'spectator';
const chatTargetId = useMemo(() => {
/*
HUD chat does not render the full composer controls, but it still sends
TTS payloads when the active target supports speech. PTZ appears only in
the session user target, not the physical rover roster, so use the same
self-target resolution as the full chat panel.
*/
const self = users.find((entry) => entry?.socketId === socketId);
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
return self?.roverId || currentRoverId || null;
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
const rover = useMemo(
() => roverRoster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
[currentRoverId, roverRoster],
() => roverRoster.find((entry) => String(entry.id) === String(chatTargetId)) || null,
[chatTargetId, roverRoster],
);
const ttsSupported = Boolean(rover?.audio?.ttsEnabled);
const ptzTtsSupported = Boolean(
ptz?.id &&
String(chatTargetId) === String(ptz.id) &&
ptz?.audio?.enabled &&
(ptz?.isOperator || ptz?.queuedPosition),
);
const ttsSupported = Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported);
const ttsPayload = useMemo(() => {
if (!ttsSupported) return null;
const engine =
@@ -7,9 +7,10 @@ import SocialButton from '../../SocialButton/index.jsx';
function TurnsOverlay({
roverId = null,
mobileHud = false,
turnModel = null,
}) {
const assignedRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
const effectiveRoverId = roverId ?? assignedRoverId;
const effectiveRoverId = turnModel?.targetId ?? roverId ?? assignedRoverId;
const mode = useSessionSelector((state) => state.session?.mode || null);
const roster = useSessionSelector((state) => state.session?.roster ?? []);
const users = useSessionSelector((state) => state.session?.users ?? []);
@@ -29,21 +30,30 @@ function TurnsOverlay({
const subClass = mobileHud ? 'text-xs' : 'text-sm';
const cueTimerClass = mobileHud ? 'text-[0.55rem]' : 'text-[0.75rem]';
const cuePadClass = mobileHud ? 'px-4 py-3' : 'px-6 py-4';
const turnInfo = effectiveRoverId ? turnQueues?.[effectiveRoverId] : null;
const activeDriverId = effectiveRoverId ? activeDrivers?.[effectiveRoverId] : null;
const isActiveDriver = Boolean(socketId && activeDriverId === socketId);
const isTurnsMode = mode === 'turns';
const turnInfo = turnModel ? null : effectiveRoverId ? turnQueues?.[effectiveRoverId] : null;
const activeDriverId = turnModel
? turnModel.activeId || null
: effectiveRoverId
? activeDrivers?.[effectiveRoverId] || null
: null;
const isActiveDriver = turnModel
? Boolean(turnModel.isActive)
: Boolean(socketId && activeDriverId === socketId);
const isTurnsMode = turnModel ? Boolean(turnModel.enabled) : mode === 'turns';
const now = useSharedClock(1000, isTurnsMode);
const nextDriverId = useMemo(() => {
if (turnModel) return turnModel.nextId || null;
const queue = turnInfo?.queue || [];
if (!queue.length || !turnInfo?.current || queue.length <= 1) return null;
const idx = queue.findIndex((id) => id === turnInfo.current);
if (idx === -1) return queue[0] || null;
return queue[(idx + 1) % queue.length] || null;
}, [turnInfo]);
const isNextDriver = Boolean(socketId && nextDriverId === socketId);
const deadline = turnInfo?.deadline || null;
const idleDeadline = turnInfo?.idleDeadline || null;
}, [turnInfo, turnModel]);
const isNextDriver = turnModel
? Boolean(turnModel.isNext)
: Boolean(socketId && nextDriverId === socketId);
const deadline = turnModel ? turnModel.deadline || null : turnInfo?.deadline || null;
const idleDeadline = turnModel ? turnModel.idleDeadline || null : turnInfo?.idleDeadline || null;
const msUntilTurn = deadline ? deadline - now : null;
const msUntilIdleSkip = idleDeadline ? idleDeadline - now : null;
const totalRovers = roster.length;
@@ -59,10 +69,14 @@ function TurnsOverlay({
});
return unique.size;
}, [users]);
const shouldUsePreviewByLoad = isTurnsMode && totalDrivers > totalRovers;
const shouldUsePreviewByLoad = turnModel
? Boolean(turnModel.showPreviewReason)
: isTurnsMode && totalDrivers > totalRovers;
const isPreSwitchWindow =
isTurnsMode && isNextDriver && msUntilTurn != null && msUntilTurn <= 5000 && msUntilTurn > 0;
const showNotTurnNotice = isTurnsMode && !isActiveDriver;
const showNotTurnNotice = turnModel
? Boolean(turnModel.showNotTurnNotice ?? (isTurnsMode && !isActiveDriver))
: isTurnsMode && !isActiveDriver;
const showPreviewReason = showNotTurnNotice && !isPreSwitchWindow && shouldUsePreviewByLoad;
const turnSeconds =
msUntilTurn != null && Number.isFinite(msUntilTurn) ? Math.max(0, Math.ceil(msUntilTurn / 1000)) : null;
@@ -82,19 +96,39 @@ function TurnsOverlay({
const turnTimerFlashActive = noticeFlashActive;
useEffect(() => {
if (mode !== 'turns') {
setShowTurnCue(false);
setTurnCueStartAt(null);
/*
Rover turns and PTZ turns now arrive through the same render path. Use the
normalized isTurnsMode flag here instead of checking the server's rover
mode directly, otherwise PTZ can render the notice but never trigger the
"your turn" cue when camera ownership changes.
*/
if (!isTurnsMode) {
lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null };
return;
/*
React Compiler's lint rules disallow immediate state writes from effect
bodies. Defer the visual reset one macrotask; the ref reset above stays
synchronous so later turn comparisons do not see stale ownership.
*/
const resetTimer = setTimeout(() => {
setShowTurnCue(false);
setTurnCueStartAt(null);
}, 0);
return () => clearTimeout(resetTimer);
}
const lastTurn = lastTurnRef.current;
const nextActiveDriverId = activeDriverId || null;
if (!socketId || !effectiveRoverId) {
setShowTurnCue(false);
setTurnCueStartAt(null);
lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null };
return;
/*
No socket/target means there is no turn identity to compare. Reset the
comparison ref immediately, then defer the visual state reset for the
same React Compiler reason documented in the mode-disabled branch.
*/
const resetTimer = setTimeout(() => {
setShowTurnCue(false);
setTurnCueStartAt(null);
}, 0);
return () => clearTimeout(resetTimer);
}
if (!lastTurn.initialized || lastTurn.roverId !== effectiveRoverId) {
lastTurnRef.current = { initialized: true, roverId: effectiveRoverId, activeDriverId: nextActiveDriverId };
@@ -104,37 +138,76 @@ function TurnsOverlay({
Boolean(lastTurn.activeDriverId) &&
lastTurn.activeDriverId !== socketId &&
nextActiveDriverId === socketId;
let cueTimer = 0;
if (becameActive) {
setShowTurnCue(true);
setTurnCueStartAt(Date.now());
const cueStartAt = Date.now();
/*
The active-turn cue is still caused by this ownership transition, but
React Compiler wants visual state writes scheduled from an async edge.
Capture the timestamp now so the cue dismissal logic compares against
the actual transition time, not the later timer callback time.
*/
cueTimer = setTimeout(() => {
setShowTurnCue(true);
setTurnCueStartAt(cueStartAt);
}, 0);
} else if (nextActiveDriverId !== socketId && showTurnCue) {
setShowTurnCue(false);
setTurnCueStartAt(null);
cueTimer = setTimeout(() => {
setShowTurnCue(false);
setTurnCueStartAt(null);
}, 0);
}
lastTurnRef.current = { initialized: true, roverId: effectiveRoverId, activeDriverId: nextActiveDriverId };
}, [activeDriverId, mode, effectiveRoverId, socketId, showTurnCue]);
return () => {
if (cueTimer) clearTimeout(cueTimer);
};
}, [activeDriverId, effectiveRoverId, isTurnsMode, socketId, showTurnCue]);
useEffect(() => {
if (!showTurnCue || !turnCueStartAt) return;
if (lastControlIntentAt > turnCueStartAt) {
setShowTurnCue(false);
/*
Hide the large "your turn" cue after the first control intent, but
schedule the state write outside the effect body so this shared overlay
remains compatible with the repo's React Compiler lint settings.
*/
const timer = setTimeout(() => setShowTurnCue(false), 0);
return () => clearTimeout(timer);
}
return undefined;
}, [lastControlIntentAt, showTurnCue, turnCueStartAt]);
useEffect(() => {
const lastIntent = Number(lastIntentRef.current) || 0;
const nextIntent = Number(lastControlIntentAt) || 0;
if (nextIntent > lastIntent && showNotTurnNotice) {
setNotTurnFlashAt(Date.now());
const flashAt = Date.now();
/*
The "not your turn" flash is a direct response to a recorded control
intent. Deferring only the state write preserves the timestamp while
satisfying the same effect-state lint rule as the turn cue reset.
*/
const timer = setTimeout(() => setNotTurnFlashAt(flashAt), 0);
lastIntentRef.current = nextIntent;
return () => clearTimeout(timer);
}
lastIntentRef.current = nextIntent;
return undefined;
}, [lastControlIntentAt, showNotTurnNotice]);
useEffect(() => {
if (!showNotTurnNotice || !notTurnFlashAt) return undefined;
setNoticeFlashActive(true);
const timer = setTimeout(() => setNoticeFlashActive(false), 650);
return () => clearTimeout(timer);
/*
The flash has two timed edges: activate on the next task, then clear after
the visible pulse duration. Owning both timers here keeps cleanup local
when the user becomes operator or leaves the PTZ/rover turn surface.
*/
const startTimer = setTimeout(() => setNoticeFlashActive(true), 0);
const endTimer = setTimeout(() => setNoticeFlashActive(false), 650);
return () => {
clearTimeout(startTimer);
clearTimeout(endTimer);
};
}, [showNotTurnNotice, notTurnFlashAt]);
return (
@@ -28,7 +28,7 @@ function getSpeedModeConfig(speedMode) {
return DRIVE_PAD_SPEED_MODES.find((mode) => mode.id === speedMode) || DRIVE_PAD_SPEED_MODES[1];
}
export default function ControlPadPanel({ disabled = false }) {
export default function ControlPadPanel({ compact = false, disabled = false }) {
const rawKeymap = useControlSelector((control) => control.state.keymap);
const { setCameraPrecisionMode, setDriveVector, registerInputState } = useControlActions();
const { value: inputSettings } = useSettingsNamespace('inputs', INPUT_SETTINGS_DEFAULTS);
@@ -163,8 +163,8 @@ export default function ControlPadPanel({ disabled = false }) {
}, [disabled, setCameraPrecisionMode, stopDrivePad]);
return (
<div className="mobile-touch-control flex flex-1 min-h-0 flex-col overflow-hidden rounded-xl border-2 border-slate-700 bg-slate-900 text-slate-100 shadow-md">
<div className="mobile-touch-control grid grid-cols-3 gap-0.5 border-b border-slate-700 bg-slate-950 p-0.5">
<div className={`mobile-touch-control flex flex-1 min-h-0 flex-col overflow-hidden rounded-xl border-2 border-slate-700 bg-slate-900 text-slate-100 shadow-md ${compact ? 'h-full' : ''}`}>
<div className={`mobile-touch-control grid grid-cols-3 gap-0.5 border-b border-slate-700 bg-slate-950 ${compact ? 'p-0.25' : 'p-0.5'}`}>
{DRIVE_PAD_SPEED_MODES.map((mode) => {
const active = speedMode === mode.id;
const speedValue =
@@ -177,7 +177,7 @@ export default function ControlPadPanel({ disabled = false }) {
<button
key={mode.id}
type="button"
className={`mobile-touch-control min-h-9 rounded-md px-1 text-xs font-semibold ${
className={`mobile-touch-control rounded-md px-1 text-xs font-semibold ${compact ? 'min-h-7' : 'min-h-9'} ${
active
? 'bg-cyan-300 text-slate-950'
: 'bg-slate-800 text-slate-200'
@@ -186,7 +186,7 @@ export default function ControlPadPanel({ disabled = false }) {
disabled={disabled}
>
<span className="block leading-tight">{mode.label}</span>
<span className="block font-mono text-[0.7rem] leading-tight">{speedValue}</span>
{!compact ? <span className="block font-mono text-[0.7rem] leading-tight">{speedValue}</span> : null}
</button>
);
})}
@@ -194,6 +194,7 @@ export default function ControlPadPanel({ disabled = false }) {
<div className="mobile-touch-control min-h-0 flex-1">
<FloatingJoystick
activeInputLabel={activeInputLabel}
compact={compact}
disabled={disabled}
onCellChange={handleCellChange}
onStop={() => stopDrivePad('stop')}
@@ -119,7 +119,13 @@ function FloatingPadOverlay({ center, size, activeCellId }) {
);
}
export default function FloatingJoystick({ activeInputLabel = 'stop', disabled, onCellChange, onStop }) {
export default function FloatingJoystick({
activeInputLabel = 'stop',
compact = false,
disabled,
onCellChange,
onStop,
}) {
const containerRef = useRef(null);
const pointerIdRef = useRef(null);
const activePadRef = useRef(null);
@@ -209,7 +215,7 @@ export default function FloatingJoystick({ activeInputLabel = 'stop', disabled,
<div
ref={containerRef}
role="presentation"
className="mobile-touch-control mobile-drag-control relative flex h-full min-h-[10rem] w-full select-none items-center justify-center overflow-hidden text-slate-100"
className={`mobile-touch-control mobile-drag-control relative flex h-full w-full select-none items-center justify-center overflow-hidden text-slate-100 ${compact ? 'min-h-[7rem]' : 'min-h-[10rem]'}`}
// Pointer drags are the whole control model here, so this inline value
// reinforces the utility class even if future class churn changes it.
style={{ touchAction: 'none' }}
@@ -222,17 +228,23 @@ export default function FloatingJoystick({ activeInputLabel = 'stop', disabled,
}}
onContextMenu={(event) => event.preventDefault()}
>
<div className="pointer-events-none absolute inset-x-0 top-0 border-b border-slate-700 bg-slate-950 px-1.5 py-0.5 text-center">
{/* The readout lives inside the pointer target instead of above it, so the
visual indicator does not consume any non-drivable space on small phones. */}
<span className="font-mono text-xs font-semibold text-cyan-200">
{activeInputLabel}
</span>
</div>
<div className="pointer-events-none flex flex-col items-center gap-0.5 px-2 pt-5 text-center">
<span className="text-sm font-semibold text-slate-100">drive pad</span>
<span className="text-xs leading-tight text-slate-300">hold and drag</span>
</div>
{!compact ? (
<div className="pointer-events-none absolute inset-x-0 top-0 border-b border-slate-700 bg-slate-950 px-1.5 py-0.5 text-center">
{/* The readout lives inside the pointer target instead of above it, so the
visual indicator does not consume any non-drivable space on small phones. */}
<span className="font-mono text-xs font-semibold text-cyan-200">
{activeInputLabel}
</span>
</div>
) : null}
{!compact ? (
<div className="pointer-events-none flex flex-col items-center gap-0.5 px-2 pt-5 text-center">
<span className="text-sm font-semibold text-slate-100">drive pad</span>
<span className="text-xs leading-tight text-slate-300">hold and drag</span>
</div>
) : (
<div className="pointer-events-none h-8 w-8 rounded-full border border-cyan-300/60 bg-cyan-300/20" />
)}
</div>
{activePad ? (
<FloatingPadOverlay
+813
View File
@@ -0,0 +1,813 @@
// PTZ Camera UI
// Purpose: Integrates the single PTZ camera into the main rover UI flow as a
// queueable controllable target instead of a VIP-panel card.
// Scope: Owns PTZ entry card and fullscreen composition; PTZ command authority,
// queue ownership, and stream authorization remain server-owned.
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import CardFrame from '../CardFrame/index.jsx';
import ChatPanel from '../ChatPanel/index.jsx';
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx';
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
import PtzLiveVideo, { PTZ_CAMERA_ID } from '../PtzLiveVideo/index.jsx';
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
import QueueTargetRow, { QueueUserChips } from '../QueueTargetRow/index.jsx';
import TurnsOverlay from '../HudOverlays/TurnsOverlay/index.jsx';
import KeyPill from '../vip/VipAudioUploadCard/KeyPill.jsx';
import { useControlActions, useControlSelector } from '../../controls/index.js';
import { formatKeyLabel } from '../../controls/keymapUtils.js';
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
import { useSharedClock } from '../../hooks/useSharedClock.js';
import { isFeatureEnabled } from '../../lib/features.js';
import { trackAnalyticsEvent } from '../../analytics/index.js';
const PTZ_DEFAULT_COLOR = '#38bdf8';
function formatRemaining(deadline, now) {
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - now) / 1000));
if (!remaining) return '--';
const minutes = Math.floor(remaining / 60);
const seconds = remaining % 60;
return `${minutes}:${String(seconds).padStart(2, '0')}`;
}
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeIrMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function nextIrMode(currentMode) {
const current = normalizeIrMode(currentMode);
if (current === 'Auto') return 'On';
if (current === 'On') return 'Off';
return 'Auto';
}
function normalizePtzQueue(ptz = null) {
/*
The PTZ service exposes the current operator separately from the waiting
queue, while the rover queue row expects one ordered queue plus a current id.
Normalizing once keeps every PTZ surface consistent with the shared queue
renderer without making that renderer understand PTZ service internals.
*/
const currentId = ptz?.operatorSocketId || null;
const waiting = Array.isArray(ptz?.queue)
? ptz.queue.map((entry) => entry?.socketId || entry).filter(Boolean)
: [];
const queue = currentId ? [currentId, ...waiting.filter((id) => id !== currentId)] : waiting;
const nextId = currentId ? waiting[0] || null : queue[0] || null;
return { queue, currentId, nextId };
}
function usePtzQueueLookup(ptz = null) {
const users = useSessionSelector((state) => state.session?.users ?? []);
return useCallback(
(socketId) => {
const fromUsers = users.find((entry) => entry.socketId === socketId);
if (fromUsers) return fromUsers;
if (ptz?.operatorSocketId === socketId) {
return { socketId, nickname: ptz?.operatorLabel || null, role: null };
}
const fromQueue = Array.isArray(ptz?.queue)
? ptz.queue.find((entry) => (entry?.socketId || entry) === socketId)
: null;
return {
socketId,
nickname: fromQueue?.label || null,
role: null,
};
},
[ptz, users],
);
}
function PtzSnapshotPreview({ feed, label = 'PTZ Camera', className = 'h-full w-full' }) {
return (
<div className={`relative overflow-hidden bg-black ${className}`}>
{feed?.objectUrl ? (
<img src={feed.objectUrl} alt={label} className="h-full w-full object-contain" />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
)}
{/*
Snapshot mode should look like the regular rover video player: the
camera name belongs to the surrounding card/menu, while the media pane
only exposes stream health in the small top-left diagnostic overlay.
*/}
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
</div>
</div>
</div>
);
}
function StatusRow({ label, value, tone = '' }) {
return (
<div className="flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">{label}</span>
<span className={`min-w-0 truncate font-medium ${tone || 'text-slate-100'}`}>{value}</span>
</div>
);
}
function PtzStatePanel({ ptz, compact = false }) {
const now = useSharedClock(1000, Boolean(ptz?.deadline));
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const publisher = ptz?.publisher || {};
const publisherStatus = publisher.running
? 'running'
: publisher.restartAt
? 'restarting'
: publisher.lastEvent || 'stopped';
const mode = ptz?.isOperator ? 'operator' : ptz?.queuedPosition ? `queued ${ptz.queuedPosition}` : 'spectator';
return (
<CardFrame title="Camera state" bodyClassName="space-y-0.5 p-1 text-sm">
<StatusRow label="Mode" value={mode} tone={ptz?.isOperator ? 'text-emerald-300' : ''} />
<StatusRow label="Operator" value={ptz?.operatorLabel || 'none'} />
<StatusRow label="Remaining" value={formatRemaining(ptz?.deadline, now)} />
<StatusRow label="Spotlight" value={spotlightOn ? 'On' : 'Off'} tone={spotlightOn ? 'text-emerald-300' : 'text-slate-200'} />
<StatusRow label="Infrared mode" value={irMode} />
<StatusRow label="Stream" value={ptz?.status || ptz?.error || 'idle'} tone={ptz?.error ? 'text-amber-300' : ''} />
{!compact ? <StatusRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} /> : null}
{ptz?.blocked?.message ? (
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
{ptz.blocked.message}
</div>
) : null}
</CardFrame>
);
}
function PtzQueueSummary({ ptz, title = 'PTZ queue' }) {
const selfId = useSessionSelector((state) => state.session?.socketId || null);
const lookupUser = usePtzQueueLookup(ptz);
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
return (
<CardFrame title={title} bodyClassName="space-y-0.5 p-1 text-sm">
<QueueUserChips
targetId={ptz?.id || PTZ_CAMERA_ID}
queue={queue}
currentId={currentId}
nextId={nextId}
selfId={selfId}
lookupUser={lookupUser}
/>
</CardFrame>
);
}
function PtzLightingControls({ ptz, disabled = false }) {
const { ptzSpotlight, ptzIr } = useSessionActions();
const [busy, setBusy] = useState('');
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const toggleSpotlight = async (nextOn) => {
if (disabled) return;
setBusy('spotlight');
try {
await ptzSpotlight({ state: nextOn ? 1 : 0 });
} finally {
setBusy('');
}
};
const cycleIr = async () => {
if (disabled) return;
setBusy('ir');
try {
await ptzIr({ state: nextIrMode(irMode) });
} finally {
setBusy('');
}
};
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<GPIOToggleControl
label="Spotlight"
on={spotlightOn}
disabled={disabled || busy === 'spotlight'}
onToggle={toggleSpotlight}
heightClass="min-h-14"
/>
<button
type="button"
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
disabled={disabled || busy === 'ir'}
onClick={cycleIr}
>
<span className="text-sm font-semibold">Infrared</span>
<span className="rounded bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
</button>
</div>
);
}
function PtzMobileZoomButtons({ disabled = false }) {
const { nudgeServo, stopAllMotion } = useControlActions();
const repeatTimerRef = useRef(null);
const stopZoom = useCallback(() => {
/*
Mobile zoom is intentionally routed through the normal camera-up/down
control action instead of emitting PTZ socket commands directly. That
keeps the zoom buttons on the same path as keyboard/gamepad camera tilt,
and the PTZ adapter remains the one place that translates "camera nudge"
into Reolink zoom pulses.
*/
if (repeatTimerRef.current) {
clearInterval(repeatTimerRef.current);
repeatTimerRef.current = null;
}
stopAllMotion();
}, [stopAllMotion]);
const startZoom = useCallback(
(direction) => (event) => {
/*
Send an immediate nudge and then repeat while held. The adapter turns
each nudge into a short zoom pulse, so repeating the standard action is
the simplest way to get continuous hold-to-zoom without adding another
PTZ-specific command loop.
*/
event.preventDefault();
if (disabled) return;
stopZoom();
nudgeServo(direction);
repeatTimerRef.current = setInterval(() => {
nudgeServo(direction);
}, 120);
},
[disabled, nudgeServo, stopZoom],
);
const stopFromPointer = useCallback(
(event) => {
event?.preventDefault?.();
if (disabled) return;
stopZoom();
},
[disabled, stopZoom],
);
useEffect(
() => () => {
/*
A touch surface can unmount during orientation changes or fullscreen
close while a pointer is still down. Clear the repeat timer here so a
held zoom button cannot keep firing camera-up/down actions after the
mobile controls have disappeared.
*/
if (repeatTimerRef.current) {
clearInterval(repeatTimerRef.current);
repeatTimerRef.current = null;
}
},
[],
);
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(-1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onPointerLeave={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom out
</button>
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onPointerLeave={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom in
</button>
</div>
);
}
function PtzMobileControlsPanel({ ptz, disabled = false }) {
return (
<div className="mobile-touch-control space-y-0.5">
<PtzMobileZoomButtons disabled={disabled} />
<div className="mobile-touch-control h-44 min-h-0">
{/*
Reuse the rover control pad so touch intent still enters the normal
control system. The PTZ adapter translates that same drive vector into
pan/tilt commands only while this user is the PTZ operator.
*/}
<ControlPadPanel compact disabled={disabled} />
</div>
<PtzLightingControls ptz={ptz} disabled={disabled} />
</div>
);
}
function keyLabelFor(keymap, actionId) {
return formatKeyLabel(keymap?.[actionId]?.[0]);
}
function PtzControlReference() {
const keymap = useControlSelector((control) => control.state.keymap);
const rows = [
['Tilt up', 'driveForward'],
['Tilt down', 'driveBackward'],
['Pan left', 'driveLeft'],
['Pan right', 'driveRight'],
['Zoom in', 'cameraUp'],
['Zoom out', 'cameraDown'],
['Spotlight', 'headlightToggle'],
['Infrared mode', 'laserToggle'],
];
return (
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
{rows.map(([label, actionId]) => (
<div key={label} className="surface flex items-center justify-between gap-1">
<span className="text-slate-400">{label}</span>
<KeyPill label={keyLabelFor(keymap, actionId)} />
</div>
))}
</CardFrame>
);
}
function PtzPresetPanel({ ptz }) {
const role = useSessionSelector((state) => state.session?.role || null);
const {
ptzListPresets,
ptzGotoPreset,
ptzCreatePreset,
ptzRemovePreset,
} = useSessionActions();
const [name, setName] = useState('');
const [busy, setBusy] = useState('');
const presets = Array.isArray(ptz?.presets) ? ptz.presets : [];
const isPresetAdmin = role === 'admin' || role === 'lockdown';
const canMoveToPreset = Boolean(ptz?.isOperator);
const refreshPresets = async () => {
if (busy) return;
setBusy('refresh');
try {
/*
Presets live on the camera, not in browser state. A manual refresh gives
admins a simple recovery path if another admin or the camera's native
app changes preset storage while this UI is already open.
*/
await ptzListPresets();
} catch (err) {
alert(err.message || 'Failed to refresh PTZ presets.');
} finally {
setBusy('');
}
};
const goToPreset = async (preset) => {
if (!canMoveToPreset || busy || !preset?.token) return;
setBusy(`goto:${preset.token}`);
try {
/*
Moving to a preset is a physical camera move, so the server still checks
that this browser owns the active PTZ turn before accepting the command.
*/
await ptzGotoPreset({ token: preset.token });
} catch (err) {
alert(err.message || 'Failed to move to PTZ preset.');
} finally {
setBusy('');
}
};
const createPreset = async (event) => {
event.preventDefault();
const trimmed = name.trim();
if (!isPresetAdmin || busy || !trimmed) return;
setBusy('create');
try {
/*
ONVIF setPreset stores the camera's current physical position. The UI
only sends the admin's label; the server supplies the active profile
token so browser code does not need to know camera profile internals.
*/
await ptzCreatePreset({ name: trimmed });
setName('');
} catch (err) {
alert(err.message || 'Failed to create PTZ preset.');
} finally {
setBusy('');
}
};
const removePreset = async (preset) => {
if (!isPresetAdmin || busy || !preset?.token) return;
const confirmed = window.confirm(`Remove preset "${preset.name}"?`);
if (!confirmed) return;
setBusy(`remove:${preset.token}`);
try {
/*
The token is the camera's durable preset identifier. Names are only UI
labels and may not be unique, so deletion always targets the token.
*/
await ptzRemovePreset({ token: preset.token });
} catch (err) {
alert(err.message || 'Failed to remove PTZ preset.');
} finally {
setBusy('');
}
};
return (
<CardFrame
title="Position presets"
fillHeight
actions={(
<button type="button" className="button-dark text-xs" disabled={Boolean(busy)} onClick={refreshPresets}>
Refresh
</button>
)}
bodyClassName="flex min-h-0 flex-col gap-1 p-1 text-xs"
>
{ptz?.presetsError ? (
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-amber-100">
{ptz.presetsError}
</div>
) : null}
<div className="min-h-0 flex-1 space-y-0.5 overflow-y-auto">
{presets.length ? presets.map((preset) => {
const gotoBusy = busy === `goto:${preset.token}`;
const removeBusy = busy === `remove:${preset.token}`;
return (
<div key={preset.token} className="surface grid grid-cols-[minmax(0,1fr)_auto] items-center gap-1">
<button
type="button"
className="button-dark min-w-0 truncate text-left text-xs disabled:opacity-50"
disabled={!canMoveToPreset || Boolean(busy)}
onClick={() => goToPreset(preset)}
title={canMoveToPreset ? `Move to ${preset.name}` : 'Your PTZ turn must be active'}
>
{gotoBusy ? 'Moving...' : preset.name}
</button>
{isPresetAdmin ? (
<button
type="button"
className="button-dark text-xs text-rose-200 disabled:opacity-50"
disabled={Boolean(busy)}
onClick={() => removePreset(preset)}
>
{removeBusy ? 'Removing...' : 'Remove'}
</button>
) : null}
</div>
);
}) : (
<div className="rounded border border-slate-700 bg-black/30 p-2 text-center text-slate-400">
No presets saved.
</div>
)}
</div>
{isPresetAdmin ? (
<form className="grid grid-cols-[minmax(0,1fr)_auto] gap-1" onSubmit={createPreset}>
<input
className="min-w-0 rounded border border-slate-700 bg-black px-2 py-1 text-xs text-slate-100 outline-none focus:border-cyan-300"
value={name}
maxLength={60}
disabled={Boolean(busy)}
onChange={(event) => setName(event.target.value)}
placeholder="Preset name"
/>
<button type="submit" className="button-dark text-xs" disabled={Boolean(busy) || !name.trim()}>
{busy === 'create' ? 'Saving...' : 'Save'}
</button>
</form>
) : null}
</CardFrame>
);
}
function buildPtzTurnModel(ptz, selfId) {
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
const isActive = Boolean(ptz?.isOperator);
const isQueued = Boolean(ptz?.queuedPosition);
return {
enabled: Boolean(ptz && (isActive || isQueued || currentId)),
targetId: ptz?.id || PTZ_CAMERA_ID,
activeId: currentId,
nextId,
isActive,
isNext: Boolean(selfId && nextId === selfId),
deadline: ptz?.deadline || null,
idleDeadline: null,
showNotTurnNotice: Boolean(!isActive && (isQueued || currentId || queue.length)),
showPreviewReason: false,
};
}
function PtzMediaPane({ ptz, open, framed = true }) {
const isOperator = Boolean(ptz?.isOperator);
const selfId = useSessionSelector((state) => state.session?.socketId || null);
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !isOperator });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const turnModel = useMemo(() => buildPtzTurnModel(ptz, selfId), [ptz, selfId]);
const media = (
<>
{isOperator ? (
<PtzLiveVideo enabled={open} startMuted={false} />
) : (
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
)}
<TurnsOverlay turnModel={turnModel} />
</>
);
if (!framed) {
return <div className="relative h-full min-h-0 w-full overflow-hidden bg-black">{media}</div>;
}
return (
<div className="flex h-full min-h-0 w-full items-center justify-center overflow-hidden bg-black">
<div className="relative aspect-video max-h-full w-full max-w-full overflow-hidden bg-black">
{media}
</div>
</div>
);
}
function PtzDesktopFullscreen({ ptz, releasePending }) {
return (
<div className="grid h-full min-h-0 grid-rows-[minmax(0,1fr)_minmax(7rem,0.22fr)] gap-0.5 overflow-hidden p-0.5">
<div className="flex min-h-0 min-w-0 gap-0.5 overflow-hidden">
<main className="min-h-0 shrink-0 overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
<PtzMediaPane ptz={ptz} open framed />
</main>
<aside className="flex min-h-0 min-w-56 flex-1 flex-col gap-0.5 overflow-y-auto bg-neutral-950 text-sm">
<PtzQueueSummary ptz={ptz} />
{ptz?.isOperator ? (
<PtzLightingControls ptz={ptz} />
) : (
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
Live PTZ controls unlock when your camera turn is active.
</CardFrame>
)}
<PtzControlReference />
<PtzStatePanel ptz={ptz} />
<ReplaySourcesPanel panelId="ptz-controller-replay" />
</aside>
</div>
<div className="grid min-h-0 grid-cols-[minmax(0,1.6fr)_minmax(16rem,0.7fr)] gap-0.5 overflow-hidden">
<ChatPanel fillHeight title="Chat" allowSpectatorInput inputTarget="overlay" />
<PtzPresetPanel ptz={ptz} />
</div>
{releasePending ? (
<div className="pointer-events-none absolute bottom-1 right-1 rounded bg-black/80 px-2 py-1 text-xs text-slate-200">
Closing...
</div>
) : null}
</div>
);
}
function PtzMobileFullscreen({ ptz, layout, onClose, releasePending = false }) {
const landscape = layout === 'mobile-landscape';
const topHeightClass = landscape ? 'h-full min-h-[calc(100dvh-0.25rem)]' : 'h-[48dvh]';
const topGridClass = landscape
? 'grid-cols-[minmax(0,1fr)_13rem]'
: 'grid-cols-[minmax(0,1fr)_11rem]';
return (
<div className="flex min-h-0 flex-1 flex-col gap-0.5 overflow-y-auto p-0.5">
<section className={`mobile-touch-control grid ${topHeightClass} min-h-48 shrink-0 ${topGridClass} gap-0.5`}>
<main className="relative min-h-0 overflow-hidden bg-black">
<button
type="button"
className="absolute left-1 top-1 z-50 rounded border border-white/40 bg-black/80 px-2 py-1 text-xs font-semibold text-white shadow disabled:opacity-50"
disabled={releasePending}
onClick={onClose}
>
Close
</button>
<PtzMediaPane ptz={ptz} open framed={false} />
</main>
<aside className="min-h-0 overflow-y-auto">
<PtzMobileControlsPanel ptz={ptz} disabled={!ptz?.isOperator} />
</aside>
</section>
<section className="grid gap-0.5 md:grid-cols-[minmax(0,1fr)_minmax(0,0.7fr)]">
<ChatPanel title="Chat" allowSpectatorInput inputTarget="overlay" />
<div className="space-y-0.5">
<PtzQueueSummary ptz={ptz} />
<PtzPresetPanel ptz={ptz} />
<PtzStatePanel ptz={ptz} compact />
<ReplaySourcesPanel panelId="ptz-controller-replay-mobile" />
</div>
</section>
</div>
);
}
export function PtzFullscreenController({ open, onClose, layout = 'desktop' }) {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const { ptzRelease } = useSessionActions();
const { stopAllMotion } = useControlActions();
const [releasePending, setReleasePending] = useState(false);
const isMobile = layout === 'mobile-portrait' || layout === 'mobile-landscape';
const releaseAndClose = useCallback(async () => {
if (releasePending) return;
setReleasePending(true);
try {
/*
Stop first so a held key/pointer cannot leave ONVIF continuous movement
running while the server removes this socket from the PTZ queue.
*/
stopAllMotion?.();
await ptzRelease();
onClose?.();
} finally {
setReleasePending(false);
}
}, [onClose, ptzRelease, releasePending, stopAllMotion]);
if (!open) return null;
const controller = (
/*
The PTZ controller needs to cover the driver page, but it must not become
the top-most application layer. Global fullscreen overlays like help,
quickstart, mode gates, and connection warnings are still part of the
active app state while PTZ is open, so this portal intentionally sits
below their z-30+ overlay stack instead of hiding them.
*/
<div className="fixed inset-0 z-20 h-[100dvh] w-[100vw] overflow-hidden bg-black text-slate-100">
<CardFrame
title={isMobile ? '' : ptz?.name || 'PTZ Camera'}
actions={isMobile ? null : (
<button type="button" className="button-dark text-xs" disabled={releasePending} onClick={releaseAndClose}>
Close
</button>
)}
hideHeader={isMobile}
fillHeight
clipOverflow={false}
className="h-[100dvh] w-[100vw] rounded-none border-0 !bg-black"
bodyClassName="relative min-h-0 flex-1"
>
{isMobile ? (
<PtzMobileFullscreen
ptz={ptz}
layout={layout}
onClose={releaseAndClose}
releasePending={releasePending}
/>
) : (
<PtzDesktopFullscreen ptz={ptz} releasePending={releasePending} />
)}
</CardFrame>
</div>
);
return createPortal(controller, document.body);
}
export default function PtzQueueCard({ layout = 'desktop' }) {
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
const role = useSessionSelector((state) => state.session?.role || null);
const selfId = useSessionSelector((state) => state.session?.socketId || null);
const { ptzClaim, ptzRelease } = useSessionActions();
const lookupUser = usePtzQueueLookup(ptz);
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
const now = useSharedClock(1000, Boolean(ptz?.deadline));
const [controllerOpen, setControllerOpen] = useState(false);
const [pending, setPending] = useState(false);
const canUse = Boolean(ptz?.canUse || isVerified || role === 'admin' || role === 'lockdown');
const isParticipant = Boolean(ptz?.isOperator || ptz?.queuedPosition);
const timerLabel = ptz?.isOperator && ptz?.deadline ? `${formatRemaining(ptz.deadline, now)} left` : '';
if (!featureEnabled) return null;
const handleRequest = async () => {
if (!canUse || pending) return;
if (isParticipant) {
setControllerOpen(true);
return;
}
setPending(true);
trackAnalyticsEvent('ptz_queue_join', { layout });
try {
const response = await ptzClaim();
/*
The server is authoritative for whether the click became an active turn
or a queued wait. Open only after it confirms one of those states so a
dock-guard rejection does not strand the user in fullscreen.
*/
if (response?.state?.isOperator || response?.state?.queuedPosition) {
setControllerOpen(true);
}
trackAnalyticsEvent('ptz_queue_join_result', { layout, status: 'accepted' });
} catch (err) {
trackAnalyticsEvent('ptz_queue_join_result', {
layout,
status: 'failed',
reason: err?.message || 'unknown',
});
alert(err.message || 'PTZ request failed.');
} finally {
setPending(false);
}
};
const handleLeave = async () => {
if (pending) return;
setPending(true);
try {
await ptzRelease();
} catch (err) {
alert(err.message || 'Failed to leave PTZ camera.');
} finally {
setPending(false);
}
};
const actionLabel = pending
? '...'
: ptz?.isOperator
? 'Open'
: ptz?.queuedPosition
? 'Open'
: 'request';
return (
<>
<CardFrame title={ptz?.name || 'PTZ camera'} bodyClassName="relative space-y-0.5 text-sm">
<ul className="space-y-0.5 text-sm">
<QueueTargetRow
target={{
id: ptz?.id || PTZ_CAMERA_ID,
name: ptz?.name || 'PTZ Camera',
color: ptz?.color || PTZ_DEFAULT_COLOR,
// description: ptz?.isOperator
// ? 'Live camera turn active'
// : ptz?.queuedPosition
// ? `Queue position ${ptz.queuedPosition}`
// : 'Pan, tilt, and zoom camera',
}}
queue={queue}
currentId={currentId}
nextId={nextId}
selfId={selfId}
lookupUser={lookupUser}
canClick={canUse && !pending}
pending={pending}
buttonLabel={actionLabel}
batteryLabel={ptz?.isOperator ? 'LIVE' : ptz?.queuedPosition ? `#${ptz.queuedPosition}` : '--'}
batteryClassName={ptz?.isOperator ? 'text-emerald-300' : ptz?.queuedPosition ? 'text-sky-300' : 'text-slate-400'}
timerLabel={timerLabel}
onRequest={handleRequest}
showAction={canUse}
/>
</ul>
{isParticipant ? (
<button type="button" className="button-dark w-full text-xs" disabled={pending} onClick={handleLeave}>
Leave PTZ queue
</button>
) : null}
{!canUse ? (
<div className="absolute inset-0 z-10 flex items-center justify-center rounded bg-black/75 px-2 text-center text-sm font-semibold text-slate-100">
Verify your account to use the PTZ camera.
</div>
) : null}
</CardFrame>
<PtzFullscreenController open={controllerOpen} onClose={() => setControllerOpen(false)} layout={layout} />
</>
);
}
+213
View File
@@ -0,0 +1,213 @@
// PTZ Live Video
// Purpose: Plays the single PTZ camera WHEP stream with the same fresh-session retry loop used by rover video.
// Scope: Owns browser-side WHEP playback/retry only; server authorization and snapshot fallback policy stay outside.
import { useCallback, useEffect, useRef, useState } from 'react';
import { useVideoRequests } from '../../hooks/useVideoRequests.js';
import { WhepPlayer } from '../../lib/whepPlayer.js';
import { RESTART_DELAY_MS } from '../RoverMediaPlayer/constants.js';
export const PTZ_CAMERA_ID = 'ptz-camera';
const PTZ_AUDIO_RETRY_MS = 1000;
const TERMINAL_WHEP_STATES = new Set(['error', 'failed', 'disconnected', 'closed']);
const AUTHORIZATION_ERROR_RE = /not authorized/i;
function isAuthorizationError(error) {
return AUTHORIZATION_ERROR_RE.test(String(error || ''));
}
export default function PtzLiveVideo({
enabled = true,
startMuted = true,
className = 'relative h-full w-full bg-black',
videoClassName = 'h-full w-full object-contain',
statusClassName = 'pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]',
fallback = null,
}) {
const videoRef = useRef(null);
const retryTimerRef = useRef(null);
const playTimerRef = useRef(null);
const enabledRef = useRef(enabled);
const fallbackRef = useRef(false);
const playerGenerationRef = useRef(0);
const [status, setStatus] = useState('idle');
const [detail, setDetail] = useState(null);
const [restartToken, setRestartToken] = useState(0);
const sources = useVideoRequests(
[{ type: 'ptz', id: PTZ_CAMERA_ID, key: PTZ_CAMERA_ID }],
{ enabled, version: restartToken },
);
const source = sources[PTZ_CAMERA_ID] || null;
const shouldUseFallback = Boolean(source?.error && isAuthorizationError(source.error));
useEffect(() => {
enabledRef.current = enabled;
}, [enabled]);
useEffect(() => {
fallbackRef.current = shouldUseFallback;
}, [shouldUseFallback]);
useEffect(() => {
if (enabled && !shouldUseFallback) return undefined;
/*
A retry that was scheduled before the server denied live access should not
keep firing in snapshot mode. Clear it when live playback is no longer the
active display policy.
*/
if (retryTimerRef.current) {
clearTimeout(retryTimerRef.current);
retryTimerRef.current = null;
}
return undefined;
}, [enabled, shouldUseFallback]);
const scheduleRestart = useCallback(() => {
if (!enabledRef.current || fallbackRef.current) return;
/*
WHEP sessions are one-shot browser/server negotiations. When the camera
reboots, the old PeerConnection and token can look alive enough to keep a
black element on screen, but they are not useful anymore. Bumping this
token forces useVideoRequests to ask the server for a new MediaMTX auth
session before creating the next WhepPlayer.
*/
clearTimeout(retryTimerRef.current);
retryTimerRef.current = setTimeout(() => {
retryTimerRef.current = null;
setRestartToken(Date.now());
}, RESTART_DELAY_MS);
}, []);
useEffect(() => {
if (!enabled || shouldUseFallback || !source?.url || !videoRef.current) return undefined;
let active = true;
const generation = playerGenerationRef.current + 1;
playerGenerationRef.current = generation;
const player = new WhepPlayer({
url: source.url,
token: source.token,
video: videoRef.current,
startMuted,
onStatus: (nextStatus, info) => {
/*
Old PeerConnection callbacks can arrive after React has already
cleaned up this effect for a newer token. Only the currently-owned
generation is allowed to update status or schedule another restart.
*/
if (!active || playerGenerationRef.current !== generation) return;
const normalized = String(nextStatus || '').toLowerCase();
setStatus(nextStatus || 'unknown');
setDetail(info || null);
if (TERMINAL_WHEP_STATES.has(normalized)) {
scheduleRestart();
}
},
});
player.start().catch((err) => {
if (!active || playerGenerationRef.current !== generation) return;
setStatus('error');
setDetail(err.message || 'WHEP start failed');
scheduleRestart();
});
return () => {
/*
Mark inactive before stop() because WhepPlayer reports "stopped" during
normal cleanup. Cleanup-driven stops should not immediately schedule the
next retry; only the replacement effect should own the new connection.
*/
active = false;
player.stop();
};
}, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted]);
useEffect(() => {
const video = videoRef.current;
if (!enabled || shouldUseFallback || !source?.url || !video) return undefined;
const generation = playerGenerationRef.current;
const handleEnded = () => {
if (playerGenerationRef.current !== generation) return;
setStatus('stopped');
setDetail('ended');
scheduleRestart();
};
const handleError = () => {
if (playerGenerationRef.current !== generation) return;
setStatus('error');
setDetail(video.error?.message || 'video element error');
scheduleRestart();
};
video.addEventListener('ended', handleEnded);
video.addEventListener('error', handleError);
return () => {
video.removeEventListener('ended', handleEnded);
video.removeEventListener('error', handleError);
};
}, [enabled, scheduleRestart, shouldUseFallback, source?.url]);
useEffect(() => {
const video = videoRef.current;
if (!enabled || shouldUseFallback || startMuted || !source?.url || !video) {
if (playTimerRef.current) clearInterval(playTimerRef.current);
playTimerRef.current = null;
return undefined;
}
/*
PTZ carries inline Opus audio. When the operator opened the camera from a
user gesture, keep retrying audible playback so browser autoplay timing
does not leave the element permanently muted after a reconnect.
*/
const attemptPlay = () => {
const target = videoRef.current;
if (!target) return;
target.muted = false;
if (!target.paused && !target.ended) return;
target.play().catch(() => {});
};
attemptPlay();
playTimerRef.current = setInterval(attemptPlay, PTZ_AUDIO_RETRY_MS);
return () => {
if (playTimerRef.current) clearInterval(playTimerRef.current);
playTimerRef.current = null;
};
}, [enabled, shouldUseFallback, source?.url, startMuted, status]);
useEffect(() => () => {
if (retryTimerRef.current) clearTimeout(retryTimerRef.current);
if (playTimerRef.current) clearInterval(playTimerRef.current);
}, []);
if (shouldUseFallback && typeof fallback === 'function') {
return fallback({ source, status, detail });
}
const displayStatus = source?.error || detail || status;
return (
<div className={className}>
{source?.url ? (
<video ref={videoRef} className={videoClassName} playsInline autoPlay muted={startMuted} />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">
{source?.error || 'Waiting for PTZ video...'}
</div>
)}
{/*
PTZ video uses the same low-profile diagnostic shape as the rover
players: no in-frame camera title, just a compact top-corner status.
This keeps the media pane visually interchangeable with rover streams
while still exposing WHEP/session failures during reconnects.
*/}
<div className={statusClassName}>
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {displayStatus}</span>
</div>
</div>
</div>
);
}
@@ -0,0 +1,196 @@
// Queue Target Row
// Purpose: Renders the shared queue-row visual language used by rover queues and PTZ.
// Scope: Owns row chrome, queue chips, timer labels, and row/button event plumbing;
// callers still own target-specific permission checks and request actions.
import RoverLabel from '../RoverLabel/index.jsx';
function classNames(...values) {
return values.filter(Boolean).join(' ');
}
function roleColors(role) {
switch (role) {
case 'admin':
case 'lockdown':
return 'text-amber-300';
case 'spectator':
return 'text-slate-400';
default:
return 'text-sky-300';
}
}
function formatQueueUserLabel(user, selfId) {
/*
Queue chips need to be readable even when a socket has no nickname yet.
Keeping the socket-prefix fallback here means rover queues and PTZ queues
degrade identically instead of each target inventing its own anonymous label.
*/
if (!user) return '';
const base = user.nickname || user.label || user.socketId?.slice(0, 6) || 'unknown';
if (user.socketId && user.socketId === selfId) {
return `${base} (you)`;
}
return base;
}
export function QueueUserChips({
targetId,
queue = [],
currentId = null,
nextId = null,
selfId = null,
lookupUser,
}) {
if (!queue.length) {
return <p className="text-[0.7rem] text-slate-500">No queue.</p>;
}
return (
<div className="flex flex-wrap items-center gap-0.5">
{queue.map((socketId, idx) => {
const user = lookupUser?.(socketId) || { socketId, nickname: null, role: null };
const isCurrent = socketId === currentId;
const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
/*
These classes intentionally mirror the original rover queue styling.
PTZ feeds the same current/next model into this component, so the user
does not have to learn a different visual vocabulary for camera turns.
*/
const highlightClass = isCurrent
? 'bg-sky-600 text-white ring-2 ring-amber-300'
: isNext
? 'bg-emerald-700/60 text-emerald-100 ring-1 ring-emerald-300/70'
: 'bg-slate-800 text-slate-200';
return (
<span
key={`${targetId}-${socketId}-${idx}`}
className={`flex items-center gap-0.5 rounded px-1 text-[0.7rem] ${highlightClass}`}
>
<span className={`${roleColors(user.role)} font-semibold`}>
{formatQueueUserLabel(user, selfId)}
</span>
{isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>}
{isNext && <span className="text-[0.65rem] text-emerald-100">next</span>}
</span>
);
})}
</div>
);
}
export default function QueueTargetRow({
target,
queue = [],
currentId = null,
nextId = null,
selfId = null,
lookupUser,
canClick = false,
pending = false,
locked = false,
lockedBlocked = false,
privateOpen = false,
buttonLabel = '',
batteryLabel = '',
batteryClassName = 'text-slate-400',
timerLabel = '',
thumbnailUrl = '',
onRequest,
showAction = true,
}) {
const targetId = String(target?.id || '');
const targetLabel = target?.label || target?.name || targetId;
return (
<li
className={classNames(
'surface flex flex-wrap items-start justify-between gap-0.5',
canClick && 'cursor-pointer',
locked
? 'bg-red-900/40'
: privateOpen
? 'bg-amber-700/35 border border-amber-200/30'
: null,
)}
onClick={() => {
/*
The whole row is a large target because queue selection is one of the
main touch/click actions on the page. The caller still decides whether
clicking is currently allowed, so disabled PTZ and locked rover states
cannot accidentally request control through the shared renderer.
*/
if (!canClick) return;
onRequest?.(targetId);
}}
>
{thumbnailUrl ? (
<img
src={thumbnailUrl}
alt=""
className="h-8 w-10 shrink-0 rounded border border-slate-700 bg-black object-cover"
loading="lazy"
/>
) : null}
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
<div className="flex items-center justify-between gap-0.5">
<div className="flex min-w-0 items-center gap-0.5">
<p className="min-w-0 flex items-center gap-0.5 whitespace-nowrap text-slate-200">
<RoverLabel
rover={target?.rover || null}
roverId={target?.roverId ?? targetId}
name={targetLabel}
color={target?.color || null}
fallback={targetId}
/>
{target?.description ? (
<span className="min-w-0 flex-1 truncate text-[0.7rem] text-slate-400">
{target.description}
</span>
) : null}
</p>
{timerLabel ? (
<span className="rounded bg-slate-800 px-1 text-[0.7rem] text-slate-200">
{timerLabel}
</span>
) : null}
</div>
{batteryLabel ? (
<span className={classNames('text-[0.75rem] font-semibold', batteryClassName)}>
{batteryLabel}
</span>
) : null}
</div>
<QueueUserChips
targetId={targetId}
queue={queue}
currentId={currentId}
nextId={nextId}
selfId={selfId}
lookupUser={lookupUser}
/>
</div>
{showAction ? (
<button
type="button"
onClick={(event) => {
/*
Stop propagation so button clicks do not double-fire the row
request. This keeps mouse, touch, and keyboard activation on the
explicit button consistent with clicking the row background.
*/
event.stopPropagation();
onRequest?.(targetId);
}}
disabled={pending || lockedBlocked || !canClick}
className={classNames(
'button-dark disabled:opacity-40',
locked && 'bg-red-600/70 text-white hover:bg-red-600',
)}
>
{buttonLabel}
</button>
) : null}
</li>
);
}
@@ -134,7 +134,9 @@ export default function ReplaySourcesPanel({ panelId = 'replay-sources', fillHei
const grouped = useMemo(() => {
return {
rovers: sources.filter((source) => source.type === 'rover'),
rooms: sources.filter((source) => source.type === 'room'),
// PTZ is presented with room cameras because there is only one fixed room
// PTZ camera and it should not create a separate source category.
rooms: sources.filter((source) => source.type === 'room' || source.type === 'ptz'),
};
}, [sources]);
+6 -2
View File
@@ -9,6 +9,7 @@ import HelpPanel from '../HelpPanel/index.jsx';
import ChatPanel from '../ChatPanel/index.jsx';
import { LinkButtonsPanel } from '../UserListPanel/index.jsx';
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
import PtzQueueCard from '../PtzCamera/index.jsx';
import Tabs, { Tab, TabList, TabPanel, TabPanels } from '../Tabs/index.jsx';
import TopDownMap from '../TopDownMap/index.jsx';
import DriveDockAction from '../DriveDockAction/index.jsx';
@@ -201,7 +202,10 @@ function QueueReplayLinksRow() {
<div className="min-w-0 basis-0 grow-[0.9]">
<ReplaySourcesPanel panelId="replay-sources-desktop" fillHeight />
</div>
<LinkButtonsPanel className="min-w-0 basis-0 grow-[0.75]" />
<div className={`min-w-0 basis-0 grow-[0.75] ${themeStackClass}`}>
<LinkButtonsPanel fillHeight={false} />
<PtzQueueCard layout="desktop" />
</div>
</div>
);
}
@@ -425,7 +429,7 @@ export default function RightPaneTabs({ layout, onOpenHelpOverlay }) {
{/* VIP tab */}
<TabPanel id="vip" keepMounted>
<VipPanel isActive={activeTab === 'vip'} />
<VipPanel isActive={activeTab === 'vip'} layout={layout} />
</TabPanel>
{/* help tab */}
+35 -9
View File
@@ -5,6 +5,7 @@ import { useEffect, useMemo, useRef, useState } from 'react';
import { useSessionSelector } from '../../context/SessionContext.jsx';
import { useSettingsNamespace } from '../../settings/index.js';
import { useRoomCameraSnapshots } from '../../hooks/useRoomCameraSnapshots.js';
import { PTZ_CAMERA_ID, usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
import RoomCameraFeed from '../RoomCameraFeed/index.jsx';
import CardFrame from '../CardFrame/index.jsx';
import { isFeatureEnabled } from '../../lib/features.js';
@@ -105,11 +106,14 @@ function useCameraPanelSubscriptionGate() {
}
export default function RoomCameraPanel(props) {
const enabled = useSessionSelector((state) => isFeatureEnabled(state, 'roomCameras'));
const enabled = useSessionSelector((state) =>
isFeatureEnabled(state, 'roomCameras') || isFeatureEnabled(state, 'ptzCamera'),
);
/*
Room-camera visibility belongs with the room-camera panel. This keeps every
route free to mount the panel without duplicating the server feature rule.
Camera-panel visibility belongs with the panel. PTZ is included here because
the user-facing request is "show it as a room camera"; the rendering path
still uses the same RoomCameraFeed tile as ordinary room cameras.
*/
if (!enabled) return null;
@@ -123,13 +127,35 @@ function RoomCameraPanelContent({
hideHeader = false,
panelId = null,
}) {
const roomCamerasEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'roomCameras'));
const ptzEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const cameras = useSessionSelector((state) => state.session?.roomCameras || []);
const cameraSources = useMemo(() => {
const base = roomCamerasEnabled ? cameras : [];
if (!ptzEnabled || !ptz) return base;
/*
PTZ snapshots use a different socket namespace from room cameras, but the
display model is intentionally the same: an id, a label, and a feed object.
Marking the type lets the subscription layer stay separate while the tile
renderer remains shared.
*/
return [
...base,
{
id: PTZ_CAMERA_ID,
name: ptz.name || 'PTZ Camera',
type: 'ptz',
},
];
}, [cameras, ptz, ptzEnabled, roomCamerasEnabled]);
const cameraIds = useMemo(
() => cameras.map((camera) => camera.id),
[cameras],
() => cameraSources.filter((camera) => camera.type !== 'ptz').map((camera) => camera.id),
[cameraSources],
);
const { panelRef, isPanelVisible } = useCameraPanelSubscriptionGate();
const feedMap = useRoomCameraSnapshots(cameraIds, { enabled: isPanelVisible });
const ptzFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: isPanelVisible && ptzEnabled });
const { value: orientationSettings, save: saveOrientationSettings } = useSettingsNamespace('roomCameraPanels', {});
const [orientation, setOrientation] = useState(() =>
normalizeOrientation(
@@ -152,7 +178,7 @@ function RoomCameraPanelContent({
);
const containerClass =
effectiveOrientation === 'vertical' ? 'flex flex-col gap-0.5' : 'grid gap-0.5 md:grid-cols-2';
const showLayoutToggle = !hideLayoutToggle && !forcedOrientation && cameras.length > 0;
const showLayoutToggle = !hideLayoutToggle && !forcedOrientation && cameraSources.length > 0;
const applyOrientation = (next) => {
setOrientation(next);
if (panelId) {
@@ -160,7 +186,7 @@ function RoomCameraPanelContent({
}
};
if (cameras.length === 0) {
if (cameraSources.length === 0) {
return (
<div ref={panelRef}>
<EmptyState />
@@ -195,8 +221,8 @@ function RoomCameraPanelContent({
bodyClassName="space-y-0.5 text-base"
>
<div className={containerClass}>
{cameras.map((camera) => {
const feed = feedMap[camera.id] || null;
{cameraSources.map((camera) => {
const feed = camera.type === 'ptz' ? ptzFeeds[camera.id] || null : feedMap[camera.id] || null;
return (
<article key={camera.id} className="w-full space-y-0.5 p-0.5">
{/* <header className="space-y-0.5">
+54 -149
View File
@@ -5,17 +5,13 @@ import { useMemo, useState } from 'react';
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
import { useSharedClock } from '../../hooks/useSharedClock.js';
import CardFrame from '../CardFrame/index.jsx';
import RoverLabel from '../RoverLabel/index.jsx';
import QueueTargetRow from '../QueueTargetRow/index.jsx';
import { trackAnalyticsEvent } from '../../analytics/index.js';
import { openExternalRover } from '../../lib/interInstanceTransfer.js';
import { ExternalInstancesCompact } from '../InterInstancePanel/index.jsx';
import { isFeatureEnabled } from '../../lib/features.js';
import { useSettingsNamespace } from '../../settings/index.js';
function classNames(...values) {
return values.filter(Boolean).join(' ');
}
function formatBattery(rover) {
const percent = rover?.batteryState?.percentDisplay;
if (percent == null) return '--';
@@ -29,27 +25,6 @@ function batteryClass(rover) {
return 'text-emerald-300';
}
function roleColors(role) {
switch (role) {
case 'admin':
case 'lockdown':
return 'text-amber-300';
case 'spectator':
return 'text-slate-400';
default:
return 'text-sky-300';
}
}
function formatLabel(user, selfId) {
if (!user) return '';
const base = user.nickname || user.socketId?.slice(0, 6) || 'unknown';
if (user.socketId && user.socketId === selfId) {
return `${base} (you)`;
}
return base;
}
export default function RoverQueuesPanel({
title = 'Rovers',
roster: rosterOverride = null,
@@ -192,129 +167,59 @@ export default function RoverQueuesPanel({
) : (
<ul className="space-y-0.5 text-sm">
{rosterItems.map((rover) => {
const roverId = String(rover.id);
const info = turnQueues?.[roverId] || null;
const queue = info?.queue || [];
const deadline = info?.idleDeadline || info?.deadline || null;
const remainingSeconds =
deadline && deadline > now ? Math.ceil((deadline - now) / 1000) : deadline ? 0 : null;
const currentId = info?.current || null;
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1;
const nextId =
queue.length > 1
? currentIdx >= 0
? queue[(currentIdx + 1) % queue.length]
: queue[0]
: null;
const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId);
const isSelfNext = Boolean(selfId && nextId && nextId === selfId);
const showTimer = remainingSeconds != null && (isSelfCurrent || isSelfNext);
const isPrivateOpen = Boolean(rover?.private?.enabled && rover?.private?.open);
const isGrantedClosedPrivate = Boolean(rover?.private?.enabled && !rover?.private?.open);
const locked = Boolean(rover.locked);
const lockedBlocked = locked && (externalMode || (!adminCapable && !isGrantedClosedPrivate));
const lockLabel = rover.lockReason ? `locked: ${rover.lockReason}` : 'locked';
const buttonLabel = pending[roverId]
? '...'
: lockedBlocked
? lockLabel
: externalMode
? 'Open'
: 'request';
const canClickRow = canRequest && !lockedBlocked && !pending[roverId];
return (
<li
key={rover.id}
className={classNames(
'surface flex flex-wrap items-start justify-between gap-0.5',
canClickRow && 'cursor-pointer',
locked
? 'bg-red-900/40'
: isPrivateOpen
? 'bg-amber-700/35 border border-amber-200/30'
: null,
)}
onClick={() => {
if (!canClickRow) return;
handleRequest(rover.id);
}}
>
{externalMode && rover?.snapshots?.latestUrl ? (
<img
src={rover.snapshots.latestUrl}
alt=""
className="h-8 w-10 shrink-0 rounded border border-slate-700 bg-black object-cover"
loading="lazy"
/>
) : null}
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
<div className="flex items-center justify-between gap-0.5">
<div className="flex min-w-0 items-center gap-0.5">
<p className="min-w-0 flex items-center gap-0.5 whitespace-nowrap text-slate-200">
<RoverLabel rover={rover} fallback={roverId} />
{rover.description ? (
<span className="min-w-0 flex-1 truncate text-[0.7rem] text-slate-400">
{rover.description}
</span>
) : null}
</p>
{showTimer ? (
<span className="rounded bg-slate-800 px-1 text-[0.7rem] text-slate-200">
{isSelfCurrent ? `${remainingSeconds}s left` : `Your turn in ${remainingSeconds}s`}
</span>
) : null}
</div>
<span className={classNames('text-[0.75rem] font-semibold', batteryClass(rover))}>
{formatBattery(rover)}
</span>
</div>
{queue.length === 0 ? (
<p className="text-[0.7rem] text-slate-500">No queue.</p>
) : (
<div className="flex flex-wrap items-center gap-0.5">
{queue.map((socketId, idx) => {
const user = lookupUser(socketId);
const isCurrent = socketId === currentId;
const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
const highlightClass = isCurrent
? 'bg-sky-600 text-white ring-2 ring-amber-300'
: isNext
? 'bg-emerald-700/60 text-emerald-100 ring-1 ring-emerald-300/70'
: 'bg-slate-800 text-slate-200';
return (
<span
key={`${roverId}-${socketId}-${idx}`}
className={`flex items-center gap-0.5 rounded px-1 text-[0.7rem] ${highlightClass}`}
>
<span className={`${roleColors(user.role)} font-semibold`}>
{formatLabel(user, selfId)}
</span>
{isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>}
{isNext && <span className="text-[0.65rem] text-emerald-100">next</span>}
</span>
);
})}
</div>
)}
</div>
{canRequest ? (
<button
type="button"
onClick={(event) => {
event.stopPropagation();
handleRequest(rover.id);
}}
disabled={pending[roverId] || lockedBlocked}
className={classNames(
'button-dark disabled:opacity-40',
locked && 'bg-red-600/70 text-white hover:bg-red-600',
)}
>
{buttonLabel}
</button>
) : null}
</li>
);
const roverId = String(rover.id);
const info = turnQueues?.[roverId] || null;
const queue = info?.queue || [];
const deadline = info?.idleDeadline || info?.deadline || null;
const remainingSeconds =
deadline && deadline > now ? Math.ceil((deadline - now) / 1000) : deadline ? 0 : null;
const currentId = info?.current || null;
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1;
const nextId =
queue.length > 1
? currentIdx >= 0
? queue[(currentIdx + 1) % queue.length]
: queue[0]
: null;
const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId);
const isSelfNext = Boolean(selfId && nextId && nextId === selfId);
const showTimer = remainingSeconds != null && (isSelfCurrent || isSelfNext);
const isPrivateOpen = Boolean(rover?.private?.enabled && rover?.private?.open);
const isGrantedClosedPrivate = Boolean(rover?.private?.enabled && !rover?.private?.open);
const locked = Boolean(rover.locked);
const lockedBlocked = locked && (externalMode || (!adminCapable && !isGrantedClosedPrivate));
const lockLabel = rover.lockReason ? `locked: ${rover.lockReason}` : 'locked';
const buttonLabel = pending[roverId]
? '...'
: lockedBlocked
? lockLabel
: externalMode
? 'Open'
: 'request';
const canClickRow = canRequest && !lockedBlocked && !pending[roverId];
return (
<QueueTargetRow
key={rover.id}
target={{ ...rover, rover, roverId, id: roverId }}
queue={queue}
currentId={currentId}
nextId={nextId}
selfId={selfId}
lookupUser={lookupUser}
canClick={canClickRow}
pending={Boolean(pending[roverId])}
locked={locked}
lockedBlocked={lockedBlocked}
privateOpen={isPrivateOpen}
buttonLabel={buttonLabel}
batteryLabel={formatBattery(rover)}
batteryClassName={batteryClass(rover)}
timerLabel={showTimer ? (isSelfCurrent ? `${remainingSeconds}s left` : `Your turn in ${remainingSeconds}s`) : ''}
thumbnailUrl={externalMode ? rover?.snapshots?.latestUrl : ''}
onRequest={handleRequest}
showAction={Boolean(canRequest)}
/>
);
})}
</ul>
)}
+2 -2
View File
@@ -19,7 +19,7 @@ export function NicknameEntryPanel({ compact = false }) {
);
}
export function LinkButtonsPanel({ className = '' }) {
export function LinkButtonsPanel({ className = '', fillHeight = true }) {
const enabled = useSessionSelector((state) => isFeatureEnabled(state, 'socials'));
/*
@@ -32,7 +32,7 @@ export function LinkButtonsPanel({ className = '' }) {
return (
<CardFrame
title="Links!"
fillHeight
fillHeight={fillHeight}
className={className}
bodyClassName="flex flex-1 min-h-0 flex-col gap-0.5 text-base"
>
@@ -0,0 +1,574 @@
// Vip PTZ Camera Card
// Purpose: Provides the verified-user entry point and fullscreen controller for the single Reolink PTZ camera.
// Scope: Owns PTZ UI state only; server-side PTZ ownership, rover handoff, and command authorization remain authoritative.
import { useCallback, useMemo, useState } from 'react';
import { createPortal } from 'react-dom';
import ChatPanel from '../ChatPanel/index.jsx';
import CardFrame from '../CardFrame/index.jsx';
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx';
import PtzLiveVideo from '../PtzLiveVideo/index.jsx';
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
import KeyPill from './VipAudioUploadCard/KeyPill.jsx';
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
import { useControlSelector } from '../../controls/index.js';
import { formatKeyLabel } from '../../controls/keymapUtils.js';
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
import { isFeatureEnabled } from '../../lib/features.js';
const PTZ_CAMERA_ID = 'ptz-camera';
const PTZ_ZOOM_SPEED = 0.55;
function formatRemaining(deadline) {
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - Date.now()) / 1000));
if (!remaining) return '--';
const minutes = Math.floor(remaining / 60);
const seconds = remaining % 60;
return `${minutes}:${String(seconds).padStart(2, '0')}`;
}
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeIrMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function nextIrMode(currentMode) {
const current = normalizeIrMode(currentMode);
if (current === 'Auto') return 'On';
if (current === 'On') return 'Off';
return 'Auto';
}
function PtzSnapshotPreview({ feed, label = 'PTZ Camera' }) {
return (
<div className="relative w-full overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
{feed?.objectUrl ? (
<img src={feed.objectUrl} alt={label} className="h-full w-full object-cover" />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
)}
{/*
Even on this legacy VIP surface, keep the video pane itself identical
to rover media panes: no camera title inside the frame, only the small
top-left stream status. Any PTZ name/context belongs to the card chrome
around the media, not the media player.
*/}
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
</div>
</div>
</div>
);
}
function StatusRow({ label, value, tone = '' }) {
return (
<div className="flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">{label}</span>
<span className={`min-w-0 truncate font-medium ${tone || 'text-slate-100'}`}>{value}</span>
</div>
);
}
function formatPublisherProgress(progress = null) {
/*
ffmpeg progress is intentionally shown as raw operational numbers instead
of translated prose. When the PTZ video feels delayed, fps/speed/drop/out
time make it obvious whether ffmpeg itself is keeping up or the delay is
somewhere before/after the transcoder.
*/
if (!progress) return null;
return [
progress.fps ? `fps ${progress.fps}` : null,
progress.speed ? `speed ${progress.speed}` : null,
progress.drop_frames ? `drop ${progress.drop_frames}` : null,
progress.out_time ? `out ${progress.out_time}` : null,
].filter(Boolean).join(' | ');
}
function PtzQueueList({ queue = [], operatorLabel = '' }) {
const hasQueue = Array.isArray(queue) && queue.length > 0;
return (
<div className="space-y-0.5">
<div className="panel-muted text-xs">Turn queue</div>
<div className="space-y-0.5">
{operatorLabel ? (
<div className="surface flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">Now</span>
<span className="min-w-0 truncate text-emerald-200">{operatorLabel}</span>
</div>
) : null}
{hasQueue ? queue.map((entry, index) => (
<div key={entry.socketId || `${entry.label}-${index}`} className="surface flex items-center justify-between gap-1 text-xs">
<span className="text-slate-400">{index + 1}</span>
<span className="min-w-0 truncate text-slate-100">{entry.label || entry.socketId || 'queued user'}</span>
</div>
)) : (
<div className="surface text-xs text-slate-400">No one waiting</div>
)}
</div>
</div>
);
}
function PtzStatePanel({ ptz, onClose, onRelease, releaseDisabled = false }) {
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const publisher = ptz?.publisher || {};
const publisherStatus = publisher.running
? `running${publisher.pid ? ` ${publisher.pid}` : ''}`
: publisher.restartAt
? 'restarting'
: publisher.lastEvent || 'stopped';
const publisherProgress = formatPublisherProgress(publisher.progress);
const statusTone = ptz?.error ? 'text-amber-300' : ptz?.isOperator ? 'text-emerald-300' : 'text-slate-100';
return (
<CardFrame
title="Camera state"
actions={onClose ? <button type="button" className="button-dark text-xs" onClick={onClose}>Close</button> : null}
bodyClassName="space-y-0.5 p-1 text-sm"
>
<StatusRow label="Mode" value={ptz?.isOperator ? 'operator' : ptz?.queuedPosition ? `queued ${ptz.queuedPosition}` : 'spectator'} tone={statusTone} />
<StatusRow label="Operator" value={ptz?.operatorLabel || 'none'} />
<StatusRow label="Remaining" value={formatRemaining(ptz?.deadline)} />
<StatusRow label="Spotlight" value={spotlightOn ? 'On' : 'Off'} tone={spotlightOn ? 'text-emerald-300' : 'text-slate-200'} />
<StatusRow label="Infrared mode" value={irMode} />
<StatusRow label="Stream" value={ptz?.status || ptz?.error || 'idle'} tone={ptz?.error ? 'text-amber-300' : ''} />
<StatusRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} />
{publisherProgress ? <StatusRow label="Progress" value={publisherProgress} /> : null}
{publisher.lastStderr ? (
<div className="surface max-h-24 overflow-y-auto whitespace-pre-wrap break-words font-mono text-[0.68rem] leading-tight text-slate-200">
{publisher.lastStderr}
</div>
) : null}
{ptz?.blocked?.message ? (
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
{ptz.blocked.message}
</div>
) : null}
{onRelease ? (
<button type="button" className="button-dark w-full text-xs" disabled={releaseDisabled} onClick={onRelease}>
Release camera
</button>
) : null}
</CardFrame>
);
}
function PtzLightingControls({ ptz, disabled = false }) {
const { ptzSpotlight, ptzIr } = useSessionActions();
const [busy, setBusy] = useState('');
const spotlightOn = isSpotlightOn(ptz?.light);
const irMode = normalizeIrMode(ptz?.ir?.state);
const toggleSpotlight = async (nextOn) => {
if (disabled) return;
setBusy('spotlight');
try {
await ptzSpotlight({ state: nextOn ? 1 : 0 });
} finally {
setBusy('');
}
};
const cycleIr = async () => {
if (disabled) return;
setBusy('ir');
try {
await ptzIr({ state: nextIrMode(irMode) });
} finally {
setBusy('');
}
};
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<GPIOToggleControl
label="Spotlight"
on={spotlightOn}
disabled={disabled || busy === 'spotlight'}
onToggle={toggleSpotlight}
heightClass="min-h-14"
/>
<button
type="button"
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
disabled={disabled || busy === 'ir'}
onClick={cycleIr}
>
<span className="text-sm font-semibold">Infrared</span>
<span className="rounded bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
</button>
</div>
);
}
function PtzMobileZoomButtons({ disabled = false }) {
const { ptzMove, ptzStop } = useSessionActions();
const stopZoom = useCallback(() => {
ptzStop().catch(() => {});
}, [ptzStop]);
const startZoom = useCallback(
(direction) => (event) => {
/*
Mobile needs explicit zoom targets because the regular mobile drive pad
is already used for pan/tilt. Desktop does not render these buttons; it
uses the mapped camera up/down controls shown in the reference panel.
*/
event.preventDefault();
if (disabled) return;
ptzMove({ pan: 0, tilt: 0, zoom: direction * PTZ_ZOOM_SPEED }).catch(() => {});
},
[disabled, ptzMove],
);
const stopFromPointer = useCallback(
(event) => {
event?.preventDefault?.();
if (disabled) return;
stopZoom();
},
[disabled, stopZoom],
);
return (
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(-1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onPointerLeave={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom out
</button>
<button
type="button"
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
disabled={disabled}
onPointerDown={startZoom(1)}
onPointerUp={stopFromPointer}
onPointerCancel={stopFromPointer}
onPointerLeave={stopFromPointer}
onContextMenu={(event) => event.preventDefault()}
>
Zoom in
</button>
</div>
);
}
function PtzMobileControlsPanel({ ptz, disabled = false }) {
return (
<div className="mobile-touch-control space-y-0.5">
<PtzMobileZoomButtons disabled={disabled} />
<div className="mobile-touch-control h-44 min-h-0">
{/*
Reuse the rover mobile movement card instead of building a second PTZ
joystick. Its drive vector goes through the shared internal control
layer, where the PTZ adapter already converts movement plus speed mode
into camera pan/tilt commands.
*/}
<ControlPadPanel compact disabled={disabled} />
</div>
<PtzLightingControls ptz={ptz} disabled={disabled} />
</div>
);
}
function keyLabelFor(keymap, actionId) {
return formatKeyLabel(keymap?.[actionId]?.[0]);
}
function PtzControlReference() {
const keymap = useControlSelector((control) => control.state.keymap);
const rows = [
['Tilt up', 'driveForward'],
['Tilt down', 'driveBackward'],
['Pan left', 'driveLeft'],
['Pan right', 'driveRight'],
['Zoom in', 'cameraUp'],
['Zoom out', 'cameraDown'],
['Spotlight', 'headlightToggle'],
['Infrared mode', 'laserToggle'],
];
return (
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
{rows.map(([label, actionId]) => (
<div key={label} className="surface flex items-center justify-between gap-1">
<span className="text-slate-400">{label}</span>
{/* Use the same key display component as the rest of the UI so PTZ
controls read as normal mapped controls instead of custom labels. */}
<KeyPill label={keyLabelFor(keymap, actionId)} />
</div>
))}
</CardFrame>
);
}
function PtzController({ open, onClose, layout = 'desktop' }) {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isOperator = Boolean(ptz?.isOperator);
const isMobile = layout === 'mobile-portrait' || layout === 'mobile-landscape';
const { ptzRelease } = useSessionActions();
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !isOperator });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const [releasePending, setReleasePending] = useState(false);
if (!open) return null;
const releaseAndClose = async () => {
setReleasePending(true);
try {
await ptzRelease();
onClose();
} finally {
setReleasePending(false);
}
};
const desktopSidebar = (
<>
<div className="shrink-0">
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
</div>
{isOperator ? (
<div className="shrink-0">
<PtzLightingControls ptz={ptz} />
</div>
) : (
<div className="shrink-0">
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
Live PTZ controls unlock when your camera turn is active.
</CardFrame>
</div>
)}
<div className="shrink-0">
<PtzControlReference />
</div>
<div className="min-h-[12rem] flex-1">
{/*
The global chat key focuses the input registered by ChatPanel through
ChatContext. Keeping a real ChatPanel mounted inside the PTZ fullscreen
sidebar lets the normal keyboard path focus chat and lets Enter submit
through the existing chat composer form instead of adding PTZ-specific
chat handling.
*/}
<ChatPanel fillHeight title="Chat" />
</div>
<div className="shrink-0">
<ReplaySourcesPanel panelId="ptz-controller-replay" />
</div>
<div className="shrink-0">
<PtzStatePanel
ptz={ptz}
onClose={onClose}
onRelease={isOperator ? releaseAndClose : null}
releaseDisabled={releasePending}
/>
</div>
</>
);
const mobileSidebar = (
<>
<div className="shrink-0">
<PtzMobileControlsPanel ptz={ptz} disabled={!isOperator} />
</div>
<div className="min-h-[10rem] flex-1">
{/*
Mobile uses the same ChatPanel registration as desktop so the mapped
chat key and the on-screen input stay on one shared chat implementation.
*/}
<ChatPanel fillHeight title="Chat" />
</div>
<div className="shrink-0">
<ReplaySourcesPanel panelId="ptz-controller-replay-mobile" />
</div>
<div className="shrink-0">
<PtzStatePanel
ptz={ptz}
onClose={onClose}
onRelease={isOperator ? releaseAndClose : null}
releaseDisabled={releasePending}
/>
</div>
<div className="shrink-0">
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
</div>
</>
);
const sidebarWidthClass = isMobile
? 'grid-cols-[minmax(0,1fr)_14rem]'
: 'grid-cols-[minmax(0,1fr)_20rem]';
const controller = (
<div className="fixed inset-0 z-[110] h-[100dvh] w-[100vw] overflow-hidden bg-black text-slate-100">
<CardFrame
hideHeader
fillHeight
clipOverflow={false}
className="h-[100dvh] w-[100vw] rounded-none border-0 !bg-black"
bodyClassName={`grid h-full min-h-0 overflow-hidden ${sidebarWidthClass}`}
>
<main className="relative min-h-0 min-w-0 bg-black">
{isOperator ? (
<PtzLiveVideo enabled startMuted={false} />
) : (
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
)}
</main>
<aside className="flex h-full min-h-0 items-stretch overflow-hidden border-l border-neutral-600 bg-neutral-950 text-sm">
<div className="flex min-h-0 w-full flex-col gap-0.5 overflow-y-auto p-0.5">
{isMobile ? mobileSidebar : desktopSidebar}
</div>
</aside>
</CardFrame>
</div>
);
/*
The controller is a true fullscreen surface, so mount it directly under
document.body instead of inside the VIP tab/card tree. That keeps tab panel
spacing, mobile banners, and parent overflow rules from creating visible
gaps around a fixed-position camera interface.
*/
return createPortal(controller, document.body);
}
export default function VipPtzCameraCard({ onMessage, fullWidth = false, layout = 'desktop' }) {
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
const { ptzClaim, ptzRelease } = useSessionActions();
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: Boolean(featureEnabled) });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
const [controllerOpen, setControllerOpen] = useState(false);
const [pending, setPending] = useState(false);
const wrapClass = fullWidth ? 'w-full' : 'mx-auto w-full max-w-xl';
const queueText = useMemo(() => {
if (ptz?.isOperator) return 'Your turn';
if (ptz?.queuedPosition) return `Queue position ${ptz.queuedPosition}`;
if (ptz?.operatorLabel) return `${ptz.operatorLabel} operating`;
return 'Available';
}, [ptz?.isOperator, ptz?.operatorLabel, ptz?.queuedPosition]);
if (!featureEnabled) return null;
const handleClaim = async () => {
setPending(true);
onMessage?.('');
try {
const response = await ptzClaim();
/*
Requesting the PTZ camera is only enough to open fullscreen after the
server confirms the user actually became the operator or entered the
queue. Dock-required failures reject before queue entry, so they must
leave the user on this card with the dock/charge message visible.
*/
if (response?.state?.isOperator || response?.state?.queuedPosition) setControllerOpen(true);
if (response?.state?.isOperator) onMessage?.('PTZ camera turn active.');
else if (response?.state?.queuedPosition) onMessage?.(`Joined PTZ queue at position ${response.state.queuedPosition}.`);
} catch (err) {
/*
Rover-to-rover switch denial uses the browser alert popup for the
dock-and-charge message. PTZ claim denial should feel identical because
it is enforcing the same "do not abandon an undocked rover" rule.
*/
alert(err.message || 'PTZ request failed.');
onMessage?.(err.message || 'PTZ request failed.');
} finally {
setPending(false);
}
};
const handleRelease = async () => {
setPending(true);
try {
await ptzRelease();
onMessage?.('Left PTZ camera.');
} catch (err) {
onMessage?.(err.message || 'Failed to leave PTZ camera.');
} finally {
setPending(false);
}
};
return (
<>
<CardFrame title="PTZ camera" className={wrapClass} bodyClassName="text-sm text-slate-300">
<div className="grid w-full gap-0.5 md:grid-cols-[minmax(16rem,0.8fr)_minmax(0,1.2fr)] md:items-start">
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
<div className="flex w-full min-w-0 flex-col gap-0.5">
<div className="grid w-full grid-cols-2 gap-0.5 text-left text-xs">
<div className="surface-muted p-1">
<p className="text-slate-500">State</p>
<p className="truncate text-slate-100">{queueText}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Remaining</p>
<p className="text-slate-100">{formatRemaining(ptz?.deadline)}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Spotlight</p>
<p className="truncate text-slate-100">{isSpotlightOn(ptz?.light) ? 'On' : 'Off'}</p>
</div>
<div className="surface-muted p-1">
<p className="text-slate-500">Infrared</p>
<p className="truncate text-slate-100">{normalizeIrMode(ptz?.ir?.state)}</p>
</div>
</div>
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
{ptz?.blocked?.message ? (
<p className="w-full rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
{ptz.blocked.message}
</p>
) : null}
<div className="grid w-full grid-cols-2 gap-0.5">
{ptz?.isOperator ? (
<>
<button type="button" className="button-dark text-xs" onClick={() => setControllerOpen(true)}>
Open controller
</button>
<button type="button" className="button-dark text-xs" disabled={pending} onClick={handleRelease}>
Release
</button>
</>
) : (
<button
type="button"
className="button-dark col-span-2 text-xs"
disabled={pending || !isVerified}
onClick={ptz?.queuedPosition ? handleRelease : handleClaim}
>
{ptz?.queuedPosition ? 'Leave queue' : pending ? 'Requesting...' : 'Claim camera'}
</button>
)}
</div>
</div>
</div>
</CardFrame>
<PtzController open={controllerOpen} onClose={() => setControllerOpen(false)} layout={layout} />
</>
);
}
+14 -2
View File
@@ -61,6 +61,7 @@ export function ChatProvider({ children }) {
const [isChatFocused, setIsChatFocused] = useState(false);
const panelInputRef = useRef(null);
const hudInputRef = useRef(null);
const overlayInputRef = useRef(null);
const audioRef = useRef(null);
const typingRef = useRef(new Map());
const typingAlertRef = useRef(new Map());
@@ -236,7 +237,17 @@ export function ChatProvider({ children }) {
);
const registerInputRef = useCallback((el, options = {}) => {
const target = options?.target === 'hud' ? 'hud' : 'panel';
const target = options?.target === 'overlay' ? 'overlay' : options?.target === 'hud' ? 'hud' : 'panel';
/*
Fullscreen surfaces such as PTZ need chat-key focus while they are mounted
without stealing the normal HUD ref permanently. Keep overlay chat as a
separate highest-priority slot so unmounting the portal only clears the
overlay target and leaves the driver HUD/panel refs intact.
*/
if (target === 'overlay') {
overlayInputRef.current = el;
return;
}
if (target === 'hud') {
hudInputRef.current = el;
} else {
@@ -246,11 +257,12 @@ export function ChatProvider({ children }) {
const focusChat = useCallback(() => {
setIsChatFocused(true);
(hudInputRef.current || panelInputRef.current)?.focus();
(overlayInputRef.current || hudInputRef.current || panelInputRef.current)?.focus();
}, []);
const blurChat = useCallback(() => {
setIsChatFocused(false);
overlayInputRef.current?.blur();
hudInputRef.current?.blur();
panelInputRef.current?.blur();
}, []);
+10
View File
@@ -420,6 +420,16 @@ export function SessionProvider({ children }) {
setAudioLevels: (levels = {}) => emitWithAck('audioLevels:set', levels),
setPrivateSafety: (roverId, safety = {}) =>
emitWithAck('session:privateSafety:set', { roverId, safety }),
ptzClaim: () => emitWithAck('ptzCamera:claim'),
ptzRelease: () => emitWithAck('ptzCamera:release'),
ptzMove: (payload = {}) => emitWithAck('ptzCamera:move', payload),
ptzStop: () => emitWithAck('ptzCamera:stop'),
ptzSpotlight: (payload = {}) => emitWithAck('ptzCamera:spotlight', payload),
ptzIr: (payload = {}) => emitWithAck('ptzCamera:ir', payload),
ptzListPresets: () => emitWithAck('ptzCamera:presets:list'),
ptzGotoPreset: (payload = {}) => emitWithAck('ptzCamera:preset:goto', payload),
ptzCreatePreset: (payload = {}) => emitWithAck('ptzCamera:preset:create', payload),
ptzRemovePreset: (payload = {}) => emitWithAck('ptzCamera:preset:remove', payload),
llmControl: (action, controls = {}) =>
emitWithAck('llm:control', { controls: { action, ...controls } }),
overseerControl: (action, controls = {}) =>
+41 -10
View File
@@ -35,6 +35,7 @@ import {
applyDriveOvercurrentScale,
useOvercurrentLimiter,
} from './overcurrentLimiter.js';
import { usePtzControlAdapter } from './ptzControlAdapter.js';
const ControlSystemContext = createContext(null);
@@ -179,6 +180,7 @@ export function ControlSystemProvider({ children }) {
[overcurrentLimiter.adminImmune, overcurrentLimiter.scales],
);
const pipeline = useCommandPipeline({ driveTransform, auxTransform });
const ptzControls = usePtzControlAdapter();
const turnOnAllLights = useCallback(() => {
const entities = homeAssistantEntities || [];
@@ -327,9 +329,10 @@ export function ControlSystemProvider({ children }) {
payload: { ...computed, source: meta.source ?? null },
});
recordControlIntent();
if (ptzControls.applyDriveVector(vector, meta)) return;
pipeline.sendDriveDirect(computed.speeds);
},
[pipeline, recordControlIntent, state.manualDockAssist?.active],
[pipeline, ptzControls, recordControlIntent, state.manualDockAssist?.active],
);
const setAuxMotors = useCallback(
@@ -372,6 +375,21 @@ export function ControlSystemProvider({ children }) {
const setServoAngle = useCallback(
(value, options = {}) => {
if (ptzControls.isActive) {
/*
Servo-capable rover controls converge here from keyboard, gamepad,
desktop, and mobile. When the active control target is the PTZ camera,
route the intent through the PTZ adapter instead of making the rover
command pipeline understand camera zoom semantics.
*/
const baseline = typeof servoAngleRef.current === 'number' ? servoAngleRef.current : 0;
const numeric = Number(value);
if (!Number.isFinite(numeric)) return;
ptzControls.pulseZoom(numeric - baseline);
servoAngleRef.current = numeric;
recordControlIntent();
return;
}
if (!pipeline.servoConfig) return;
const force = Boolean(options?.force);
if (state.manualDockAssist?.active && !force) return;
@@ -381,23 +399,23 @@ export function ControlSystemProvider({ children }) {
servoAngleRef.current = clamped;
recordControlIntent();
},
[pipeline, recordControlIntent, state.manualDockAssist?.active],
[pipeline, ptzControls, recordControlIntent, state.manualDockAssist?.active],
);
const nudgeServo = useCallback(
(delta = 0) => {
const config = pipeline.servoConfig;
if (!config) return;
const step = typeof delta === 'number' && delta !== 0 ? delta : config.nudgeDegrees || 1;
if (!config && !ptzControls.isActive) return;
const step = typeof delta === 'number' && delta !== 0 ? delta : config?.nudgeDegrees || 1;
const baseline =
typeof servoAngleRef.current === 'number'
? servoAngleRef.current
: typeof config.homeAngle === 'number'
: typeof config?.homeAngle === 'number'
? config.homeAngle
: 0;
setServoAngle(baseline + step);
},
[pipeline.servoConfig, setServoAngle],
[pipeline.servoConfig, ptzControls.isActive, setServoAngle],
);
const goServoHome = useCallback(() => {
@@ -460,9 +478,13 @@ export function ControlSystemProvider({ children }) {
source: 'system-stop',
},
});
if (ptzControls.isActive) {
ptzControls.stopMotion();
return;
}
pipeline.sendDriveDirect({ left: 0, right: 0 });
pipeline.sendAuxMotors({ main: 0, side: 0, vacuum: 0 });
}, [pipeline]);
}, [pipeline, ptzControls]);
const sendOiCommand = useCallback(
(command) => {
@@ -484,6 +506,10 @@ export function ControlSystemProvider({ children }) {
const setHeadlight = useCallback(
(headlightOn) => {
if (ptzControls.setSpotlight(headlightOn)) {
recordControlIntent();
return;
}
if (!pipeline.headlight) return;
// Web controls now speak in logical device state. Any electrical
// inversion needed by the actual GPIO driver is handled by roverd's
@@ -492,7 +518,7 @@ export function ControlSystemProvider({ children }) {
pipeline.sendHeadlight(action);
recordControlIntent();
},
[pipeline, recordControlIntent],
[pipeline, ptzControls, recordControlIntent],
);
const toggleHeadlight = useCallback(() => {
@@ -501,6 +527,10 @@ export function ControlSystemProvider({ children }) {
const setLaser = useCallback(
(laserOn) => {
if (ptzControls.setIr(laserOn)) {
recordControlIntent();
return;
}
if (!pipeline.laser) return;
if (roomLightsLockedOn && laserOn !== false) return;
// The laser shares the same logical toggle contract as the headlight; it
@@ -509,7 +539,7 @@ export function ControlSystemProvider({ children }) {
pipeline.sendLaser(action);
recordControlIntent();
},
[pipeline, recordControlIntent, roomLightsLockedOn],
[pipeline, ptzControls, recordControlIntent, roomLightsLockedOn],
);
const toggleLaser = useCallback(() => {
@@ -739,10 +769,11 @@ export function ControlSystemProvider({ children }) {
state,
dispatch,
pipeline,
ptzControls,
overcurrentLimiter,
actions: stableActions,
}),
[state, pipeline, overcurrentLimiter, stableActions],
[state, pipeline, ptzControls, overcurrentLimiter, stableActions],
);
if (snapshotRef.current == null) {
+1 -2
View File
@@ -46,7 +46,6 @@ export function useCommandPipeline(options = {}) {
const headlightState = useMemo(() => rosterEntry?.headlight?.state ?? null, [rosterEntry]);
const laserState = useMemo(() => rosterEntry?.laser?.state ?? null, [rosterEntry]);
const emitCommand = useCallback(
(payload, cb) => {
if (!roverId) return;
@@ -65,7 +64,6 @@ export function useCommandPipeline(options = {}) {
const sendDriveDirect = useCallback(
(speeds) => {
if (!roverId) return null;
const rawPayload = {
left: clampRange(speeds?.left ?? 0, [-500, 500]),
right: clampRange(speeds?.right ?? 0, [-500, 500]),
@@ -75,6 +73,7 @@ export function useCommandPipeline(options = {}) {
left: clampRange(transformed?.left ?? 0, [-500, 500]),
right: clampRange(transformed?.right ?? 0, [-500, 500]),
};
if (!roverId) return null;
emitCommand({
type: 'drive',
data: { driveDirect: payload },
+2 -2
View File
@@ -12,8 +12,8 @@ export const OVERCURRENT_GROUPS = [
export const DEFAULT_OVERCURRENT_LIMITS = {
downRatePerSec: 0.5,
upRatePerSec: 0.5,
releaseDelaySec: 2,
upRatePerSec: 0.7,
releaseDelaySec: 1,
outputRateMs: 250,
};
+236
View File
@@ -0,0 +1,236 @@
// PTZ Control Adapter
// Purpose: Hooks the single PTZ camera into the internal control action layer.
// Scope: Owns PTZ-specific control mixing and socket commands so keyboard,
// mobile, desktop, and gamepad inputs do not each learn camera-specific rules.
import { useCallback, useEffect, useMemo, useRef } from 'react';
import { useSocket } from '../context/SocketContext.jsx';
import { useSessionSelector } from '../context/SessionContext.jsx';
const PTZ_STOP = { pan: 0, tilt: 0, zoom: 0 };
const PTZ_SPEEDS = {
slow: 0.1,
medium: 0.5,
fast: 1,
};
const ZOOM_PULSE_MS = 220;
function clampUnit(value) {
const number = Number(value) || 0;
return Math.max(-1, Math.min(1, number));
}
function axisSign(value) {
const number = Number(value) || 0;
if (number > 0.05) return 1;
if (number < -0.05) return -1;
return 0;
}
function pickPanTiltSpeed(vector = {}, meta = {}) {
/*
PTZ movement should not inherit rover wheel speeds. Inputs only choose a
speed tier here: precision/low-magnitude input is slow, normal input is
medium, and explicit boost is fast. That gives every control surface the
same camera feel without remixing differential-drive output.
*/
if (vector?.boost) return PTZ_SPEEDS.fast;
const maxAxis = Math.max(Math.abs(Number(vector?.x) || 0), Math.abs(Number(vector?.y) || 0));
const baseSpeed = Number(meta?.speedOptions?.baseSpeed);
if (maxAxis > 0 && maxAxis <= 0.45) return PTZ_SPEEDS.slow;
if (Number.isFinite(baseSpeed) && baseSpeed > 0 && baseSpeed <= 150) return PTZ_SPEEDS.slow;
return PTZ_SPEEDS.medium;
}
function payloadSignature(payload = PTZ_STOP) {
return [
Number(payload.pan || 0).toFixed(3),
Number(payload.tilt || 0).toFixed(3),
Number(payload.zoom || 0).toFixed(3),
].join(':');
}
function isIdlePayload(payload = PTZ_STOP) {
return !payload.pan && !payload.tilt && !payload.zoom;
}
function buildPanTiltPayload(vector = {}, meta = {}) {
const panSign = axisSign(vector.x);
const tiltSign = axisSign(vector.y);
if (!panSign && !tiltSign) return PTZ_STOP;
const speed = pickPanTiltSpeed(vector, meta);
const diagonalScale = panSign && tiltSign ? Math.SQRT1_2 : 1;
return {
pan: clampUnit(panSign * speed * diagonalScale),
tilt: clampUnit(tiltSign * speed * diagonalScale),
zoom: 0,
};
}
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeIrMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function nextIrMode(currentMode) {
/*
The rover laser button becomes the PTZ camera's IR mode control while the
PTZ camera is active. Cycle all three modes exposed by this Reolink camera
instead of reducing the control to a two-state toggle.
*/
const current = normalizeIrMode(currentMode);
if (current === 'Auto') return 'On';
if (current === 'On') return 'Off';
return 'Auto';
}
export function usePtzControlAdapter() {
const socket = useSocket();
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
const isActive = Boolean(ptz?.isOperator);
const lastMotionSignatureRef = useRef(payloadSignature(PTZ_STOP));
const zoomStopTimerRef = useRef(null);
const emitPtz = useCallback(
(eventName, payload = {}) => {
if (!socket || !isActive) return;
socket.emit(eventName, payload);
},
[isActive, socket],
);
const stopMotion = useCallback(() => {
if (zoomStopTimerRef.current) {
clearTimeout(zoomStopTimerRef.current);
zoomStopTimerRef.current = null;
}
const stopSignature = payloadSignature(PTZ_STOP);
if (lastMotionSignatureRef.current === stopSignature) return;
lastMotionSignatureRef.current = stopSignature;
emitPtz('ptzCamera:stop');
}, [emitPtz]);
const sendMotion = useCallback(
(payload, options = {}) => {
if (!isActive) return false;
const nextPayload = {
pan: clampUnit(payload?.pan),
tilt: clampUnit(payload?.tilt),
zoom: clampUnit(payload?.zoom),
};
const nextSignature = payloadSignature(nextPayload);
if (!options.force && lastMotionSignatureRef.current === nextSignature) return true;
lastMotionSignatureRef.current = nextSignature;
if (isIdlePayload(nextPayload)) {
emitPtz('ptzCamera:stop');
} else {
emitPtz('ptzCamera:move', nextPayload);
}
return true;
},
[emitPtz, isActive],
);
const applyDriveVector = useCallback(
(vector, meta = {}) => {
if (!isActive) return false;
sendMotion(buildPanTiltPayload(vector, meta));
return true;
},
[isActive, sendMotion],
);
const pulseZoom = useCallback(
(direction) => {
if (!isActive) return false;
const sign = axisSign(direction);
if (!sign) {
stopMotion();
return true;
}
/*
Zoom is different from pan/tilt because it is driven by repeated nudge
events from existing camera controls. Force each pulse through even when
the payload is identical, otherwise holding "camera up" only sends the
first zoom command and every later nudge is de-duped away.
*/
sendMotion({ pan: 0, tilt: 0, zoom: sign * PTZ_SPEEDS.medium }, { force: true });
if (zoomStopTimerRef.current) clearTimeout(zoomStopTimerRef.current);
/*
Existing rover camera controls are nudge/slider based, not hold-based.
Treat each nudge as a short PTZ zoom pulse, then stop from the adapter
so delayed stop behavior is owned by the camera layer only.
*/
zoomStopTimerRef.current = setTimeout(() => {
zoomStopTimerRef.current = null;
stopMotion();
}, ZOOM_PULSE_MS);
return true;
},
[isActive, sendMotion, stopMotion],
);
const setSpotlight = useCallback(
(nextOn) => {
if (!isActive) return false;
const desiredOn = typeof nextOn === 'boolean' ? nextOn : !isSpotlightOn(ptz?.light);
emitPtz('ptzCamera:spotlight', { state: desiredOn ? 1 : 0 });
return true;
},
[emitPtz, isActive, ptz?.light],
);
const setIr = useCallback(
(nextOn) => {
if (!isActive) return false;
const desiredState = typeof nextOn === 'boolean'
? (nextOn ? 'On' : 'Off')
: nextIrMode(ptz?.ir?.state);
emitPtz('ptzCamera:ir', { state: desiredState });
return true;
},
[emitPtz, isActive, ptz?.ir?.state],
);
useEffect(() => {
if (isActive) return undefined;
lastMotionSignatureRef.current = payloadSignature(PTZ_STOP);
if (zoomStopTimerRef.current) {
clearTimeout(zoomStopTimerRef.current);
zoomStopTimerRef.current = null;
}
return undefined;
}, [isActive]);
useEffect(
() => () => {
if (zoomStopTimerRef.current) clearTimeout(zoomStopTimerRef.current);
},
[],
);
return useMemo(
() => ({
isActive,
state: ptz,
applyDriveVector,
pulseZoom,
setSpotlight,
setIr,
stopMotion,
}),
[applyDriveVector, isActive, ptz, pulseZoom, setIr, setSpotlight, stopMotion],
);
}
+136
View File
@@ -0,0 +1,136 @@
// Hook: usePtzCameraSnapshots
// Purpose: Subscribes to PTZ snapshot streams using the same map-shaped contract as rover snapshots.
// Scope: Keeps PTZ snapshot lifecycle boring: callers pass source ids and receive feeds keyed by id.
import { useEffect, useMemo, useRef, useState } from 'react';
import { useSocket } from '../context/SocketContext.jsx';
export const PTZ_CAMERA_ID = 'ptz-camera';
export function usePtzCameraSnapshots(sourceList = [], options = {}) {
const socket = useSocket();
const { enabled = true, version = null } = options;
const [feeds, setFeeds] = useState({});
const objectUrls = useRef(new Map());
const ids = useMemo(
() => sourceList.map((entry) => (typeof entry === 'string' ? entry : entry?.id)).filter(Boolean),
[sourceList],
);
const idsKey = useMemo(() => {
const base = ids.join('|');
return version ? `${base}|v:${version}` : base;
}, [ids, version]);
const idsRef = useRef([]);
const [connectionNonce, setConnectionNonce] = useState(0);
const statsRef = useRef(new Map());
const debugSnapshots =
typeof window !== 'undefined' && new URLSearchParams(window.location.search).has('debugSnapshots');
useEffect(() => {
if (!socket) return undefined;
const handleConnect = () => setConnectionNonce((prev) => prev + 1);
socket.on('connect', handleConnect);
return () => socket.off('connect', handleConnect);
}, [socket]);
useEffect(() => {
idsRef.current = ids;
}, [idsKey, ids]);
useEffect(() => {
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
objectUrls.current.clear();
setFeeds({});
}, [idsKey]);
useEffect(() => {
if (!enabled) {
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
objectUrls.current.clear();
setFeeds({});
return undefined;
}
if (!idsRef.current.length || !socket) return undefined;
let cancelled = false;
const currentIds = idsRef.current;
const handleFrame = (meta = {}, buffer) => {
if (cancelled || !meta.id || !buffer) return;
const sizeBytes = buffer.byteLength ?? buffer.length ?? 0;
const now = Date.now();
const prevStats = statsRef.current.get(meta.id) || {
count: 0,
totalBytes: 0,
lastLogAt: 0,
};
const nextStats = {
count: prevStats.count + 1,
totalBytes: prevStats.totalBytes + sizeBytes,
lastLogAt: prevStats.lastLogAt,
};
if (debugSnapshots && (!nextStats.lastLogAt || now - nextStats.lastLogAt >= 10000)) {
const avgBytes = nextStats.count ? nextStats.totalBytes / nextStats.count : 0;
console.log(
'[ptzSnapshot]',
meta.id,
`frame=${sizeBytes}B`,
`avg=${Math.round(avgBytes)}B`,
`count=${nextStats.count}`,
);
nextStats.lastLogAt = now;
}
statsRef.current.set(meta.id, nextStats);
const url = URL.createObjectURL(new Blob([buffer], { type: 'image/jpeg' }));
const prevUrl = objectUrls.current.get(meta.id);
if (prevUrl) URL.revokeObjectURL(prevUrl);
objectUrls.current.set(meta.id, url);
setFeeds((prev) => ({
...prev,
[meta.id]: {
status: 'playing',
ts: meta.ts || Date.now(),
error: null,
objectUrl: url,
},
}));
};
const handleStatus = (meta = {}) => {
if (cancelled || !meta.id) return;
setFeeds((prev) => ({
...prev,
[meta.id]: {
...(prev[meta.id] || {}),
status: meta.error ? 'error' : prev[meta.id]?.status || 'connecting',
error: meta.error || null,
ts: meta.ts || prev[meta.id]?.ts || null,
objectUrl: prev[meta.id]?.objectUrl || null,
},
}));
};
socket.on('ptzCamera:snapshotFrame', handleFrame);
socket.on('ptzCamera:snapshotStatus', handleStatus);
socket.emit('ptzCamera:snapshotSubscribe', { ids: currentIds }, () => {});
return () => {
cancelled = true;
socket.emit('ptzCamera:snapshotUnsubscribe', { ids: currentIds });
socket.off('ptzCamera:snapshotFrame', handleFrame);
socket.off('ptzCamera:snapshotStatus', handleStatus);
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
objectUrls.current.clear();
};
}, [socket, idsKey, enabled, connectionNonce, debugSnapshots]);
return feeds;
}
export function usePtzCameraSnapshot(options = {}) {
/*
This wrapper keeps older callers working while new PTZ UI uses the same
keyed feed shape as useRoverSnapshots(). It should stay tiny so the real
subscription behavior only has one implementation to maintain.
*/
const feeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], options);
return feeds[PTZ_CAMERA_ID] || null;
}
+6 -1
View File
@@ -88,7 +88,12 @@ export function useVideoRequests(sourceList = [], options = {}) {
let cancelled = false;
function requestEntry(entry) {
const payload = entry.type === 'room' ? { roomCameraId: entry.id } : { roverId: entry.id };
const payload =
entry.type === 'room'
? { roomCameraId: entry.id }
: entry.type === 'ptz'
? { type: 'ptz', id: entry.id }
: { roverId: entry.id };
socket.emit('video:request', payload, (resp = {}) => {
if (cancelled) return;
if (resp?.error) {
+18 -3
View File
@@ -29,12 +29,19 @@ function buildAuthHeader(token) {
}
export class WhepPlayer {
constructor({ url, token, video, onStatus, audioOnly = false, receiveAudio = true }) {
constructor({ url, token, video, onStatus, audioOnly = false, receiveAudio = true, startMuted = null }) {
this.url = url;
this.token = token;
this.video = video;
this.audioOnly = audioOnly;
this.receiveAudio = receiveAudio;
/*
Browser autoplay usually requires normal video streams to start muted,
while audio-only streams need to start audible. Keep that historical
default, but allow a caller that is already behind a user gesture, such as
the PTZ fullscreen controller, to request audible inline audio.
*/
this.startMuted = startMuted === null ? !audioOnly : Boolean(startMuted);
this.pc = null;
this.abortController = null;
this.onStatus = onStatus;
@@ -50,7 +57,7 @@ export class WhepPlayer {
if (!this.video) return;
this.video.playsInline = true;
this.video.autoplay = true;
this.video.muted = this.audioOnly ? false : true;
this.video.muted = this.startMuted;
if (typeof this.video.disableRemotePlayback !== 'undefined') {
this.video.disableRemotePlayback = true;
}
@@ -113,7 +120,15 @@ export class WhepPlayer {
signal: this.abortController.signal,
});
if (!response.ok) {
throw new Error(`WHEP request failed: ${response.status}`);
/*
MediaMTX includes the useful rejection reason in the response body
for many 4xx WHEP failures, such as unsupported codecs or malformed
SDP. Surface that body so camera/video debugging does not stop at a
generic HTTP status code.
*/
const body = await response.text().catch(() => '');
const detail = body ? `: ${body.slice(0, 180)}` : '';
throw new Error(`WHEP request failed: ${response.status}${detail}`);
}
const answerSdp = await response.text();
await pc.setRemoteDescription({ type: 'answer', sdp: answerSdp });
@@ -0,0 +1,127 @@
// PTZ Spectator Card
// Purpose: Adds the single room PTZ camera to the spectator rover grid.
// Scope: Uses live WHEP only when the server authorizes this socket; otherwise
// falls back to the PTZ snapshot feed that remote spectators are allowed to see.
import PtzLiveVideo from '../../../components/PtzLiveVideo/index.jsx';
import { useSessionSelector } from '../../../context/SessionContext.jsx';
import { PTZ_CAMERA_ID, usePtzCameraSnapshots } from '../../../hooks/usePtzCameraSnapshot.js';
function formatRemaining(deadline) {
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - Date.now()) / 1000));
if (!remaining) return '--';
const minutes = Math.floor(remaining / 60);
const seconds = remaining % 60;
return `${minutes}:${String(seconds).padStart(2, '0')}`;
}
function isSpotlightOn(light = {}) {
if (typeof light?.on === 'boolean') return light.on;
const raw = light?.state;
if (typeof raw === 'string') {
const normalized = raw.trim().toLowerCase();
return !['', '0', 'off', 'false'].includes(normalized);
}
return Boolean(Number(raw));
}
function normalizeInfraredMode(mode) {
const normalized = String(mode || '').trim().toLowerCase();
if (normalized === 'on') return 'On';
if (normalized === 'off') return 'Off';
return 'Auto';
}
function InfoRow({ label, value, tone = '' }) {
return (
<div className="flex items-center justify-between gap-1">
<span className="text-slate-400">{label}</span>
<span className={`min-w-0 truncate ${tone || 'text-slate-100'}`}>{value}</span>
</div>
);
}
function formatPublisherProgress(progress = null) {
/*
Keep the spectator card dense, but expose enough ffmpeg progress to tell if
the PTZ transcoder is running behind when the live feed looks delayed.
*/
if (!progress) return null;
return [
progress.fps ? `fps ${progress.fps}` : null,
progress.speed ? `speed ${progress.speed}` : null,
progress.drop_frames ? `drop ${progress.drop_frames}` : null,
].filter(Boolean).join(' | ');
}
function PtzSnapshotFallback({ label, source }) {
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: true });
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
return (
<div className="relative aspect-video w-full overflow-hidden rounded bg-black">
{snapshot?.objectUrl ? (
<img src={snapshot.objectUrl} alt={label} className="h-full w-full object-cover" />
) : (
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">
{snapshot?.error || source?.error || 'Waiting for PTZ snapshot...'}
</div>
)}
{/*
Keep the PTZ snapshot fallback visually aligned with RoverMediaPlayer:
the video surface owns only playback health, while the card around it
owns identity/context. That prevents a second in-frame camera title and
keeps fallback mode from looking different than live WHEP mode.
*/}
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
<div className="flex flex-col gap-0.5 leading-none">
<span>Status: {snapshot?.status || 'snapshot'}</span>
</div>
</div>
</div>
);
}
function PtzLiveOrSnapshot({ label }) {
return (
<PtzLiveVideo
enabled
startMuted
className="relative aspect-video w-full overflow-hidden rounded bg-black"
fallback={({ source }) => <PtzSnapshotFallback label={label} source={source} />}
/>
);
}
export default function PtzSpectatorCard() {
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
if (!ptz?.enabled) return null;
const publisher = ptz?.publisher || {};
const publisherStatus = publisher.running
? 'running'
: publisher.restartAt
? 'restarting'
: publisher.lastEvent || 'stopped';
const publisherProgress = formatPublisherProgress(publisher.progress);
const queueCount = Array.isArray(ptz?.queue) ? ptz.queue.length : 0;
const label = ptz?.name || 'PTZ Camera';
return (
<article className="flex min-h-[16rem] flex-col rounded bg-zinc-900 p-0 sm:min-h-[18rem]">
<PtzLiveOrSnapshot label={label} />
<div className="min-h-0 flex-1 space-y-0.5 overflow-hidden p-1 text-xs">
<InfoRow label="Operator" value={ptz?.operatorLabel || 'none'} />
<InfoRow label="Remaining" value={formatRemaining(ptz?.deadline)} />
<InfoRow label="Queue" value={queueCount ? `${queueCount} waiting` : 'empty'} />
<InfoRow label="Spotlight" value={isSpotlightOn(ptz?.light) ? 'On' : 'Off'} />
<InfoRow label="Infrared" value={normalizeInfraredMode(ptz?.ir?.state)} />
<InfoRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} />
{publisherProgress ? <InfoRow label="Progress" value={publisherProgress} /> : null}
{publisher.lastStderr ? (
<div className="line-clamp-2 break-words font-mono text-[0.65rem] leading-tight text-slate-400">
{publisher.lastStderr}
</div>
) : null}
</div>
</article>
);
}
@@ -2,16 +2,16 @@
// Purpose: Defines the Rover Row module and the local helpers/components used in this file.
// Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
import RoverSpectatorCard from './RoverSpectatorCard.jsx';
import PtzSpectatorCard from './PtzSpectatorCard.jsx';
export default function RoverRow({ roster }) {
if (roster.length === 0) {
return <p className="col-span-full text-slate-400">No rovers registered.</p>;
}
return (
<section className="grid grid-cols-1 gap-0.5 md:grid-cols-2">
{roster.length === 0 ? <p className="col-span-full text-slate-400">No rovers registered.</p> : null}
{roster.map((rover) => (
<RoverSpectatorCard key={rover.id} rover={rover} />
))}
<PtzSpectatorCard />
</section>
);
}