mirror of
https://github.com/legop3/MultiRoombaRover.git
synced 2026-09-15 17:12:59 -04:00
@@ -5,6 +5,8 @@ create_2_Open_Interface_Spec.txt
|
||||
|
||||
logs
|
||||
node_modules/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.pio
|
||||
.vscode/
|
||||
config.h
|
||||
|
||||
Binary file not shown.
+130
-1
@@ -39,4 +39,133 @@
|
||||
- uhhh idk
|
||||
- obviously, camera on the screen
|
||||
- probably add a variant of the mobile controls just to retitle the things from the rover controls to the camera controls
|
||||
- and just use the same control columns
|
||||
- and just use the same control columns
|
||||
|
||||
-- slop generated below --
|
||||
|
||||
## clarified implementation direction
|
||||
|
||||
This is not intended to become a generic ONVIF camera framework. The camera integration is for one specific Reolink PTZ camera. Once the camera arrives, we will run a one-time ONVIF capability discovery against that exact camera, record what it exposes, and then build the integration around those known capabilities.
|
||||
|
||||
The one-time discovery should capture:
|
||||
- ONVIF services exposed by the camera
|
||||
- media profiles and stream URIs
|
||||
- snapshot URI support
|
||||
- PTZ support and movement modes
|
||||
- pan/tilt/zoom ranges and speed ranges
|
||||
- preset/home support
|
||||
- imaging controls
|
||||
- any ONVIF-exposed spotlight, IR, or night-vision controls
|
||||
- whether PTZ status reporting is reliable
|
||||
|
||||
After that, runtime code should assume this known camera profile instead of trying to dynamically support every possible ONVIF camera.
|
||||
|
||||
## claiming and operator rules
|
||||
|
||||
The PTZ camera is a single scarce controllable resource.
|
||||
|
||||
Only verified/VIP users can claim it during normal operation. Only one user can operate it at a time. The active operator gets live WebRTC video and PTZ control for a limited turn, probably around five minutes. Other remote users should only receive slow snapshots. Local spectators may be allowed live video because LAN traffic is not the bandwidth problem.
|
||||
|
||||
A user operating the PTZ camera must not also be operating a rover. When a user tries to move from a rover to PTZ, the existing rover-switch safety rule should be reused: switching is allowed if another driver remains on that rover, or if the current rover is docked and charging. Otherwise, the server should block the PTZ handoff and tell the user to dock and charge their rover first.
|
||||
|
||||
This should be implemented by refactoring the existing rover switching check into a shared helper, such as `canLeaveCurrentRover(socket)`, then using that helper from both rover switching and PTZ claiming.
|
||||
|
||||
## streaming model
|
||||
|
||||
Camera video should come from the Reolink camera over the local network, likely RTSP into MediaMTX. Browser playback should use the existing MediaMTX WHEP/WebRTC pipeline.
|
||||
|
||||
The existing video session and MediaMTX auth system should be extended with a `ptz` source type. Remote live WHEP access should be allowed for the current PTZ operator, local spectators, and authorized admins according to normal server rules. Remote non-operators should not get live video.
|
||||
|
||||
Slow snapshots should use a PTZ-specific snapshot path or socket gateway, modeled after the existing room camera snapshot system, but with PTZ-specific authorization rules.
|
||||
|
||||
## lockdown behavior
|
||||
|
||||
No extra UI work is needed for lockdown because the app already visually blocks things in lockdown mode.
|
||||
|
||||
Server-side lockdown enforcement is still required everywhere. In lockdown mode, only lockdown admins/users may claim, queue, operate, subscribe to snapshots, request live PTZ video, or use PTZ replay sources. If lockdown starts while a normal user is operating PTZ, the server should immediately revoke their operator state, remove them from the PTZ queue if needed, revoke PTZ video sessions, and stop accepting PTZ commands from them.
|
||||
|
||||
## reusable existing systems
|
||||
|
||||
Strong reuse targets:
|
||||
- rover switch safety logic from `roverManager/roverLifecycle.js`
|
||||
- `videoSessions`
|
||||
- `videoSocketService`
|
||||
- `videoAuthService`
|
||||
- `WhepPlayer`
|
||||
- `sessionService` session sync
|
||||
- VIP panel/card structure
|
||||
- alert system
|
||||
- replay source validation and replay worker architecture
|
||||
|
||||
Adapted reuse targets:
|
||||
- turn queue/timer structure from `turnService`
|
||||
- turn alert listener behavior
|
||||
- room camera snapshot socket/feed pattern
|
||||
- `RoomCameraFeed` for slow preview display
|
||||
- replay source catalog and ffmpeg workers
|
||||
- existing control input concepts, but with a PTZ-specific command pipeline
|
||||
|
||||
Do not directly merge PTZ into `roomCameraService` or `commandService`. PTZ should have its own service boundary because it has ownership, queueing, ONVIF control, video authorization, and camera-specific state.
|
||||
|
||||
## operator UI and controls
|
||||
|
||||
The VIP tab should get a PTZ camera card. The card should be technical/utilitarian and match the existing site style. It should show:
|
||||
- current camera operator
|
||||
- queue/turn state
|
||||
- turn time remaining when relevant
|
||||
- whether the current user can claim or must wait
|
||||
- whether the current user must dock and charge before switching
|
||||
- a slow snapshot preview
|
||||
- a button to open the fullscreen PTZ controller when the user is the active operator
|
||||
|
||||
The fullscreen PTZ controller should take over the whole app surface while open. It should not feel like a normal side panel. When active, the user is in camera-operation mode, not rover-driving mode.
|
||||
|
||||
Desktop controls:
|
||||
- movement input pans and tilts the camera
|
||||
- camera up/down or equivalent camera tilt controls zoom in/out
|
||||
- available special controls expose only what the one-time ONVIF probe proved exists
|
||||
- if ONVIF exposes presets/home, provide those controls
|
||||
- if ONVIF exposes spotlight, IR, or night mode, provide those controls
|
||||
- if those features are not exposed through ONVIF, leave them out until a Reolink-specific fallback is intentionally added
|
||||
- include a compact right-side status/control panel with operator, queue, camera state, and available controls
|
||||
|
||||
Mobile controls:
|
||||
- reuse the existing mobile control layout concept where practical
|
||||
- relabel/re-map rover movement controls for PTZ movement
|
||||
- keep the camera view as the main screen
|
||||
- use the existing mobile control columns/pads as inspiration, but send PTZ commands instead of rover commands
|
||||
|
||||
Input/control implementation:
|
||||
- do not send PTZ through the existing rover `commandService`
|
||||
- create PTZ-specific socket events/handlers owned by the PTZ camera service
|
||||
- use a PTZ-specific client command pipeline that maps existing input intent into PTZ commands
|
||||
- server must enforce that only the active PTZ operator can send movement/zoom/control commands
|
||||
- client-side input interception is only for UX; server-side operator checks are the real authority
|
||||
- all movement controls should send stop commands on key/button release, blur, disconnect, controller close, or turn loss
|
||||
|
||||
## NEW UI STUFF
|
||||
- desktop:
|
||||
- sidebar like there is now
|
||||
- replay panel in sidebar
|
||||
- better indicators of light and OR modes
|
||||
- list of controls using keybind things
|
||||
- mobile:
|
||||
- one sidebar on the right
|
||||
- reuse rover drive control panel for camera movement
|
||||
- reuse gpio toggle buttons for spotlight and IR
|
||||
- reuse camera tilt slider for zoom
|
||||
- relabeled variants where needed for reused mobile controls
|
||||
- scroll sidebar down to see replay panel
|
||||
- both:
|
||||
- the VIP panel
|
||||
- sucks.
|
||||
- wasted space
|
||||
- put snapshot and everything else side by side
|
||||
- add a display of ptz's turn queue
|
||||
- camera should open when you request control over it. no need to have it be another button to press
|
||||
- should show state of camera
|
||||
- the fullscreen interface
|
||||
- should be inside a cardframe, with no title bar
|
||||
- reuse anything whereever possible
|
||||
- needs to be ACTUALLY FULLSCREEN, not with space around the edges anywhere
|
||||
- needs to match the global styling, and use cardframes internally for stuff.
|
||||
@@ -0,0 +1,33 @@
|
||||
- make ptz camera better integrated
|
||||
- keep current fullscreen interface, its good.
|
||||
- but remove the card from the vip panel
|
||||
- clean up the fullscreen interface to match the rest of the page better
|
||||
- have a clear close button
|
||||
- on desktop, have some stuff in sidebar and some stuff below the video
|
||||
- video should keep the rest of the space
|
||||
- reuse rover HUD elements like chat input and not your turn indicator
|
||||
- probably make it so that the rest of the page unmounts or unloads or whatever when youre in it
|
||||
- make it feel more like youre switching to a different rover instead of switching to a completely different thing
|
||||
- simplify and reuse components wherever possible, frontend and backend
|
||||
- right now, it feels tacked on, badly integrated, and incomplete
|
||||
- needs a much better UI flow
|
||||
- still needs to be a VIP feature
|
||||
- for users on ptz, make their chats have a rover badge that has the ptz name and a color
|
||||
- make the cam show up as a room camera in the room camera panel
|
||||
- snapshot mode only
|
||||
- make the ptz queue and join button show up as one roverqueuespanel style rover row below the links panel
|
||||
- only show it open for verified users, for non-verified users overlay it with a message and dont let them click on it
|
||||
- for mobile layouts, show it below the roverqueuepanel.
|
||||
- dont worry about not being invasive, just dont break anything
|
||||
|
||||
## ui flow should be:
|
||||
1. you are verified
|
||||
2. you see the ptz camera queue in the ui, it has 2 people in it
|
||||
3. you click on it, the fullscreen UI opens
|
||||
- other people will see you in the queue in the little panel
|
||||
4. its not your turn yet. the fullscreen UI replaces the page.
|
||||
- you see snapshots, you see the "not your turn" hud, same as driving a rover
|
||||
5. its now your turn. the overlay shows up just as it does in rover hud
|
||||
6. you control the camera like usual, you want to close it
|
||||
7. you hit the close button, you get removed from the queue
|
||||
8. the page returns to normal
|
||||
@@ -0,0 +1,16 @@
|
||||
# make all bandwidth saving options toggleable in one centralized server config
|
||||
- multitab protection mode
|
||||
- allowed
|
||||
- verified only
|
||||
- not allowed
|
||||
- snapshots
|
||||
- non-turn snapshots
|
||||
- on (you see snapshots when its not your turn)
|
||||
- off (everyone gets full video all the time)
|
||||
- non-local spectator snapshots
|
||||
- on (external spectators are only allowed snapshots)
|
||||
- off (all spectators get full video)
|
||||
- external spectator access (new)
|
||||
- off (no one can access the spectate page externally)
|
||||
- on (everyone can access the spectate page externally)
|
||||
- anything else related to bandwidth savings should also get config
|
||||
Executable
+171
@@ -0,0 +1,171 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Chrome Google TTS WAV renderer.
|
||||
|
||||
Purpose: Converts the same local ChromeOS Google TTS assets used by rovers into
|
||||
server-side WAV files that can be handed to another playback transport.
|
||||
Scope: This script only renders one utterance to a file; device playback and
|
||||
camera delivery stay owned by Node services.
|
||||
"""
|
||||
import argparse
|
||||
import ctypes
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import wave
|
||||
|
||||
|
||||
ASSET_ROOT = "/opt/roverd/googletts"
|
||||
LIB_PATH = os.path.join(ASSET_ROOT, "libchrometts.so")
|
||||
VOICE_DIR = os.path.join(ASSET_ROOT, "en-us-x-multi-r30")
|
||||
PIPELINE = "pipeline.pb"
|
||||
SAMPLE_RATE = 24000
|
||||
MAX_TEXT_CHARS = 512
|
||||
|
||||
VOICES = {
|
||||
"sfg": "female",
|
||||
"iob": "female",
|
||||
"iog": "female",
|
||||
"iol": "male",
|
||||
"iom": "male",
|
||||
"tpc": "female",
|
||||
"tpd": "male",
|
||||
"tpf": "female",
|
||||
}
|
||||
DEFAULT_VOICE = "tpf"
|
||||
DEFAULT_PITCH = 1.0
|
||||
DEFAULT_SPEED = 1.0
|
||||
MIN_PITCH = 0.5
|
||||
MAX_PITCH = 2.0
|
||||
MIN_SPEED = 0.5
|
||||
MAX_SPEED = 2.0
|
||||
|
||||
|
||||
def varint(value):
|
||||
out = bytearray()
|
||||
while value >= 0x80:
|
||||
out.append((value & 0x7F) | 0x80)
|
||||
value >>= 7
|
||||
out.append(value)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def field_bytes(number, payload):
|
||||
return varint((number << 3) | 2) + varint(len(payload)) + payload
|
||||
|
||||
|
||||
def field_float(number, value):
|
||||
return varint((number << 3) | 5) + struct.pack("<f", float(value))
|
||||
|
||||
|
||||
def build_utterance(text, pitch=1.0, speed=1.0):
|
||||
params = field_float(2, pitch) + field_float(3, speed)
|
||||
msg_b = field_bytes(1, text.encode("utf-8")) + field_bytes(20, params)
|
||||
msg_a = field_bytes(1, msg_b)
|
||||
return field_bytes(1, msg_a)
|
||||
|
||||
|
||||
def build_speaker(name, gender):
|
||||
return field_bytes(1, name.encode("utf-8")) + field_bytes(2, gender.encode("utf-8"))
|
||||
|
||||
|
||||
def clamp_float(value, minimum, maximum, fallback):
|
||||
try:
|
||||
value = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return fallback
|
||||
if value <= 0:
|
||||
return fallback
|
||||
if value < minimum:
|
||||
return minimum
|
||||
if value > maximum:
|
||||
return maximum
|
||||
return value
|
||||
|
||||
|
||||
def float_to_s16le(samples):
|
||||
pcm = bytearray()
|
||||
for sample in samples:
|
||||
clipped = max(-1.0, min(1.0, float(sample)))
|
||||
pcm.extend(struct.pack("<h", int(clipped * 32767)))
|
||||
return bytes(pcm)
|
||||
|
||||
|
||||
class ChromeTTS:
|
||||
def __init__(self):
|
||||
self.lib = ctypes.CDLL(LIB_PATH)
|
||||
self.lib.GoogleTtsInit.argtypes = [ctypes.c_char_p, ctypes.c_char_p]
|
||||
self.lib.GoogleTtsInit.restype = ctypes.c_bool
|
||||
self.lib.GoogleTtsInitBuffered.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int, ctypes.c_int]
|
||||
self.lib.GoogleTtsInitBuffered.restype = ctypes.c_bool
|
||||
self.lib.GoogleTtsGetFramesInAudioBuffer.argtypes = []
|
||||
self.lib.GoogleTtsGetFramesInAudioBuffer.restype = ctypes.c_size_t
|
||||
self.lib.GoogleTtsReadBuffered.argtypes = [
|
||||
ctypes.POINTER(ctypes.c_float),
|
||||
ctypes.POINTER(ctypes.c_size_t),
|
||||
]
|
||||
self.lib.GoogleTtsReadBuffered.restype = ctypes.c_int
|
||||
self.lib.GoogleTtsShutdown.argtypes = []
|
||||
self.lib.GoogleTtsShutdown.restype = None
|
||||
|
||||
voice_dir = os.path.abspath(VOICE_DIR) + os.sep
|
||||
pipeline = os.path.join(voice_dir, PIPELINE)
|
||||
if not self.lib.GoogleTtsInit(pipeline.encode("utf-8"), voice_dir.encode("utf-8")):
|
||||
raise RuntimeError("GoogleTtsInit failed")
|
||||
self.frames = int(self.lib.GoogleTtsGetFramesInAudioBuffer())
|
||||
if self.frames <= 0:
|
||||
raise RuntimeError("invalid Google TTS audio buffer size")
|
||||
self.buffer = (ctypes.c_float * self.frames)()
|
||||
|
||||
def render_wav(self, text, output_path, voice, pitch=DEFAULT_PITCH, speed=DEFAULT_SPEED):
|
||||
voice = voice if voice in VOICES else DEFAULT_VOICE
|
||||
pitch = clamp_float(pitch, MIN_PITCH, MAX_PITCH, DEFAULT_PITCH)
|
||||
speed = clamp_float(speed, MIN_SPEED, MAX_SPEED, DEFAULT_SPEED)
|
||||
text = text.strip()
|
||||
if not text:
|
||||
raise ValueError("text required")
|
||||
text = text[:MAX_TEXT_CHARS]
|
||||
|
||||
utterance = build_utterance(text, pitch=pitch, speed=speed)
|
||||
speaker = build_speaker(voice, VOICES[voice])
|
||||
if not self.lib.GoogleTtsInitBuffered(utterance, speaker, len(utterance), len(speaker)):
|
||||
raise RuntimeError("GoogleTtsInitBuffered failed")
|
||||
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output_path)), exist_ok=True)
|
||||
with wave.open(output_path, "wb") as wav:
|
||||
wav.setnchannels(1)
|
||||
wav.setsampwidth(2)
|
||||
wav.setframerate(SAMPLE_RATE)
|
||||
frames_written = ctypes.c_size_t(0)
|
||||
while self.lib.GoogleTtsReadBuffered(self.buffer, ctypes.byref(frames_written)) > 0:
|
||||
count = int(frames_written.value)
|
||||
if count > 0:
|
||||
wav.writeframes(float_to_s16le(self.buffer[:count]))
|
||||
|
||||
def shutdown(self):
|
||||
self.lib.GoogleTtsShutdown()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Render Chrome Google TTS to a WAV file.")
|
||||
parser.add_argument("--text", required=True)
|
||||
parser.add_argument("--voice", default=DEFAULT_VOICE)
|
||||
parser.add_argument("--pitch", type=float, default=DEFAULT_PITCH)
|
||||
parser.add_argument("--speed", type=float, default=DEFAULT_SPEED)
|
||||
parser.add_argument("--output", required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
tts = ChromeTTS()
|
||||
try:
|
||||
tts.render_wav(args.text, args.output, args.voice, args.pitch, args.speed)
|
||||
finally:
|
||||
tts.shutdown()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
raise SystemExit(main())
|
||||
except Exception as exc:
|
||||
sys.stderr.write(f"chromegtts-wav failed: {exc}\n")
|
||||
raise SystemExit(1)
|
||||
@@ -128,6 +128,22 @@ roomCameras:
|
||||
url: "http://192.168.0.51/snapshot.jpg"
|
||||
streamUrl: "http://192.168.0.51/stream.mjpg"
|
||||
|
||||
ptzCamera:
|
||||
enabled: false
|
||||
name: "PTZ Camera"
|
||||
host: "192.168.0.8"
|
||||
onvifPort: 8000
|
||||
username: "admin"
|
||||
password: "REPLACE_WITH_CAMERA_PASSWORD"
|
||||
# The Reolink TrackMix autotrack profile was token 003 during commissioning.
|
||||
# Keeping this configurable lets firmware/profile resets be fixed without code
|
||||
# changes while the integration still remains a single-camera feature.
|
||||
profileToken: "003"
|
||||
turnDurationMs: 300000
|
||||
# PTZ replay capture needs a known-good replay encoder on the server. Keep it
|
||||
# off by default so adding live PTZ does not start a broken replay worker loop.
|
||||
replayEnabled: false
|
||||
|
||||
kinect:
|
||||
enabled: false
|
||||
# Capture requests are global across 3d/color so one person cannot spam room
|
||||
|
||||
@@ -26,6 +26,7 @@ require('./src/services/overseerControlService');
|
||||
require('./src/services/globalObjectiveService');
|
||||
require('./src/services/serverControlService');
|
||||
require('./src/services/videoSessions');
|
||||
require('./src/services/ptzCameraService');
|
||||
require('./src/services/videoAuthService');
|
||||
require('./src/services/videoSocketService');
|
||||
require('./src/services/roomCameraService');
|
||||
|
||||
@@ -2,8 +2,12 @@
|
||||
set -euo pipefail
|
||||
|
||||
MEDIAMTX_VERSION="1.15.3"
|
||||
NEOLINK_VERSION="0.6.2"
|
||||
MEDIAMTX_BASE_URL="https://github.com/bluenviron/mediamtx/releases/download/v${MEDIAMTX_VERSION}"
|
||||
NEOLINK_BASE_URL="https://github.com/QuantumEntangledAndy/neolink/releases/download/v${NEOLINK_VERSION}"
|
||||
MEDIAMTX_BIN="/usr/local/bin/mediamtx"
|
||||
NEOLINK_BIN="/usr/local/bin/neolink"
|
||||
CHROMEGTTS_WAV_BIN="/usr/local/bin/chromegtts-wav"
|
||||
MEDIAMTX_CONF_DIR="/etc/mediamtx"
|
||||
MEDIAMTX_CONFIG="$MEDIAMTX_CONF_DIR/mediamtx.yml"
|
||||
ROVER_SNAPSHOT_WRITER_BIN="/usr/local/bin/rover-snapshot-writer.sh"
|
||||
@@ -29,6 +33,79 @@ SERVER_DIR="$SCRIPT_DIR"
|
||||
CONFIG_PATH="$SERVER_DIR/config.yaml"
|
||||
MEDIAMTX_TEMPLATE="$SERVER_DIR/mediamtx/mediamtx.yml"
|
||||
ROVER_SNAPSHOT_WRITER_TEMPLATE="$SERVER_DIR/mediamtx/rover-snapshot-writer.sh"
|
||||
CHROMEGTTS_WAV_TEMPLATE="$SERVER_DIR/bin/chromegtts-wav.py"
|
||||
|
||||
install_google_tts_assets() {
|
||||
local asset_dir="/opt/roverd/googletts"
|
||||
local voice_dir="${asset_dir}/en-us-x-multi-r30"
|
||||
local dist_url="https://storage.googleapis.com/chromeos-localmirror/distfiles/googletts-26.5.tar.xz"
|
||||
local lib_member=""
|
||||
local arch_name
|
||||
arch_name=$(uname -m)
|
||||
|
||||
# The PTZ camera is not a rover, so Google speech must be synthesized on the
|
||||
# server before neolink sends a WAV to the camera. These assets are the same
|
||||
# offline ChromeOS local TTS assets that rover installers already use; keeping
|
||||
# the layout identical lets the server helper and rover daemon share loader
|
||||
# assumptions.
|
||||
if [[ -f "${asset_dir}/libchrometts.so" && -f "${voice_dir}/pipeline.pb" ]]; then
|
||||
echo " Google TTS assets already installed"
|
||||
return
|
||||
fi
|
||||
|
||||
case "$arch_name" in
|
||||
x86_64|amd64)
|
||||
lib_member="libchrometts_x86_64.so"
|
||||
;;
|
||||
aarch64)
|
||||
lib_member="libchrometts_arm64.so"
|
||||
;;
|
||||
armv7l|armv6l)
|
||||
lib_member="libchrometts_armv7.so"
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported Google TTS architecture: $arch_name" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo " Installing Google TTS assets -> $asset_dir"
|
||||
curl -L -o "$tmpdir/googletts-26.5.tar.xz" "$dist_url"
|
||||
tar -xf "$tmpdir/googletts-26.5.tar.xz" -C "$tmpdir" en-us-x-multi.zvoice "$lib_member"
|
||||
install -d -o root -g root -m 0755 "$asset_dir"
|
||||
install -o root -g root -m 0644 "$tmpdir/$lib_member" "${asset_dir}/libchrometts.so"
|
||||
rm -rf "$voice_dir"
|
||||
install -d -o root -g root -m 0755 "$voice_dir"
|
||||
# The .zvoice member is a zip archive inside the outer tar.xz. Match the
|
||||
# rover installers here; trying to untar it fails after the large download.
|
||||
unzip -q "$tmpdir/en-us-x-multi.zvoice" -d "$voice_dir"
|
||||
chown -R root:root "$asset_dir"
|
||||
find "$asset_dir" -type d -exec chmod 0755 {} +
|
||||
find "$asset_dir" -type f -exec chmod 0644 {} +
|
||||
}
|
||||
|
||||
verify_google_tts_helper() {
|
||||
local smoke_wav="$tmpdir/chromegtts-smoke.wav"
|
||||
|
||||
echo " Verifying Chrome Google TTS helper"
|
||||
# libchrometts is a native ChromeOS library. Rendering one tiny WAV during
|
||||
# install catches missing shared-library dependencies, bad asset extraction,
|
||||
# and helper path mistakes before multirover.service starts accepting PTZ TTS
|
||||
# requests that would fail later in logs.
|
||||
if ! "$CHROMEGTTS_WAV_BIN" \
|
||||
--text "test" \
|
||||
--voice tpf \
|
||||
--pitch 1 \
|
||||
--speed 1 \
|
||||
--output "$smoke_wav"; then
|
||||
echo "Chrome Google TTS helper smoke render failed." >&2
|
||||
return 1
|
||||
fi
|
||||
if [[ ! -s "$smoke_wav" ]]; then
|
||||
echo "Chrome Google TTS helper did not create a WAV file." >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
echo "[1/6] Installing dependencies..."
|
||||
# The Kinect tooling uses a native libfreenect worker/probe rather than a
|
||||
@@ -40,9 +117,21 @@ dnf install -y \
|
||||
npm \
|
||||
curl \
|
||||
tar \
|
||||
unzip \
|
||||
xz \
|
||||
gcc-c++ \
|
||||
make \
|
||||
pkgconf-pkg-config \
|
||||
flite \
|
||||
espeak \
|
||||
python3 \
|
||||
libcxx \
|
||||
libcxxabi \
|
||||
gstreamer1 \
|
||||
gstreamer1-plugins-base \
|
||||
gstreamer1-plugins-good \
|
||||
gstreamer1-plugins-bad-free \
|
||||
gstreamer1-rtsp-server \
|
||||
libfreenect \
|
||||
libfreenect-devel \
|
||||
libusb1-devel >/dev/null
|
||||
@@ -62,6 +151,13 @@ EOF
|
||||
chmod 644 "$KINECT_UDEV_RULE"
|
||||
udevadm control --reload-rules
|
||||
|
||||
if [[ ! -f "$CHROMEGTTS_WAV_TEMPLATE" ]]; then
|
||||
echo "Chrome Google TTS WAV helper missing at $CHROMEGTTS_WAV_TEMPLATE" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo " Installing Chrome Google TTS WAV helper -> $CHROMEGTTS_WAV_BIN"
|
||||
install -m 0755 "$CHROMEGTTS_WAV_TEMPLATE" "$CHROMEGTTS_WAV_BIN"
|
||||
|
||||
echo "[2/6] Installing Node production deps..."
|
||||
runuser -u "$TARGET_USER" -- bash -c "cd '$SERVER_DIR' && npm install --production"
|
||||
|
||||
@@ -83,12 +179,15 @@ arch=$(uname -m)
|
||||
case "$arch" in
|
||||
x86_64|amd64)
|
||||
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_amd64.tar.gz"
|
||||
neolink_pkg="neolink_linux_x86_64_ubuntu.zip"
|
||||
;;
|
||||
aarch64)
|
||||
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_arm64.tar.gz"
|
||||
neolink_pkg="neolink_linux_arm64.zip"
|
||||
;;
|
||||
armv7l)
|
||||
mediamtx_pkg="mediamtx_v${MEDIAMTX_VERSION}_linux_armv7.tar.gz"
|
||||
neolink_pkg="neolink_linux_armhf.zip"
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported architecture: $arch" >&2
|
||||
@@ -101,6 +200,26 @@ curl -L "$MEDIAMTX_BASE_URL/$mediamtx_pkg" -o "$tmpdir/mediamtx.tgz"
|
||||
tar -xzf "$tmpdir/mediamtx.tgz" -C "$tmpdir" mediamtx
|
||||
install -m 0755 "$tmpdir/mediamtx" "$MEDIAMTX_BIN"
|
||||
|
||||
echo " Installing neolink ${NEOLINK_VERSION} -> $NEOLINK_BIN"
|
||||
curl -L "$NEOLINK_BASE_URL/$neolink_pkg" -o "$tmpdir/neolink.zip"
|
||||
unzip -q "$tmpdir/neolink.zip" -d "$tmpdir/neolink"
|
||||
neolink_extracted=$(find "$tmpdir/neolink" -type f -name neolink -perm /111 | head -n 1)
|
||||
if [[ -z "$neolink_extracted" ]]; then
|
||||
neolink_extracted=$(find "$tmpdir/neolink" -type f -name neolink | head -n 1)
|
||||
fi
|
||||
if [[ -z "$neolink_extracted" ]]; then
|
||||
echo "neolink binary missing from $neolink_pkg" >&2
|
||||
exit 1
|
||||
fi
|
||||
install -m 0755 "$neolink_extracted" "$NEOLINK_BIN"
|
||||
install_google_tts_assets
|
||||
if ! verify_google_tts_helper; then
|
||||
echo " Reinstalling Google TTS assets after failed verification"
|
||||
rm -rf /opt/roverd/googletts
|
||||
install_google_tts_assets
|
||||
verify_google_tts_helper
|
||||
fi
|
||||
|
||||
mkdir -p "$MEDIAMTX_CONF_DIR"
|
||||
if [[ ! -f "$MEDIAMTX_TEMPLATE" ]]; then
|
||||
echo "mediaMTX template missing at $MEDIAMTX_TEMPLATE" >&2
|
||||
|
||||
@@ -16,10 +16,24 @@ esac
|
||||
|
||||
mkdir -p "$SNAP_DIR"
|
||||
|
||||
FILTER="fps=1"
|
||||
QUALITY="6"
|
||||
|
||||
case "$PATH_NAME" in
|
||||
ptz-camera)
|
||||
# PTZ snapshots are shown to non-operators specifically to avoid sending the
|
||||
# full live video stream. The PTZ publisher is full-resolution 16:9 video,
|
||||
# so resize the JPEGs at the snapshot writer boundary before Node ever reads
|
||||
# and fans them out over Socket.IO.
|
||||
FILTER="fps=1,scale=480:-2"
|
||||
QUALITY="10"
|
||||
;;
|
||||
esac
|
||||
|
||||
exec ffmpeg -hide_banner -loglevel warning -nostdin -y \
|
||||
-i "srt://127.0.0.1:9000?streamid=read:${PATH_NAME}" \
|
||||
-an \
|
||||
-vf fps=1 \
|
||||
-q:v 6 \
|
||||
-vf "$FILTER" \
|
||||
-q:v "$QUALITY" \
|
||||
-update 1 \
|
||||
"${SNAP_DIR}/${PATH_NAME}.jpg"
|
||||
|
||||
@@ -19,6 +19,8 @@
|
||||
"morgan": "^1.10.0",
|
||||
"obscenity": "^0.4.6",
|
||||
"ollama": "^0.6.3",
|
||||
"onvif": "^0.8.1",
|
||||
"reolink-nvr-api": "^0.3.0",
|
||||
"sharp": "^0.33.5",
|
||||
"socket.io": "^4.7.5",
|
||||
"uuid": "^9.0.1",
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -78,8 +78,8 @@
|
||||
<script defer src="https://analytics.otter.land/script.js" data-website-id="82dd56a5-db44-4279-bd1e-a4d9fee39af7" data-domains="rover.otter.land"></script>
|
||||
<script defer src="https://analytics.otter.land/recorder.js" data-website-id="82dd56a5-db44-4279-bd1e-a4d9fee39af7" data-domains="rover.otter.land" data-sample-rate="0.15" data-mask-level="moderate" data-max-duration="300000"></script>
|
||||
<title>Roomba Rover</title>
|
||||
<script type="module" crossorigin src="/assets/index-htN8tzuK.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-BfZ_TWsb.css">
|
||||
<script type="module" crossorigin src="/assets/index-D5vPzVhj.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-BN3kEVFL.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
|
||||
@@ -49,6 +49,7 @@ function buildFeatureFlags(config = loadConfig()) {
|
||||
const barcodeGamesConfig = config.barcodeGames || {};
|
||||
const socialsConfig = config.socials || {};
|
||||
const interInstanceConfig = config.interInstance || {};
|
||||
const ptzCameraConfig = config.ptzCamera || {};
|
||||
const homeAssistant = Boolean(
|
||||
asBoolean(homeAssistantConfig.enabled) &&
|
||||
asTrimmedString(homeAssistantConfig.url) &&
|
||||
@@ -80,6 +81,12 @@ function buildFeatureFlags(config = loadConfig()) {
|
||||
),
|
||||
socials: Boolean(asBoolean(socialsConfig.enabled) && getConfiguredSocials(config).length > 0),
|
||||
interInstance: asBoolean(interInstanceConfig.enabled),
|
||||
ptzCamera: Boolean(
|
||||
asBoolean(ptzCameraConfig.enabled) &&
|
||||
asTrimmedString(ptzCameraConfig.host) &&
|
||||
asTrimmedString(ptzCameraConfig.username) &&
|
||||
asTrimmedString(ptzCameraConfig.password),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -7,8 +7,15 @@ const roverManager = require('../roverManager');
|
||||
const { getRole } = require('../roleService');
|
||||
const { describeAssignment } = require('../assignmentService');
|
||||
const { getNickname } = require('../nicknameService');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
|
||||
function resolvePtzChatTarget(socketId) {
|
||||
return ptzCameraService.getChatTargetForSocket(socketId) || null;
|
||||
}
|
||||
|
||||
function resolveRoverId(socketId) {
|
||||
const ptzTarget = resolvePtzChatTarget(socketId);
|
||||
if (ptzTarget?.roverId) return ptzTarget.roverId;
|
||||
const primary = roverManager.getPrimaryRoverForSocket(socketId);
|
||||
if (primary) return primary;
|
||||
const assignment = describeAssignment(socketId);
|
||||
@@ -17,13 +24,54 @@ function resolveRoverId(socketId) {
|
||||
|
||||
function resolveRoverColor(roverId) {
|
||||
if (!roverId) return null;
|
||||
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
|
||||
return ptzCameraService.getPublicState()?.color || null;
|
||||
}
|
||||
const record = roverManager.rovers.get(String(roverId));
|
||||
return record?.meta?.color || null;
|
||||
}
|
||||
|
||||
function resolveRoverName(roverId) {
|
||||
if (!roverId) return null;
|
||||
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) {
|
||||
return ptzCameraService.getPublicState()?.name || null;
|
||||
}
|
||||
const record = roverManager.rovers.get(String(roverId));
|
||||
return record?.meta?.name || null;
|
||||
}
|
||||
|
||||
function isPtzChatTargetId(roverId) {
|
||||
/*
|
||||
PTZ is intentionally treated as a virtual rover for chat identity only. It
|
||||
does not live in roverManager.rovers because movement, video authorization,
|
||||
and queue ownership are PTZ-service concerns, but chat needs one stable
|
||||
"rover-like" id so the existing web UI, Discord bridge, and AI transcript
|
||||
code can all render the same badge without learning PTZ internals.
|
||||
*/
|
||||
return Boolean(roverId) && String(roverId) === ptzCameraService.PTZ_CAMERA_ID;
|
||||
}
|
||||
|
||||
function isPublicChatTargetId(roverId, socket = null) {
|
||||
if (!roverId) return false;
|
||||
/*
|
||||
Normal rovers remain governed by the existing replay visibility rule, which
|
||||
is also the rule chat historically used to avoid exposing closed private
|
||||
rover activity. PTZ gets an explicit allow-list entry here because it is a
|
||||
public chat target that deliberately pretends to be a rover, even though it
|
||||
is not a roverManager record.
|
||||
*/
|
||||
if (isPtzChatTargetId(roverId)) return true;
|
||||
return roverManager.canReplayRoverId(roverId, socket) === true;
|
||||
}
|
||||
|
||||
function isPrivateClosedRoverId(roverId) {
|
||||
if (!roverId) return false;
|
||||
return roverManager.canReplayRoverId(roverId) !== true;
|
||||
/*
|
||||
PTZ uses the existing rover badge fields so chat rows can reuse RoverLabel,
|
||||
but it is not a private rover. Let PTZ-badged messages broadcast normally
|
||||
instead of falling into the closed-private rover path for unknown ids.
|
||||
*/
|
||||
return !isPublicChatTargetId(roverId);
|
||||
}
|
||||
|
||||
function normalizeProfileImageUrl(value) {
|
||||
@@ -100,6 +148,7 @@ function buildRoverCtxSnapshot(roverId) {
|
||||
function buildMessage(socket, text, meta = {}) {
|
||||
const roverId = meta.roverId || resolveRoverId(socket?.id);
|
||||
const roverColor = meta.roverColor ?? resolveRoverColor(roverId);
|
||||
const roverName = meta.roverName ?? resolveRoverName(roverId);
|
||||
const toolCalls = Array.isArray(meta.toolCalls)
|
||||
? meta.toolCalls
|
||||
.map((entry) => {
|
||||
@@ -122,6 +171,7 @@ function buildMessage(socket, text, meta = {}) {
|
||||
nickname: meta.nickname || getNickname(socket) || null,
|
||||
role: meta.role || getRole(socket),
|
||||
roverId,
|
||||
roverName,
|
||||
roverColor,
|
||||
fromDiscord: Boolean(meta.fromDiscord),
|
||||
discordGuildId: meta.discordGuildId || null,
|
||||
@@ -143,6 +193,7 @@ function buildMessage(socket, text, meta = {}) {
|
||||
function buildTypingPayload(socket, meta = {}) {
|
||||
const roverId = meta.roverId || resolveRoverId(socket?.id);
|
||||
const roverColor = meta.roverColor ?? resolveRoverColor(roverId);
|
||||
const roverName = meta.roverName ?? resolveRoverName(roverId);
|
||||
const socketId = socket?.id || null;
|
||||
const fromDiscord = Boolean(meta.fromDiscord);
|
||||
let typingId = meta.typingId || null;
|
||||
@@ -165,6 +216,7 @@ function buildTypingPayload(socket, meta = {}) {
|
||||
nickname: meta.nickname || getNickname(socket) || null,
|
||||
role: meta.role || getRole(socket),
|
||||
roverId,
|
||||
roverName,
|
||||
roverColor,
|
||||
fromDiscord,
|
||||
discordGuildId: meta.discordGuildId || null,
|
||||
@@ -179,6 +231,8 @@ function buildTypingPayload(socket, meta = {}) {
|
||||
|
||||
module.exports = {
|
||||
resolveRoverId,
|
||||
isPtzChatTargetId,
|
||||
isPublicChatTargetId,
|
||||
isPrivateClosedRoverId,
|
||||
buildRoverCtxSnapshot,
|
||||
buildMessage,
|
||||
|
||||
@@ -7,6 +7,7 @@ const { getRole } = require('../roleService');
|
||||
const roverManager = require('../roverManager');
|
||||
const { issueCommand } = require('../commandService');
|
||||
const { getAdminReason } = require('../adminReasonService');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
const {
|
||||
TYPING_NOTE_DURATION,
|
||||
ACCESS_NOTICE_COOLDOWN_MS,
|
||||
@@ -18,6 +19,13 @@ const { getLastAccessNoticeAt, setLastAccessNoticeAt } = require('./state');
|
||||
|
||||
function playTypingNote(roverId, note, socketId) {
|
||||
if (!roverId) return;
|
||||
/*
|
||||
PTZ borrows the roverId field for chat badges, but it has no rover command
|
||||
channel. Skipping the song command here keeps PTZ chat from producing noisy
|
||||
"unknown rover" command attempts while still allowing the message itself to
|
||||
behave like rover chat everywhere else.
|
||||
*/
|
||||
if (String(roverId) === ptzCameraService.PTZ_CAMERA_ID) return;
|
||||
try {
|
||||
issueCommand(roverId, {
|
||||
type: 'song',
|
||||
@@ -83,6 +91,22 @@ function maybeSendAccessNotice(message, sendSystemMessage) {
|
||||
|
||||
function maybeSpeak(socket, message, ttsOptions) {
|
||||
if (!ttsOptions || !message?.roverId) return;
|
||||
if (String(message.roverId) === ptzCameraService.PTZ_CAMERA_ID) {
|
||||
/*
|
||||
PTZ has no rover websocket, but it does have a real speaker behind the
|
||||
Reolink/neolink path. Keep PTZ routing here so chat remains the single
|
||||
place that decides whether a user's message should produce speech, while
|
||||
ptzCameraService owns camera-specific permissions and playback details.
|
||||
*/
|
||||
ptzCameraService.speakText(message.text, ttsOptions, socket)
|
||||
.then(() => {
|
||||
logger.info('PTZ TTS sent', { engine: ttsOptions.engine, socket: socket.id });
|
||||
})
|
||||
.catch((err) => {
|
||||
logger.warn('PTZ TTS send failed', { error: err.message, socket: socket.id });
|
||||
});
|
||||
return;
|
||||
}
|
||||
const record = roverManager.rovers.get(message.roverId);
|
||||
const ttsEnabled = Boolean(record?.meta?.audio?.ttsEnabled);
|
||||
if (!ttsEnabled) return;
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
// Purpose: Bridges chat and typing between Discord and site sockets.
|
||||
// Scope: Handles inbound Discord messages plus outbound webhook and typing relay.
|
||||
const { WebhookClient } = require('discord.js');
|
||||
const { isPublicChatTargetId } = require('../../chatService/contextBuilders');
|
||||
|
||||
function summarizeToolCall(entry = {}) {
|
||||
const tool = String(entry?.tool || 'unknown');
|
||||
@@ -23,7 +24,6 @@ function createChatBridgeHandlers(deps) {
|
||||
const {
|
||||
logger,
|
||||
client,
|
||||
roverManager,
|
||||
getGuildConfig,
|
||||
listGuildConfigs,
|
||||
sendExternalMessage,
|
||||
@@ -59,7 +59,13 @@ function createChatBridgeHandlers(deps) {
|
||||
function handleChatBridgeOutbound(event) {
|
||||
const payload = event?.payload;
|
||||
if (!payload) return;
|
||||
if (payload?.roverId && !roverManager.canReplayRoverId(payload.roverId)) return;
|
||||
/*
|
||||
Outbound bridge filtering must use chat visibility, not rover replay
|
||||
visibility. PTZ deliberately uses roverId: "ptz-camera" so the existing
|
||||
chat badge path can be reused, but that id is not a roverManager rover and
|
||||
would be dropped by canReplayRoverId().
|
||||
*/
|
||||
if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return;
|
||||
const guildConfigs = listGuildConfigs();
|
||||
if (!guildConfigs.length) return;
|
||||
|
||||
@@ -96,7 +102,11 @@ function createChatBridgeHandlers(deps) {
|
||||
function handleChatTypingOutbound(event) {
|
||||
const payload = event?.payload;
|
||||
if (!payload || payload.fromDiscord) return;
|
||||
if (payload?.roverId && !roverManager.canReplayRoverId(payload.roverId)) return;
|
||||
/*
|
||||
Typing indicators follow the same public-chat-target rule as messages so
|
||||
PTZ users do not look present in web chat while disappearing from Discord.
|
||||
*/
|
||||
if (payload?.roverId && !isPublicChatTargetId(payload.roverId)) return;
|
||||
const guildConfigs = listGuildConfigs();
|
||||
if (!guildConfigs.length) return;
|
||||
|
||||
|
||||
@@ -24,7 +24,14 @@ function formatWebhookUsername(payload) {
|
||||
const origin = payload.discordGuildName ? ` (From: ${payload.discordGuildName})` : '';
|
||||
return `${name}${origin}${botTag}${spectatorTag}${adminTag}`;
|
||||
}
|
||||
const roverTag = payload.roverId ? ` [${payload.roverId}]` : '';
|
||||
/*
|
||||
The chat payload already carries the resolved display name for rover-like
|
||||
targets. Prefer that name so PTZ, which is intentionally pretending to be a
|
||||
rover in chat, shows up as "PTZ Camera" instead of the internal id
|
||||
"ptz-camera"; fall back to the id for older payloads or missing metadata.
|
||||
*/
|
||||
const roverTagLabel = payload.roverName || payload.roverId;
|
||||
const roverTag = payload.roverId ? ` [${roverTagLabel}]` : '';
|
||||
return `${name}${botTag}${spectatorTag}${adminTag}${roverTag}`;
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ const { issueCommand } = require('../commandService');
|
||||
const homeAssistantService = require('../homeAssistantService');
|
||||
const neatoService = require('../neatoService');
|
||||
const liftService = require('../liftService');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
const {
|
||||
HEADLIGHT_DISABLE_ACTION,
|
||||
LASER_DISABLE_ACTION,
|
||||
@@ -103,6 +104,17 @@ async function disableAllRoverLasers() {
|
||||
return { action: 'disableRoverLasers', attempted, failed };
|
||||
}
|
||||
|
||||
async function disablePtzEmitters() {
|
||||
/*
|
||||
The PTZ camera has its own light APIs and ownership rules, so the idle
|
||||
service delegates the actual Reolink calls to ptzCameraService instead of
|
||||
pretending they are rover commands. This keeps idleService responsible only
|
||||
for "idle fired; run cleanup actions" and keeps camera-specific payload
|
||||
details beside the rest of the PTZ integration.
|
||||
*/
|
||||
return ptzCameraService.disableEmittersForIdle();
|
||||
}
|
||||
|
||||
async function sendNeatoHome() {
|
||||
try {
|
||||
await neatoService.sendHome();
|
||||
@@ -126,6 +138,7 @@ const idleActions = [
|
||||
// dockAllRovers,
|
||||
disableAllRoverHeadlights,
|
||||
disableAllRoverLasers,
|
||||
disablePtzEmitters,
|
||||
sendNeatoHome,
|
||||
raiseLift,
|
||||
];
|
||||
|
||||
@@ -1,33 +1,63 @@
|
||||
// Idle Service
|
||||
// Purpose: Triggers a modular idle action pipeline after a sustained no-driver period.
|
||||
// Scope: Observes driver activity events and coordinates timer-based idle automation execution.
|
||||
// Purpose: Triggers a modular idle action pipeline after a sustained no-operator-online period.
|
||||
// Scope: Observes user/admin socket presence and coordinates timer-based idle automation execution.
|
||||
const logger = require('../../globals/logger').child('idleService');
|
||||
const { getActiveDrivers, turnEvents } = require('../turnService');
|
||||
const roverManager = require('../roverManager');
|
||||
const { getRecentDriveActivity } = require('../commandService');
|
||||
const io = require('../../globals/io');
|
||||
const { getRole, roleEvents } = require('../roleService');
|
||||
const { IDLE_TIMEOUT_MS } = require('./constants');
|
||||
const { runtime } = require('./state');
|
||||
const { runIdleActions } = require('./actions');
|
||||
|
||||
function getActivitySnapshot() {
|
||||
const active = getActiveDrivers();
|
||||
const turnCount = active && typeof active === 'object' ? Object.keys(active).length : 0;
|
||||
const activeByTurn = turnCount;
|
||||
let onlineUsers = 0;
|
||||
let onlineAdmins = 0;
|
||||
let onlineSpectators = 0;
|
||||
let onlineIgnored = 0;
|
||||
|
||||
let liveCount = 0;
|
||||
roverManager.rovers.forEach((record) => {
|
||||
if (record?.drivers?.size > 0) liveCount += 1;
|
||||
io.sockets.sockets.forEach((socket) => {
|
||||
const role = getRole(socket);
|
||||
|
||||
/*
|
||||
Idle automation is about whether a real operator is present, not whether
|
||||
a browser tab is merely watching. Spectators can leave the room lights,
|
||||
PTZ emitters, and rovers in their automated idle state because they are
|
||||
intentionally read-only and cannot be the person still using the setup.
|
||||
*/
|
||||
if (role === 'spectator') {
|
||||
onlineSpectators += 1;
|
||||
return;
|
||||
}
|
||||
|
||||
/*
|
||||
Lockdown admins are counted with regular admins because both represent a
|
||||
person with operator-level access who may be supervising the room without
|
||||
actively driving a rover. Plain users also count even before they request
|
||||
control, which is the behavior this service now needs.
|
||||
*/
|
||||
if (role === 'admin' || role === 'lockdown') {
|
||||
onlineAdmins += 1;
|
||||
return;
|
||||
}
|
||||
|
||||
if (role === 'user') {
|
||||
onlineUsers += 1;
|
||||
return;
|
||||
}
|
||||
|
||||
/*
|
||||
Unknown future roles should not accidentally keep automation disabled.
|
||||
If a new role should count as an operator, it should be added explicitly
|
||||
above so this policy remains easy to audit.
|
||||
*/
|
||||
onlineIgnored += 1;
|
||||
});
|
||||
const activeByRoverDrivers = liveCount;
|
||||
|
||||
const recentDriveEvents = getRecentDriveActivity(IDLE_TIMEOUT_MS, { excludeAdmins: false });
|
||||
const activeByRecentDrive = recentDriveEvents.length;
|
||||
|
||||
const totalActive = Math.max(activeByTurn, activeByRoverDrivers, activeByRecentDrive);
|
||||
const totalActive = onlineUsers + onlineAdmins;
|
||||
return {
|
||||
activeByTurn,
|
||||
activeByRoverDrivers,
|
||||
activeByRecentDrive,
|
||||
onlineUsers,
|
||||
onlineAdmins,
|
||||
onlineSpectators,
|
||||
onlineIgnored,
|
||||
totalActive,
|
||||
};
|
||||
}
|
||||
@@ -53,7 +83,7 @@ function scheduleIdleTimer() {
|
||||
runtime.deadlineAt = null;
|
||||
const activity = getActivitySnapshot();
|
||||
if (activity.totalActive > 0) {
|
||||
logger.info('Idle automation skipped; active control detected', activity);
|
||||
logger.info('Idle automation skipped; user or admin online', activity);
|
||||
return;
|
||||
}
|
||||
runtime.lastTriggeredAt = Date.now();
|
||||
@@ -77,8 +107,18 @@ function refreshIdleState() {
|
||||
scheduleIdleTimer();
|
||||
}
|
||||
|
||||
turnEvents.on('activeDriver', refreshIdleState);
|
||||
turnEvents.on('queue', refreshIdleState);
|
||||
io.on('connection', (socket) => {
|
||||
/*
|
||||
A user/admin can be online without ever touching rover controls, so socket
|
||||
presence has to be a first-class idle signal. The disconnect hook is just as
|
||||
important: it is what starts the idle timeout after the last non-spectator
|
||||
leaves, even if no driving event happens around that departure.
|
||||
*/
|
||||
refreshIdleState();
|
||||
socket.on('disconnect', refreshIdleState);
|
||||
});
|
||||
|
||||
roleEvents.on('change', refreshIdleState);
|
||||
|
||||
refreshIdleState();
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
// llm Commentary Service snapshot engine
|
||||
// Purpose: Tracks rover activity/history and builds model snapshot payloads from live rover/chat state.
|
||||
// Scope: Keeps runtime behavior unchanged while isolating sensor aggregation and snapshot assembly logic.
|
||||
const { isPublicChatTargetId } = require('../chatService/contextBuilders');
|
||||
|
||||
function createSnapshotEngine(deps) {
|
||||
const {
|
||||
io,
|
||||
@@ -368,7 +370,12 @@ function createSnapshotEngine(deps) {
|
||||
.filter((entry) => {
|
||||
const roverId = entry?.roverId ? String(entry.roverId) : null;
|
||||
if (!roverId) return true;
|
||||
return roverManager.canReplayRoverId(roverId);
|
||||
/*
|
||||
This is a chat transcript filter, not a physical-rover filter. PTZ
|
||||
chat intentionally carries a rover-like id so transcript consumers can
|
||||
render it consistently, even though roverManager cannot replay that id.
|
||||
*/
|
||||
return isPublicChatTargetId(roverId);
|
||||
});
|
||||
const chatRecent = allRecentMessages
|
||||
.filter((entry) => !entry?.bot)
|
||||
|
||||
@@ -15,6 +15,7 @@ const { getState: getNeatoState, neatoEvents } = neatoService;
|
||||
const { getState: getLiftState, liftEvents } = liftService;
|
||||
const roverManager = require('../roverManager');
|
||||
const { getRecentMessages, sendSystemMessage } = require('../chatService');
|
||||
const { isPublicChatTargetId } = require('../chatService/contextBuilders');
|
||||
const { subscribe } = require('../eventBus');
|
||||
const {
|
||||
PROMPT_PATH,
|
||||
@@ -378,7 +379,12 @@ async function runDecision(triggerReason) {
|
||||
.filter((entry) => Number(entry?.ts || 0) >= runtime.contextResetAt)
|
||||
.filter((entry) => {
|
||||
if (!entry?.roverId) return true;
|
||||
return roverManager.canReplayRoverId(entry.roverId);
|
||||
/*
|
||||
Chat context should preserve every public chat target, including the
|
||||
PTZ virtual rover. Rover replay visibility alone would drop PTZ because
|
||||
it is owned by ptzCameraService instead of roverManager.
|
||||
*/
|
||||
return isPublicChatTargetId(entry.roverId);
|
||||
})
|
||||
.slice(-(MAX_CHAT_CONTEXT + MAX_BOT_CONTEXT));
|
||||
const conversationMessages = buildConversation({ recentMessages: recentConversation, name });
|
||||
|
||||
@@ -0,0 +1,324 @@
|
||||
// PTZ Camera Audio Playback
|
||||
// Purpose: Generates server-side TTS files and sends them to the Reolink TrackMix speaker through neolink.
|
||||
// Scope: Owns file/cache/process details for PTZ speech only; PTZ ownership, chat identity, and camera motion stay in index.js.
|
||||
const crypto = require('crypto');
|
||||
const fs = require('fs');
|
||||
const fsp = require('fs/promises');
|
||||
const path = require('path');
|
||||
const { spawn } = require('child_process');
|
||||
|
||||
const { resolveDataDir } = require('../../helpers/dataPaths');
|
||||
|
||||
const DEFAULT_CAMERA_NAME = 'trackmix';
|
||||
const DEFAULT_MEDIA_PORT = 9000;
|
||||
const DEFAULT_NEOLINK_BIN = '/usr/local/bin/neolink';
|
||||
const DEFAULT_CHROMEGTTS_WAV_BIN = '/usr/local/bin/chromegtts-wav';
|
||||
const DEFAULT_ESPEAK_BIN = 'espeak';
|
||||
const DEFAULT_FLITE_BIN = 'flite';
|
||||
const DEFAULT_VOLUME = 0.7;
|
||||
const MAX_TEXT_CHARS = 512;
|
||||
const PLAYBACK_TIMEOUT_MS = 45000;
|
||||
|
||||
function clampNumber(value, fallback, min, max) {
|
||||
const number = Number(value);
|
||||
if (!Number.isFinite(number)) return fallback;
|
||||
return Math.max(min, Math.min(max, number));
|
||||
}
|
||||
|
||||
function normalizeText(text) {
|
||||
return String(text || '').replace(/\s+/g, ' ').trim().slice(0, MAX_TEXT_CHARS);
|
||||
}
|
||||
|
||||
function normalizeEngine(engine) {
|
||||
const value = String(engine || '').trim().toLowerCase();
|
||||
if (value === 'espeak' || value === 'e') return 'espeak';
|
||||
if (value === 'flite' || value === 'f') return 'flite';
|
||||
if (['chromegtts', 'googletts', 'gtts', 'google'].includes(value)) return 'chromegtts';
|
||||
return 'chromegtts';
|
||||
}
|
||||
|
||||
function tomlString(value) {
|
||||
/*
|
||||
The generated neolink config is intentionally tiny, so JSON string escaping
|
||||
is enough for TOML basic strings and avoids pulling in a TOML writer just to
|
||||
persist four operator-configured values.
|
||||
*/
|
||||
return JSON.stringify(String(value || ''));
|
||||
}
|
||||
|
||||
function cacheKeyFor(text, ttsOptions) {
|
||||
return crypto
|
||||
.createHash('sha256')
|
||||
.update(JSON.stringify({ text, ttsOptions }))
|
||||
.digest('hex')
|
||||
.slice(0, 32);
|
||||
}
|
||||
|
||||
function createPtzAudioPlayback(deps) {
|
||||
const {
|
||||
logger,
|
||||
cameraConfig,
|
||||
enabled,
|
||||
getSocketLabel,
|
||||
} = deps;
|
||||
|
||||
const audioConfig = cameraConfig.audio || {};
|
||||
const audioEnabled = audioConfig.enabled === undefined ? Boolean(enabled) : Boolean(audioConfig.enabled);
|
||||
const dataRoot = path.join(resolveDataDir(), 'ptz-camera-audio');
|
||||
const cacheDir = path.join(dataRoot, 'tts-cache');
|
||||
const configPath = path.join(dataRoot, 'neolink-trackmix.toml');
|
||||
const cameraName = String(audioConfig.neolinkCameraName || DEFAULT_CAMERA_NAME).trim() || DEFAULT_CAMERA_NAME;
|
||||
const mediaPort = Number(audioConfig.mediaPort) || DEFAULT_MEDIA_PORT;
|
||||
const neolinkBin = String(audioConfig.neolinkBin || process.env.NEOLINK_BIN || DEFAULT_NEOLINK_BIN).trim();
|
||||
const chromegttsWavBin = String(
|
||||
audioConfig.chromegttsWavBin || process.env.CHROMEGTTS_WAV_BIN || DEFAULT_CHROMEGTTS_WAV_BIN,
|
||||
).trim();
|
||||
const espeakBin = String(audioConfig.espeakBin || process.env.ESPEAK_BIN || DEFAULT_ESPEAK_BIN).trim();
|
||||
const fliteBin = String(audioConfig.fliteBin || process.env.FLITE_BIN || DEFAULT_FLITE_BIN).trim();
|
||||
const volume = clampNumber(audioConfig.volume, DEFAULT_VOLUME, 0, 4);
|
||||
const fliteDefaultVoice = String(audioConfig.fliteDefaultVoice || 'kal').trim();
|
||||
|
||||
let playbackProc = null;
|
||||
let playbackSeq = 0;
|
||||
|
||||
function getState() {
|
||||
return {
|
||||
enabled: audioEnabled,
|
||||
state: playbackProc ? 'playing' : 'idle',
|
||||
/*
|
||||
This state is sent to browser sessions through ptzCamera public state.
|
||||
Keep it operationally useful without leaking server filesystem layout or
|
||||
binary paths that are only meaningful to the Node process.
|
||||
*/
|
||||
cameraName,
|
||||
volume,
|
||||
};
|
||||
}
|
||||
|
||||
async function ensureNeolinkConfig() {
|
||||
await fsp.mkdir(dataRoot, { recursive: true });
|
||||
const host = String(cameraConfig.host || '').trim();
|
||||
const username = String(cameraConfig.username || '').trim();
|
||||
const password = String(cameraConfig.password || '');
|
||||
if (!host || !username || !password) {
|
||||
throw new Error('PTZ camera host/username/password required for audio playback');
|
||||
}
|
||||
|
||||
const body = [
|
||||
'bind = "127.0.0.1"',
|
||||
'',
|
||||
'[[cameras]]',
|
||||
`name = ${tomlString(cameraName)}`,
|
||||
`username = ${tomlString(username)}`,
|
||||
`password = ${tomlString(password)}`,
|
||||
`address = ${tomlString(`${host}:${mediaPort}`)}`,
|
||||
'stream = "subStream"',
|
||||
'',
|
||||
].join('\n');
|
||||
|
||||
/*
|
||||
Write on every playback instead of trying to detect config drift. The file
|
||||
is small, and this guarantees a camera password/host change in config.yaml
|
||||
is reflected without an extra migration path or manual cleanup.
|
||||
*/
|
||||
await fsp.writeFile(configPath, body, { mode: 0o600 });
|
||||
return configPath;
|
||||
}
|
||||
|
||||
function spawnChecked(label, command, args, options = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const proc = spawn(command, args, {
|
||||
stdio: ['ignore', 'ignore', 'pipe'],
|
||||
...options,
|
||||
});
|
||||
let stderr = '';
|
||||
const timer = setTimeout(() => {
|
||||
try {
|
||||
proc.kill('SIGKILL');
|
||||
} catch {
|
||||
// noop
|
||||
}
|
||||
}, options.timeoutMs || PLAYBACK_TIMEOUT_MS);
|
||||
proc.stderr?.on('data', (chunk) => {
|
||||
stderr = `${stderr}${String(chunk || '')}`.slice(-4000);
|
||||
});
|
||||
proc.on('error', (err) => {
|
||||
clearTimeout(timer);
|
||||
reject(new Error(`${label} failed to start: ${err.message}`));
|
||||
});
|
||||
proc.on('exit', (code, signal) => {
|
||||
clearTimeout(timer);
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
reject(new Error(`${label} exited code=${code} signal=${signal || 'none'} ${stderr.trim()}`.trim()));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function renderEspeak(text, ttsOptions, filePath) {
|
||||
const args = ['-w', filePath];
|
||||
const pitch = clampNumber(ttsOptions.pitch, 50, 0, 99);
|
||||
if (pitch > 0) args.push('-p', String(Math.round(pitch)));
|
||||
args.push(text);
|
||||
await spawnChecked('espeak', espeakBin, args, { timeoutMs: 20000 });
|
||||
}
|
||||
|
||||
async function renderFlite(text, ttsOptions, filePath) {
|
||||
const args = ['-o', filePath];
|
||||
const voice = String(ttsOptions.voice || fliteDefaultVoice || '').trim();
|
||||
if (voice) args.push('-voice', voice);
|
||||
args.push('-t', text);
|
||||
await spawnChecked('flite', fliteBin, args, { timeoutMs: 20000 });
|
||||
}
|
||||
|
||||
async function renderChromeGoogleTts(text, ttsOptions, filePath) {
|
||||
const args = [
|
||||
'--text',
|
||||
text,
|
||||
'--voice',
|
||||
String(ttsOptions.voice || 'tpf'),
|
||||
'--pitch',
|
||||
String(clampNumber(ttsOptions.pitch, 1, 0.5, 2)),
|
||||
'--speed',
|
||||
String(clampNumber(ttsOptions.speed, 1, 0.5, 2)),
|
||||
'--output',
|
||||
filePath,
|
||||
];
|
||||
await spawnChecked('chromegtts-wav', chromegttsWavBin, args, { timeoutMs: 30000 });
|
||||
}
|
||||
|
||||
async function ensureTtsFile(text, rawOptions = {}) {
|
||||
const cleanText = normalizeText(text);
|
||||
if (!cleanText) throw new Error('PTZ TTS text required');
|
||||
const engine = normalizeEngine(rawOptions.engine);
|
||||
const ttsOptions = {
|
||||
engine,
|
||||
voice: typeof rawOptions.voice === 'string' ? rawOptions.voice.trim() : '',
|
||||
pitch: Number.isFinite(rawOptions.pitch) ? rawOptions.pitch : undefined,
|
||||
speed: Number.isFinite(rawOptions.speed) ? rawOptions.speed : undefined,
|
||||
};
|
||||
await fsp.mkdir(cacheDir, { recursive: true });
|
||||
const filePath = path.join(cacheDir, `${cacheKeyFor(cleanText, ttsOptions)}.wav`);
|
||||
try {
|
||||
const stat = await fsp.stat(filePath);
|
||||
if (stat.isFile() && stat.size > 44) return { filePath, engine, cached: true };
|
||||
} catch (err) {
|
||||
if (err.code !== 'ENOENT') throw err;
|
||||
}
|
||||
|
||||
const tmpPath = `${filePath}.${process.pid}.${Date.now()}.tmp.wav`;
|
||||
/*
|
||||
Every renderer writes a normal WAV file. Neolink/GStreamer handles the
|
||||
final ADPCM talkback encoding that the Reolink camera expects, so the TTS
|
||||
renderer stays concerned only with faithfully matching the selected rover
|
||||
TTS engine's voice options.
|
||||
*/
|
||||
if (engine === 'espeak') await renderEspeak(cleanText, ttsOptions, tmpPath);
|
||||
else if (engine === 'flite') await renderFlite(cleanText, ttsOptions, tmpPath);
|
||||
else await renderChromeGoogleTts(cleanText, ttsOptions, tmpPath);
|
||||
|
||||
await fsp.rename(tmpPath, filePath);
|
||||
return { filePath, engine, cached: false };
|
||||
}
|
||||
|
||||
function stopActivePlayback(reason = 'replace') {
|
||||
if (!playbackProc) return;
|
||||
const proc = playbackProc;
|
||||
playbackProc = null;
|
||||
logger.info('Stopping PTZ TTS playback', { reason, pid: proc.pid || null });
|
||||
try {
|
||||
proc.kill('SIGTERM');
|
||||
} catch {
|
||||
// noop
|
||||
}
|
||||
setTimeout(() => {
|
||||
if (proc.exitCode == null && proc.signalCode == null) {
|
||||
try {
|
||||
proc.kill('SIGKILL');
|
||||
} catch {
|
||||
// noop
|
||||
}
|
||||
}
|
||||
}, 1200);
|
||||
}
|
||||
|
||||
async function playFile(filePath, context = {}) {
|
||||
if (!audioEnabled) throw new Error('PTZ audio disabled');
|
||||
const neolinkConfigPath = await ensureNeolinkConfig();
|
||||
const stat = await fsp.stat(filePath);
|
||||
if (!stat.isFile()) throw new Error(`PTZ TTS file is not a regular file: ${filePath}`);
|
||||
|
||||
stopActivePlayback('new-playback');
|
||||
const seq = ++playbackSeq;
|
||||
const args = [
|
||||
'talk',
|
||||
cameraName,
|
||||
'-c',
|
||||
neolinkConfigPath,
|
||||
'--volume',
|
||||
String(volume),
|
||||
'--file-path',
|
||||
filePath,
|
||||
];
|
||||
const proc = spawn(neolinkBin, args, { stdio: ['ignore', 'ignore', 'pipe'] });
|
||||
playbackProc = proc;
|
||||
let stderr = '';
|
||||
const timer = setTimeout(() => {
|
||||
if (playbackProc === proc) stopActivePlayback('timeout');
|
||||
}, PLAYBACK_TIMEOUT_MS);
|
||||
|
||||
proc.stderr?.on('data', (chunk) => {
|
||||
stderr = `${stderr}${String(chunk || '')}`.slice(-4000);
|
||||
});
|
||||
proc.on('error', (err) => {
|
||||
clearTimeout(timer);
|
||||
if (playbackProc === proc) playbackProc = null;
|
||||
logger.warn('PTZ TTS neolink spawn failed', { error: err.message, context });
|
||||
});
|
||||
proc.on('exit', (code, signal) => {
|
||||
clearTimeout(timer);
|
||||
if (playbackProc === proc) playbackProc = null;
|
||||
if (code === 0 || signal === 'SIGTERM') {
|
||||
logger.info('PTZ TTS playback finished', { code, signal, context });
|
||||
return;
|
||||
}
|
||||
logger.warn('PTZ TTS playback failed', {
|
||||
code,
|
||||
signal,
|
||||
stderr: stderr.trim().slice(-1000),
|
||||
context,
|
||||
});
|
||||
});
|
||||
|
||||
logger.info('PTZ TTS playback started', {
|
||||
pid: proc.pid || null,
|
||||
filePath,
|
||||
engine: context.engine || null,
|
||||
actor: context.socketId ? getSocketLabel(context.socketId) : null,
|
||||
seq,
|
||||
});
|
||||
return { pid: proc.pid || null, seq };
|
||||
}
|
||||
|
||||
async function speakText(text, ttsOptions = {}, context = {}) {
|
||||
const rendered = await ensureTtsFile(text, ttsOptions);
|
||||
await playFile(rendered.filePath, {
|
||||
...context,
|
||||
engine: rendered.engine,
|
||||
cached: rendered.cached,
|
||||
});
|
||||
return rendered;
|
||||
}
|
||||
|
||||
return {
|
||||
getState,
|
||||
speakText,
|
||||
stopActivePlayback,
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
createPtzAudioPlayback,
|
||||
};
|
||||
File diff suppressed because it is too large
Load Diff
@@ -49,7 +49,7 @@ const replayBuilder = createReplayBuilder({
|
||||
ensureDir,
|
||||
renderSidebarVideo: sidebarRenderer.renderSidebarVideo,
|
||||
getVideoEntriesForSource: segmentStore.getVideoEntriesForSource,
|
||||
getAudioEntriesForRover: segmentStore.getAudioEntriesForRover,
|
||||
getAudioEntriesForSource: segmentStore.getAudioEntriesForSource,
|
||||
overlapping: segmentStore.overlapping,
|
||||
});
|
||||
registerReplaySocketHooks({ tryTriggerReplay, validateSources, getDefaultWebSources });
|
||||
|
||||
@@ -51,7 +51,7 @@ function buildChatEventsForWindow(startMs, endMs, limit = 22, preWindowCount = 1
|
||||
return [...beforeWindow, ...inWindow].sort((a, b) => a.ts - b.ts);
|
||||
}
|
||||
|
||||
function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForRover, overlapping }) {
|
||||
function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo, getVideoEntriesForSource, getAudioEntriesForSource, overlapping }) {
|
||||
function resolveReplayWindow({ sources = [], nowMs, guardMs, durationMs }) {
|
||||
const tentativeEnd = nowMs - guardMs;
|
||||
const sourceEnds = [];
|
||||
@@ -75,6 +75,33 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
|
||||
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-f','concat','-safe','0','-i',listPath,'-c','copy',outPath]);
|
||||
}
|
||||
|
||||
async function pinSegmentFiles(entries, tmpDir, prefix) {
|
||||
/*
|
||||
Replay segment files live in a rolling buffer, so cleanup can unlink one
|
||||
while a slower replay build is still working. Pinning selected files into
|
||||
the per-build temp directory gives ffmpeg stable paths for the whole build.
|
||||
A hard link is preferred because it is cheap and keeps the inode alive even
|
||||
when cleanup removes the original directory entry; copyFile is the fallback
|
||||
for filesystems that do not support linking across the involved paths.
|
||||
*/
|
||||
const pinned = [];
|
||||
for (let i = 0; i < entries.length; i += 1) {
|
||||
const entry = entries[i];
|
||||
const pinnedPath = path.join(tmpDir, `${prefix}-${String(i).padStart(4, '0')}.mp4`);
|
||||
try {
|
||||
await fsp.link(entry.filePath, pinnedPath);
|
||||
} catch (linkErr) {
|
||||
try {
|
||||
await fsp.copyFile(entry.filePath, pinnedPath);
|
||||
} catch (copyErr) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
pinned.push({ ...entry, filePath: pinnedPath });
|
||||
}
|
||||
return pinned;
|
||||
}
|
||||
|
||||
async function probeMaxFrameSize(paths) {
|
||||
let maxWidth = 0, maxHeight = 0;
|
||||
for (const filePath of paths) {
|
||||
@@ -108,7 +135,11 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
|
||||
for (let i = 0; i < sources.length; i += 1) {
|
||||
const source = sources[i];
|
||||
const sourceId = String(source.id);
|
||||
const videoEntries = overlapping(getVideoEntriesForSource({ type: String(source.type), id: sourceId }), tStart, tEnd);
|
||||
const videoEntries = await pinSegmentFiles(
|
||||
overlapping(getVideoEntriesForSource({ type: String(source.type), id: sourceId }), tStart, tEnd),
|
||||
tmpDir,
|
||||
`video-${i}-seg`,
|
||||
);
|
||||
if (!videoEntries.length) { missingSources.push({ ...source, reason: 'no video coverage in replay window' }); continue; }
|
||||
|
||||
const videoConcat = path.join(tmpDir, `video-${i}.mp4`);
|
||||
@@ -122,18 +153,26 @@ function createReplayBuilder({ execFileAsync, fsp, ensureDir, renderSidebarVideo
|
||||
normalizedVideos.push({ path: videoTrimmed, source });
|
||||
usedSources.push(source);
|
||||
|
||||
if (source.type === 'rover') {
|
||||
const audioEntries = overlapping(getAudioEntriesForRover(sourceId), tStart, tEnd);
|
||||
if (audioEntries.length) {
|
||||
const audioConcat = path.join(tmpDir, `audio-${i}.m4a`);
|
||||
await concatFiles(audioEntries.map((entry) => entry.filePath), audioConcat);
|
||||
const audioTrimmed = path.join(tmpDir, `audio-${i}.trim.m4a`);
|
||||
const firstAudioStartMs = audioEntries[0].startMs;
|
||||
const ass = Math.max(0, (tStart - firstAudioStartMs) / 1000);
|
||||
const ato = Math.max(ass + 0.1, (tEnd - firstAudioStartMs) / 1000);
|
||||
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-ss',ass.toFixed(3),'-to',ato.toFixed(3),'-i',audioConcat,'-vn','-ac','1','-ar','48000','-c:a','aac','-b:a','96k',audioTrimmed]);
|
||||
normalizedAudios.push(audioTrimmed);
|
||||
}
|
||||
const audioEntries = await pinSegmentFiles(
|
||||
overlapping(getAudioEntriesForSource(source), tStart, tEnd),
|
||||
tmpDir,
|
||||
`audio-${i}-seg`,
|
||||
);
|
||||
if (audioEntries.length) {
|
||||
/*
|
||||
Audio workers are separate from selected video sources, even for PTZ
|
||||
where the live camera path carries inline Opus. Trim the matching
|
||||
source-owned audio window here and let the final graph mix every
|
||||
selected source's audio together.
|
||||
*/
|
||||
const audioConcat = path.join(tmpDir, `audio-${i}.m4a`);
|
||||
await concatFiles(audioEntries.map((entry) => entry.filePath), audioConcat);
|
||||
const audioTrimmed = path.join(tmpDir, `audio-${i}.trim.m4a`);
|
||||
const firstAudioStartMs = audioEntries[0].startMs;
|
||||
const ass = Math.max(0, (tStart - firstAudioStartMs) / 1000);
|
||||
const ato = Math.max(ass + 0.1, (tEnd - firstAudioStartMs) / 1000);
|
||||
await execFileAsync(FFMPEG_BIN, ['-y','-hide_banner','-loglevel','error','-ss',ass.toFixed(3),'-to',ato.toFixed(3),'-i',audioConcat,'-vn','-ac','1','-ar','48000','-c:a','aac','-b:a','96k',audioTrimmed]);
|
||||
normalizedAudios.push(audioTrimmed);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
// Scope: Handles user-visible replay source catalogs and default source selection rules.
|
||||
const roverManager = require('../roverManager');
|
||||
const { getRoomCameras } = require('../roomCameraService');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
|
||||
function getReplaySources(socket = null) {
|
||||
const roster = socket ? roverManager.getRosterForSocket(socket) : roverManager.getRoster();
|
||||
@@ -21,7 +22,10 @@ function getReplaySources(socket = null) {
|
||||
label: camera.name || camera.id,
|
||||
}));
|
||||
|
||||
return [...roverSources, ...roomSources];
|
||||
const ptzSource = ptzCameraService.getReplaySource();
|
||||
const ptzSources = ptzSource ? [ptzSource] : [];
|
||||
|
||||
return [...roverSources, ...roomSources, ...ptzSources];
|
||||
}
|
||||
|
||||
function normalizeSource(entry) {
|
||||
@@ -67,11 +71,14 @@ function getDefaultWebSources(assignment = {}, socket = null) {
|
||||
}
|
||||
|
||||
function getDefaultDiscordSources() {
|
||||
return getRoomCameras().map((camera) => ({
|
||||
const sources = getRoomCameras().map((camera) => ({
|
||||
type: 'room',
|
||||
id: String(camera.id),
|
||||
label: camera.name || camera.id,
|
||||
}));
|
||||
const ptzSource = ptzCameraService.getReplaySource();
|
||||
if (ptzSource) sources.push(ptzSource);
|
||||
return sources;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
|
||||
@@ -8,6 +8,7 @@ const logger = require('../../globals/logger').child('replayEngineV2');
|
||||
const { BUFFER_SECONDS, SEGMENT_SECONDS } = require('./constants');
|
||||
const { workers, segmentIndex } = require('./state');
|
||||
const { sourceKey, sourceDirForKey } = require('./sources');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
|
||||
async function ensureDir(dir) {
|
||||
await fsp.mkdir(dir, { recursive: true });
|
||||
@@ -96,6 +97,24 @@ function createSegmentStore({ getActiveSegmentRoot }) {
|
||||
return segmentIndex.get(key) || [];
|
||||
}
|
||||
|
||||
function getAudioEntriesForSource(source) {
|
||||
/*
|
||||
Rover audio is published as a separate "<rover>-audio" stream, while PTZ
|
||||
replay audio is split into an internal "ptz-camera-audio" worker from the
|
||||
same live camera stream. Keep this mapping close to the segment index so
|
||||
replayBuilder can ask for "audio that belongs to this selected source"
|
||||
without knowing every worker naming convention.
|
||||
*/
|
||||
const type = String(source?.type || '');
|
||||
const id = String(source?.id || '');
|
||||
if (type === 'rover') return getAudioEntriesForRover(id);
|
||||
if (type === 'ptz') {
|
||||
const key = sourceKey({ sourceType: 'ptz', kind: 'audio', id: `${id}-audio` });
|
||||
return segmentIndex.get(key) || [];
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
function overlapping(entries, startMs, endMs) {
|
||||
return entries.filter((entry) => entry.endMs > startMs && entry.startMs < endMs);
|
||||
}
|
||||
@@ -123,6 +142,8 @@ function createSegmentStore({ getActiveSegmentRoot }) {
|
||||
for (const camera of getRoomCameras()) {
|
||||
replaySources.push({ type: 'room', id: String(camera.id), label: camera.name || camera.id });
|
||||
}
|
||||
const ptzSource = ptzCameraService.getReplaySource();
|
||||
if (ptzSource) replaySources.push(ptzSource);
|
||||
|
||||
for (const source of replaySources) {
|
||||
const key = sourceKey({ sourceType: source.type, kind: 'video', id: String(source.id) });
|
||||
@@ -154,6 +175,7 @@ function createSegmentStore({ getActiveSegmentRoot }) {
|
||||
cleanupOldFiles,
|
||||
getVideoEntriesForSource,
|
||||
getAudioEntriesForRover,
|
||||
getAudioEntriesForSource,
|
||||
overlapping,
|
||||
bootstrapIndexFromDisk,
|
||||
getReplayHealthSnapshot,
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
const path = require('path');
|
||||
const roverManager = require('../roverManager');
|
||||
const { getRoomCameras } = require('../roomCameraService');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
const { FFMPEG_BIN, SEGMENT_SECONDS, TARGET_FPS } = require('./constants');
|
||||
|
||||
function sourceKey(source) {
|
||||
@@ -46,6 +47,10 @@ function listDesiredSources() {
|
||||
if (!streamUrl) continue;
|
||||
sources.push({ id: String(camera.id), sourceType: 'room', kind: 'video', label: camera.name || camera.id, inputUrl: streamUrl });
|
||||
}
|
||||
// The PTZ service owns its own worker list because video and microphone audio
|
||||
// both come from the same live MediaMTX path, unlike rovers where audio is a
|
||||
// separate published stream.
|
||||
sources.push(...ptzCameraService.getReplayWorkerSources());
|
||||
return sources;
|
||||
}
|
||||
|
||||
|
||||
@@ -89,6 +89,7 @@ const {
|
||||
canDrive,
|
||||
getRoversForSocket,
|
||||
getPrimaryRoverForSocket,
|
||||
canLeaveCurrentRover,
|
||||
canSwitchRover,
|
||||
} = roverLifecycle;
|
||||
|
||||
@@ -277,6 +278,7 @@ module.exports = {
|
||||
managerEvents,
|
||||
getRoversForSocket,
|
||||
getPrimaryRoverForSocket,
|
||||
canLeaveCurrentRover,
|
||||
canSeeRover,
|
||||
canRequestControl,
|
||||
applyPrivateDriveSafety,
|
||||
|
||||
@@ -137,6 +137,22 @@ function createRoverLifecycle(deps) {
|
||||
return false;
|
||||
}
|
||||
|
||||
function canLeaveCurrentRover(socket, options = {}) {
|
||||
const currentId = getPrimaryRoverForSocket(socket.id);
|
||||
/*
|
||||
Leaving a rover is only risky when this socket is the last person attached
|
||||
to an undocked rover. The same rule is used for rover-to-rover switching
|
||||
and PTZ camera claiming so there is exactly one definition of "do not
|
||||
abandon a rover in the room".
|
||||
*/
|
||||
if (!currentId || currentId === options.targetRoverId) return { ok: true, currentId };
|
||||
const currentRecord = rovers.get(currentId);
|
||||
if (!currentRecord) return { ok: true, currentId };
|
||||
if (hasOtherDrivers(currentRecord, socket.id)) return { ok: true, currentId };
|
||||
if (isDockedAndCharging(currentRecord)) return { ok: true, currentId };
|
||||
return { ok: false, currentId, message: 'Dock and charge your current rover before switching.' };
|
||||
}
|
||||
|
||||
function canSwitchRover(socket, targetRoverId, options = {}) {
|
||||
const target = rovers.get(targetRoverId);
|
||||
if (!target) return { ok: false, message: 'Unknown rover' };
|
||||
@@ -145,13 +161,7 @@ function createRoverLifecycle(deps) {
|
||||
allowClosedPrivateGrantInLockdown: Boolean(options.allowClosedPrivateGrantInLockdown),
|
||||
});
|
||||
if (denied) return { ok: false, message: denied };
|
||||
const currentId = getPrimaryRoverForSocket(socket.id);
|
||||
if (!currentId || currentId === targetRoverId) return { ok: true, currentId };
|
||||
const currentRecord = rovers.get(currentId);
|
||||
if (!currentRecord) return { ok: true, currentId };
|
||||
if (hasOtherDrivers(currentRecord, socket.id)) return { ok: true, currentId };
|
||||
if (isDockedAndCharging(currentRecord)) return { ok: true, currentId };
|
||||
return { ok: false, currentId, message: 'Dock and charge your current rover before switching.' };
|
||||
return canLeaveCurrentRover(socket, { targetRoverId });
|
||||
}
|
||||
|
||||
function canReplayRoverId(roverId, isPrivateRecord, isPrivateOpen) {
|
||||
@@ -169,6 +179,7 @@ function createRoverLifecycle(deps) {
|
||||
canDrive,
|
||||
getRoversForSocket,
|
||||
getPrimaryRoverForSocket,
|
||||
canLeaveCurrentRover,
|
||||
canSwitchRover,
|
||||
canReplayRoverId,
|
||||
};
|
||||
|
||||
@@ -10,6 +10,12 @@ const { managerEvents } = roverManager;
|
||||
const assignmentService = require('../assignmentService');
|
||||
const { getActiveDrivers, getTurnQueues, turnEvents } = require('../turnService');
|
||||
const { getRoomCameras, roomCameraEvents } = require('../roomCameraService');
|
||||
const {
|
||||
getPublicState: getPtzCameraState,
|
||||
getChatTargetForSocket: getPtzChatTargetForSocket,
|
||||
PTZ_CAMERA_ID,
|
||||
ptzCameraEvents,
|
||||
} = require('../ptzCameraService');
|
||||
const { getState: getHomeAssistantState, homeAssistantEvents } = require('../homeAssistantService');
|
||||
const { getState: getNeatoState, neatoEvents } = require('../neatoService');
|
||||
const { getState: getLiftState, liftEvents } = require('../liftService');
|
||||
@@ -61,12 +67,19 @@ function buildUserEntry(socket) {
|
||||
const role = getRole(socket);
|
||||
const assignment = assignmentService.describeAssignment(socket.id);
|
||||
const primaryRover = roverManager.getPrimaryRoverForSocket(socket.id);
|
||||
const ptzChatTarget = getPtzChatTargetForSocket(socket.id);
|
||||
return {
|
||||
socketId: socket.id,
|
||||
userId: socket?.data?.userId || null,
|
||||
nickname: getNickname(socket) || null,
|
||||
role,
|
||||
roverId: primaryRover || assignment?.roverId || null,
|
||||
/*
|
||||
PTZ is not inserted into the physical rover roster, but for chat and user
|
||||
presence it should read like the user moved to a rover-like target. Prefer
|
||||
the PTZ chat target while the socket is queued or operating so presence,
|
||||
queue lookup, and chat identity all agree.
|
||||
*/
|
||||
roverId: ptzChatTarget?.roverId || primaryRover || assignment?.roverId || null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -78,7 +91,13 @@ function buildSession(socket) {
|
||||
.filter(Boolean)
|
||||
.map((entry) => ({
|
||||
...entry,
|
||||
roverId: filterVisibleRoverId(socket, entry.roverId),
|
||||
/*
|
||||
PTZ is intentionally not a roverManager record, so the normal physical
|
||||
rover visibility filter would erase the user's PTZ chat target. Preserve
|
||||
it here because getPtzChatTargetForSocket already applied the PTZ access
|
||||
and queue/operator rules before buildUserEntry returned it.
|
||||
*/
|
||||
roverId: entry.roverId === PTZ_CAMERA_ID ? entry.roverId : filterVisibleRoverId(socket, entry.roverId),
|
||||
}));
|
||||
const roster = roverManager.getRosterForSocket(socket);
|
||||
const assignment = assignmentService.describeAssignment(socket?.id || '');
|
||||
@@ -107,6 +126,7 @@ function buildSession(socket) {
|
||||
activeDrivers,
|
||||
turnQueues,
|
||||
roomCameras: getRoomCameras(),
|
||||
ptzCamera: getPtzCameraState(socket),
|
||||
homeAssistant: getHomeAssistantState(),
|
||||
neato: getNeatoState(),
|
||||
lift: getLiftState(),
|
||||
@@ -281,6 +301,11 @@ roomCameraEvents.on('update', () => {
|
||||
syncAll();
|
||||
});
|
||||
|
||||
ptzCameraEvents.on('change', () => {
|
||||
logger.info('PTZ camera state change; syncing all clients');
|
||||
syncAll();
|
||||
});
|
||||
|
||||
homeAssistantEvents.on('update', () => {
|
||||
logger.info('Home Assistant state change; syncing all clients');
|
||||
syncAll();
|
||||
|
||||
@@ -10,6 +10,7 @@ const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
|
||||
const { isVerified } = require('../verificationService');
|
||||
const turnService = require('../turnService');
|
||||
const roverManager = require('../roverManager');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
const { getRequestIp, getSocketIp, isLocalNetwork } = require('../../helpers/ipResolver');
|
||||
const { logAdminEvent } = require('../adminLogService');
|
||||
|
||||
@@ -26,6 +27,7 @@ const { canAccessStream } = createVideoAuthPolicy({
|
||||
isVerified,
|
||||
turnService,
|
||||
roverManager,
|
||||
ptzCameraService,
|
||||
getSocketIp,
|
||||
isLocalNetwork,
|
||||
});
|
||||
|
||||
@@ -11,6 +11,7 @@ function createVideoAuthPolicy(deps) {
|
||||
isVerified,
|
||||
turnService,
|
||||
roverManager,
|
||||
ptzCameraService,
|
||||
getSocketIp,
|
||||
isLocalNetwork,
|
||||
} = deps;
|
||||
@@ -40,6 +41,10 @@ function createVideoAuthPolicy(deps) {
|
||||
}
|
||||
}
|
||||
|
||||
if (streamInfo.type === 'ptz') {
|
||||
return ptzCameraService.canRequestLiveVideo(socket);
|
||||
}
|
||||
|
||||
if (sourceType === 'roverMic' && action === 'publish') {
|
||||
const roverId = streamInfo.baseId || streamInfo.id;
|
||||
if (!isVerified(socket)) {
|
||||
|
||||
@@ -5,6 +5,7 @@ const { loadConfig } = require('../../helpers/configLoader');
|
||||
|
||||
const config = loadConfig();
|
||||
const mediaConfig = config.media || {};
|
||||
const PTZ_STREAM_PATH = 'ptz-camera';
|
||||
|
||||
function getPathPrefix() {
|
||||
const base = mediaConfig.whepBaseUrl;
|
||||
@@ -37,6 +38,14 @@ function extractStreamInfo(path) {
|
||||
const remaining = segments.slice(start, end);
|
||||
if (remaining.length === 1) {
|
||||
const rawId = remaining[0] || '';
|
||||
/*
|
||||
PTZ is intentionally published as a flat MediaMTX path so WHEP requests
|
||||
line up with the real stream name. Treat that one reserved path as PTZ
|
||||
before falling back to the normal one-segment rover parsing rules.
|
||||
*/
|
||||
if (rawId === PTZ_STREAM_PATH) {
|
||||
return { type: 'ptz', id: rawId };
|
||||
}
|
||||
if (rawId.endsWith('-fwd')) {
|
||||
return { type: 'rover', id: rawId, baseId: rawId.slice(0, -4) };
|
||||
}
|
||||
@@ -48,6 +57,10 @@ function extractStreamInfo(path) {
|
||||
return { type: 'room', id: remaining[1] || '' };
|
||||
}
|
||||
|
||||
if (remaining.length === 2 && remaining[0] === 'ptz') {
|
||||
return { type: 'ptz', id: remaining[1] || '' };
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -77,6 +90,10 @@ function extractStreamInfoFromBody(body = {}) {
|
||||
extractSrtStreamId(body.query);
|
||||
if (!srtId) return null;
|
||||
|
||||
if (srtId === PTZ_STREAM_PATH) {
|
||||
return { type: 'ptz', id: srtId };
|
||||
}
|
||||
|
||||
if (srtId.endsWith('-fwd')) {
|
||||
return { type: 'rover', id: srtId, baseId: srtId.slice(0, -4) };
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ const { getMode, MODES } = require('../modeManager');
|
||||
const { isAdmin, isLockdownAdmin, getRole } = require('../roleService');
|
||||
const videoSessions = require('../videoSessions');
|
||||
const roverManager = require('../roverManager');
|
||||
const ptzCameraService = require('../ptzCameraService');
|
||||
const { loadConfig } = require('../../helpers/configLoader');
|
||||
const { getSocketIp, isLocalNetwork } = require('../../helpers/ipResolver');
|
||||
|
||||
@@ -34,6 +35,15 @@ function buildWhepUrlForSource(source) {
|
||||
const segments = [];
|
||||
if (source.type === 'room') {
|
||||
segments.push('room', encodeURIComponent(source.id));
|
||||
} else if (source.type === 'ptz') {
|
||||
/*
|
||||
MediaMTX exposes WHEP by the exact path name that is being published.
|
||||
The PTZ ffmpeg publisher registers the single camera as "ptz-camera",
|
||||
so the browser must request "/video/ptz-camera/whep" instead of a
|
||||
namespace-like "/video/ptz/ptz-camera/whep" path that MediaMTX has never
|
||||
seen and correctly returns as 404.
|
||||
*/
|
||||
segments.push(encodeURIComponent(source.id));
|
||||
} else {
|
||||
segments.push(encodeURIComponent(source.id));
|
||||
}
|
||||
@@ -81,6 +91,9 @@ function normalizeRequest(payload = {}) {
|
||||
if (payload.roomCameraId) {
|
||||
return { type: 'room', id: String(payload.roomCameraId) };
|
||||
}
|
||||
if (payload.ptzCameraId || payload.type === 'ptz') {
|
||||
return { type: 'ptz', id: String(payload.ptzCameraId || payload.id || ptzCameraService.PTZ_CAMERA_ID) };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -109,6 +122,13 @@ io.on('connection', (socket) => {
|
||||
}
|
||||
} else if (target.type === 'room') {
|
||||
throw new Error('Room cameras now use the snapshot feed');
|
||||
} else if (target.type === 'ptz') {
|
||||
if (target.id !== ptzCameraService.PTZ_CAMERA_ID) {
|
||||
throw new Error('Unknown PTZ camera');
|
||||
}
|
||||
if (!ptzCameraService.canRequestLiveVideo(socket)) {
|
||||
throw new Error('Not authorized for PTZ video');
|
||||
}
|
||||
} else {
|
||||
throw new Error('Unsupported video source');
|
||||
}
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
1. improve spectator page, options on what to see and what not to see
|
||||
2. assign rovers based on battery percentage, give people highest one
|
||||
3. add prefix config for discord bot to replace rs with something else
|
||||
4. add config to disable client snapshot forcing, disable bandwidth saving
|
||||
5. add admin ui for VIP and private requests instead of only through discord
|
||||
6. make discord bots that can be fine with multiple in one server
|
||||
7. pull page / tab title from session, use the name from profile of interinstance if enabled, if not just default to old one
|
||||
3. add config to disable client snapshot forcing, disable bandwidth saving
|
||||
4. add admin ui for VIP and private requests instead of only through discord
|
||||
5. pull page / tab title from session, use the name from profile of interinstance if enabled, if not just default to old one
|
||||
6. make overcurrent limiter speed sensitive, slower fill at lower speeds
|
||||
7. add feature chat commands, like /neato start, /lift up, etc
|
||||
8. add more background gap themes
|
||||
8. fix this:
|
||||
9. fix this:
|
||||
`Jun 18 15:14:18 roombaserver.local node[216731]: /home/daniel/MultiRoombaRover/server/src/services/roverManager/socketHandlers.js:92
|
||||
Jun 18 15:14:18 roombaserver.local node[216731]: cb({ error: err.message });
|
||||
Jun 18 15:14:18 roombaserver.local node[216731]: ^
|
||||
|
||||
+4
-1
@@ -38,6 +38,7 @@ import useUserIdentitySync from './hooks/useUserIdentitySync.js';
|
||||
import useIncomingInterInstanceTransfer from './hooks/useIncomingInterInstanceTransfer.js';
|
||||
import GlobalObjectiveBanner from './components/GlobalObjectiveBanner/index.jsx';
|
||||
import RoverQueuesPanel from './components/RoverQueuesPanel/index.jsx';
|
||||
import PtzQueueCard from './components/PtzCamera/index.jsx';
|
||||
import VipPanel from './components/VipPanel/index.jsx';
|
||||
import { useSessionSelector } from './context/SessionContext.jsx';
|
||||
import { useTelemetryVisualPolicy } from './context/TelemetryContext.jsx';
|
||||
@@ -206,7 +207,7 @@ function MobileFeatureTabs({
|
||||
</div>
|
||||
</TabPanel>
|
||||
<TabPanel id="vip" keepMounted>
|
||||
<VipPanel isActive={activeTab === 'vip'} />
|
||||
<VipPanel isActive={activeTab === 'vip'} layout={layout} />
|
||||
</TabPanel>
|
||||
<TabPanel id="roomcontrols">
|
||||
<div className={themeStackClass}>
|
||||
@@ -253,6 +254,7 @@ function MobilePortraitLayout({ onOpenHelpOverlay, swapMobileControlColumns = fa
|
||||
<ReplaySourcesPanel panelId="replay-sources-mobile-portrait" />
|
||||
<div className="space-y-0.5">
|
||||
<RoverQueuesPanel />
|
||||
<PtzQueueCard layout="mobile-portrait" />
|
||||
</div>
|
||||
</div>
|
||||
{/* <ControlSummary /> */}
|
||||
@@ -283,6 +285,7 @@ function MobileLandscapeLayout({ onOpenHelpOverlay, swapMobileControlColumns = f
|
||||
<ReplaySourcesPanel panelId="replay-sources-mobile-landscape" />
|
||||
<div className="space-y-0.5">
|
||||
<RoverQueuesPanel />
|
||||
<PtzQueueCard layout="mobile-landscape" />
|
||||
</div>
|
||||
</div>
|
||||
{/* <TelemetryPanel /> */}
|
||||
|
||||
@@ -127,6 +127,7 @@ export function ChatIdentity({ message, toolsToggle = null }) {
|
||||
{message.roverId && (
|
||||
<RoverLabel
|
||||
roverId={message.roverId}
|
||||
name={message.roverName}
|
||||
color={message.roverColor}
|
||||
fallback={message.roverId}
|
||||
className="shrink-0 text-[0.7rem]"
|
||||
|
||||
@@ -49,17 +49,38 @@ function resolveTtsSettings(settings) {
|
||||
|
||||
function useChatComposerSessionState(allowSpectatorInput) {
|
||||
const role = useSessionSelector((state) => state.session?.role || null);
|
||||
const socketId = useSessionSelector((state) => state.session?.socketId || null);
|
||||
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
|
||||
const users = useSessionSelector((state) => state.session?.users ?? []);
|
||||
const roster = useSessionSelector((state) => state.session?.roster ?? []);
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
|
||||
const chatTargetId = useMemo(() => {
|
||||
/*
|
||||
PTZ is intentionally not a roster entry, but session users expose the
|
||||
current chat target for each socket. Prefer the self user entry so the TTS
|
||||
control follows PTZ queue/operation state instead of only physical rover
|
||||
assignment state.
|
||||
*/
|
||||
const self = users.find((entry) => entry?.socketId === socketId);
|
||||
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
|
||||
return self?.roverId || currentRoverId || null;
|
||||
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
|
||||
|
||||
const rover = useMemo(
|
||||
() => roster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
|
||||
[currentRoverId, roster],
|
||||
() => roster.find((entry) => String(entry.id) === String(chatTargetId)) || null,
|
||||
[chatTargetId, roster],
|
||||
);
|
||||
const ptzTtsSupported = Boolean(
|
||||
ptz?.id &&
|
||||
String(chatTargetId) === String(ptz.id) &&
|
||||
ptz?.audio?.enabled &&
|
||||
(ptz?.isOperator || ptz?.queuedPosition),
|
||||
);
|
||||
|
||||
return {
|
||||
canChat: role !== 'spectator' || allowSpectatorInput,
|
||||
ttsSupported: Boolean(rover?.audio?.ttsEnabled),
|
||||
ttsSupported: Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -214,6 +235,7 @@ function TtsControls({
|
||||
function ChatComposer({
|
||||
allowSpectatorInput = false,
|
||||
hideSpectatorNotice = false,
|
||||
inputTarget = 'panel',
|
||||
}) {
|
||||
const {
|
||||
sendMessage,
|
||||
@@ -310,7 +332,7 @@ function ChatComposer({
|
||||
setTypingActive(false);
|
||||
}
|
||||
}}
|
||||
ref={(el) => registerInputRef(el, { target: 'panel' })}
|
||||
ref={(el) => registerInputRef(el, { target: inputTarget })}
|
||||
placeholder={canChat ? 'Type a message…' : hideSpectatorNotice ? '' : 'Spectators cannot chat'}
|
||||
disabled={!canChat}
|
||||
/>
|
||||
@@ -346,6 +368,7 @@ export default function ChatPanel({
|
||||
allowSpectatorInput = false,
|
||||
title = 'Chat and speech',
|
||||
minimal = false,
|
||||
inputTarget = 'panel',
|
||||
}) {
|
||||
const effectiveHideInput = minimal || hideInput;
|
||||
const effectiveTitle = minimal ? '' : title;
|
||||
@@ -362,6 +385,7 @@ export default function ChatPanel({
|
||||
<MemoizedChatComposer
|
||||
allowSpectatorInput={allowSpectatorInput}
|
||||
hideSpectatorNotice={hideSpectatorNotice}
|
||||
inputTarget={inputTarget}
|
||||
/>
|
||||
)}
|
||||
</CardFrame>
|
||||
|
||||
@@ -24,8 +24,11 @@ function detectSafari() {
|
||||
|
||||
function HudChatInput({ compact = false }) {
|
||||
const role = useSessionSelector((state) => state.session?.role || null);
|
||||
const socketId = useSessionSelector((state) => state.session?.socketId || null);
|
||||
const currentRoverId = useSessionSelector((state) => state.session?.assignment?.roverId || null);
|
||||
const users = useSessionSelector((state) => state.session?.users || []);
|
||||
const roverRoster = useSessionSelector((state) => state.session?.roster || []);
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const { sendMessage, onInputFocus, onInputBlur, blurChat, registerInputRef, setTypingActive } = useChatActions();
|
||||
const { value: ttsSettings } = useSettingsNamespace('tts', {
|
||||
// HUD chat shares the normal browser TTS defaults, but it has no visible
|
||||
@@ -41,11 +44,28 @@ function HudChatInput({ compact = false }) {
|
||||
const [sending, setSending] = useState(false);
|
||||
const canChat = role !== 'spectator';
|
||||
const hideHudChat = role === 'spectator';
|
||||
const chatTargetId = useMemo(() => {
|
||||
/*
|
||||
HUD chat does not render the full composer controls, but it still sends
|
||||
TTS payloads when the active target supports speech. PTZ appears only in
|
||||
the session user target, not the physical rover roster, so use the same
|
||||
self-target resolution as the full chat panel.
|
||||
*/
|
||||
const self = users.find((entry) => entry?.socketId === socketId);
|
||||
if (!self?.roverId && ptz?.id && (ptz?.isOperator || ptz?.queuedPosition)) return ptz.id;
|
||||
return self?.roverId || currentRoverId || null;
|
||||
}, [currentRoverId, ptz?.id, ptz?.isOperator, ptz?.queuedPosition, socketId, users]);
|
||||
const rover = useMemo(
|
||||
() => roverRoster.find((entry) => String(entry.id) === String(currentRoverId)) || null,
|
||||
[currentRoverId, roverRoster],
|
||||
() => roverRoster.find((entry) => String(entry.id) === String(chatTargetId)) || null,
|
||||
[chatTargetId, roverRoster],
|
||||
);
|
||||
const ttsSupported = Boolean(rover?.audio?.ttsEnabled);
|
||||
const ptzTtsSupported = Boolean(
|
||||
ptz?.id &&
|
||||
String(chatTargetId) === String(ptz.id) &&
|
||||
ptz?.audio?.enabled &&
|
||||
(ptz?.isOperator || ptz?.queuedPosition),
|
||||
);
|
||||
const ttsSupported = Boolean(rover?.audio?.ttsEnabled || ptzTtsSupported);
|
||||
const ttsPayload = useMemo(() => {
|
||||
if (!ttsSupported) return null;
|
||||
const engine =
|
||||
|
||||
@@ -7,9 +7,10 @@ import SocialButton from '../../SocialButton/index.jsx';
|
||||
function TurnsOverlay({
|
||||
roverId = null,
|
||||
mobileHud = false,
|
||||
turnModel = null,
|
||||
}) {
|
||||
const assignedRoverId = useSessionSelector((state) => state.session?.assignment?.roverId ?? null);
|
||||
const effectiveRoverId = roverId ?? assignedRoverId;
|
||||
const effectiveRoverId = turnModel?.targetId ?? roverId ?? assignedRoverId;
|
||||
const mode = useSessionSelector((state) => state.session?.mode || null);
|
||||
const roster = useSessionSelector((state) => state.session?.roster ?? []);
|
||||
const users = useSessionSelector((state) => state.session?.users ?? []);
|
||||
@@ -29,21 +30,30 @@ function TurnsOverlay({
|
||||
const subClass = mobileHud ? 'text-xs' : 'text-sm';
|
||||
const cueTimerClass = mobileHud ? 'text-[0.55rem]' : 'text-[0.75rem]';
|
||||
const cuePadClass = mobileHud ? 'px-4 py-3' : 'px-6 py-4';
|
||||
const turnInfo = effectiveRoverId ? turnQueues?.[effectiveRoverId] : null;
|
||||
const activeDriverId = effectiveRoverId ? activeDrivers?.[effectiveRoverId] : null;
|
||||
const isActiveDriver = Boolean(socketId && activeDriverId === socketId);
|
||||
const isTurnsMode = mode === 'turns';
|
||||
const turnInfo = turnModel ? null : effectiveRoverId ? turnQueues?.[effectiveRoverId] : null;
|
||||
const activeDriverId = turnModel
|
||||
? turnModel.activeId || null
|
||||
: effectiveRoverId
|
||||
? activeDrivers?.[effectiveRoverId] || null
|
||||
: null;
|
||||
const isActiveDriver = turnModel
|
||||
? Boolean(turnModel.isActive)
|
||||
: Boolean(socketId && activeDriverId === socketId);
|
||||
const isTurnsMode = turnModel ? Boolean(turnModel.enabled) : mode === 'turns';
|
||||
const now = useSharedClock(1000, isTurnsMode);
|
||||
const nextDriverId = useMemo(() => {
|
||||
if (turnModel) return turnModel.nextId || null;
|
||||
const queue = turnInfo?.queue || [];
|
||||
if (!queue.length || !turnInfo?.current || queue.length <= 1) return null;
|
||||
const idx = queue.findIndex((id) => id === turnInfo.current);
|
||||
if (idx === -1) return queue[0] || null;
|
||||
return queue[(idx + 1) % queue.length] || null;
|
||||
}, [turnInfo]);
|
||||
const isNextDriver = Boolean(socketId && nextDriverId === socketId);
|
||||
const deadline = turnInfo?.deadline || null;
|
||||
const idleDeadline = turnInfo?.idleDeadline || null;
|
||||
}, [turnInfo, turnModel]);
|
||||
const isNextDriver = turnModel
|
||||
? Boolean(turnModel.isNext)
|
||||
: Boolean(socketId && nextDriverId === socketId);
|
||||
const deadline = turnModel ? turnModel.deadline || null : turnInfo?.deadline || null;
|
||||
const idleDeadline = turnModel ? turnModel.idleDeadline || null : turnInfo?.idleDeadline || null;
|
||||
const msUntilTurn = deadline ? deadline - now : null;
|
||||
const msUntilIdleSkip = idleDeadline ? idleDeadline - now : null;
|
||||
const totalRovers = roster.length;
|
||||
@@ -59,10 +69,14 @@ function TurnsOverlay({
|
||||
});
|
||||
return unique.size;
|
||||
}, [users]);
|
||||
const shouldUsePreviewByLoad = isTurnsMode && totalDrivers > totalRovers;
|
||||
const shouldUsePreviewByLoad = turnModel
|
||||
? Boolean(turnModel.showPreviewReason)
|
||||
: isTurnsMode && totalDrivers > totalRovers;
|
||||
const isPreSwitchWindow =
|
||||
isTurnsMode && isNextDriver && msUntilTurn != null && msUntilTurn <= 5000 && msUntilTurn > 0;
|
||||
const showNotTurnNotice = isTurnsMode && !isActiveDriver;
|
||||
const showNotTurnNotice = turnModel
|
||||
? Boolean(turnModel.showNotTurnNotice ?? (isTurnsMode && !isActiveDriver))
|
||||
: isTurnsMode && !isActiveDriver;
|
||||
const showPreviewReason = showNotTurnNotice && !isPreSwitchWindow && shouldUsePreviewByLoad;
|
||||
const turnSeconds =
|
||||
msUntilTurn != null && Number.isFinite(msUntilTurn) ? Math.max(0, Math.ceil(msUntilTurn / 1000)) : null;
|
||||
@@ -82,19 +96,39 @@ function TurnsOverlay({
|
||||
const turnTimerFlashActive = noticeFlashActive;
|
||||
|
||||
useEffect(() => {
|
||||
if (mode !== 'turns') {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
/*
|
||||
Rover turns and PTZ turns now arrive through the same render path. Use the
|
||||
normalized isTurnsMode flag here instead of checking the server's rover
|
||||
mode directly, otherwise PTZ can render the notice but never trigger the
|
||||
"your turn" cue when camera ownership changes.
|
||||
*/
|
||||
if (!isTurnsMode) {
|
||||
lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null };
|
||||
return;
|
||||
/*
|
||||
React Compiler's lint rules disallow immediate state writes from effect
|
||||
bodies. Defer the visual reset one macrotask; the ref reset above stays
|
||||
synchronous so later turn comparisons do not see stale ownership.
|
||||
*/
|
||||
const resetTimer = setTimeout(() => {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
}, 0);
|
||||
return () => clearTimeout(resetTimer);
|
||||
}
|
||||
const lastTurn = lastTurnRef.current;
|
||||
const nextActiveDriverId = activeDriverId || null;
|
||||
if (!socketId || !effectiveRoverId) {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
lastTurnRef.current = { initialized: false, roverId: null, activeDriverId: null };
|
||||
return;
|
||||
/*
|
||||
No socket/target means there is no turn identity to compare. Reset the
|
||||
comparison ref immediately, then defer the visual state reset for the
|
||||
same React Compiler reason documented in the mode-disabled branch.
|
||||
*/
|
||||
const resetTimer = setTimeout(() => {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
}, 0);
|
||||
return () => clearTimeout(resetTimer);
|
||||
}
|
||||
if (!lastTurn.initialized || lastTurn.roverId !== effectiveRoverId) {
|
||||
lastTurnRef.current = { initialized: true, roverId: effectiveRoverId, activeDriverId: nextActiveDriverId };
|
||||
@@ -104,37 +138,76 @@ function TurnsOverlay({
|
||||
Boolean(lastTurn.activeDriverId) &&
|
||||
lastTurn.activeDriverId !== socketId &&
|
||||
nextActiveDriverId === socketId;
|
||||
let cueTimer = 0;
|
||||
if (becameActive) {
|
||||
setShowTurnCue(true);
|
||||
setTurnCueStartAt(Date.now());
|
||||
const cueStartAt = Date.now();
|
||||
/*
|
||||
The active-turn cue is still caused by this ownership transition, but
|
||||
React Compiler wants visual state writes scheduled from an async edge.
|
||||
Capture the timestamp now so the cue dismissal logic compares against
|
||||
the actual transition time, not the later timer callback time.
|
||||
*/
|
||||
cueTimer = setTimeout(() => {
|
||||
setShowTurnCue(true);
|
||||
setTurnCueStartAt(cueStartAt);
|
||||
}, 0);
|
||||
} else if (nextActiveDriverId !== socketId && showTurnCue) {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
cueTimer = setTimeout(() => {
|
||||
setShowTurnCue(false);
|
||||
setTurnCueStartAt(null);
|
||||
}, 0);
|
||||
}
|
||||
lastTurnRef.current = { initialized: true, roverId: effectiveRoverId, activeDriverId: nextActiveDriverId };
|
||||
}, [activeDriverId, mode, effectiveRoverId, socketId, showTurnCue]);
|
||||
return () => {
|
||||
if (cueTimer) clearTimeout(cueTimer);
|
||||
};
|
||||
}, [activeDriverId, effectiveRoverId, isTurnsMode, socketId, showTurnCue]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!showTurnCue || !turnCueStartAt) return;
|
||||
if (lastControlIntentAt > turnCueStartAt) {
|
||||
setShowTurnCue(false);
|
||||
/*
|
||||
Hide the large "your turn" cue after the first control intent, but
|
||||
schedule the state write outside the effect body so this shared overlay
|
||||
remains compatible with the repo's React Compiler lint settings.
|
||||
*/
|
||||
const timer = setTimeout(() => setShowTurnCue(false), 0);
|
||||
return () => clearTimeout(timer);
|
||||
}
|
||||
return undefined;
|
||||
}, [lastControlIntentAt, showTurnCue, turnCueStartAt]);
|
||||
|
||||
useEffect(() => {
|
||||
const lastIntent = Number(lastIntentRef.current) || 0;
|
||||
const nextIntent = Number(lastControlIntentAt) || 0;
|
||||
if (nextIntent > lastIntent && showNotTurnNotice) {
|
||||
setNotTurnFlashAt(Date.now());
|
||||
const flashAt = Date.now();
|
||||
/*
|
||||
The "not your turn" flash is a direct response to a recorded control
|
||||
intent. Deferring only the state write preserves the timestamp while
|
||||
satisfying the same effect-state lint rule as the turn cue reset.
|
||||
*/
|
||||
const timer = setTimeout(() => setNotTurnFlashAt(flashAt), 0);
|
||||
lastIntentRef.current = nextIntent;
|
||||
return () => clearTimeout(timer);
|
||||
}
|
||||
lastIntentRef.current = nextIntent;
|
||||
return undefined;
|
||||
}, [lastControlIntentAt, showNotTurnNotice]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!showNotTurnNotice || !notTurnFlashAt) return undefined;
|
||||
setNoticeFlashActive(true);
|
||||
const timer = setTimeout(() => setNoticeFlashActive(false), 650);
|
||||
return () => clearTimeout(timer);
|
||||
/*
|
||||
The flash has two timed edges: activate on the next task, then clear after
|
||||
the visible pulse duration. Owning both timers here keeps cleanup local
|
||||
when the user becomes operator or leaves the PTZ/rover turn surface.
|
||||
*/
|
||||
const startTimer = setTimeout(() => setNoticeFlashActive(true), 0);
|
||||
const endTimer = setTimeout(() => setNoticeFlashActive(false), 650);
|
||||
return () => {
|
||||
clearTimeout(startTimer);
|
||||
clearTimeout(endTimer);
|
||||
};
|
||||
}, [showNotTurnNotice, notTurnFlashAt]);
|
||||
|
||||
return (
|
||||
|
||||
@@ -28,7 +28,7 @@ function getSpeedModeConfig(speedMode) {
|
||||
return DRIVE_PAD_SPEED_MODES.find((mode) => mode.id === speedMode) || DRIVE_PAD_SPEED_MODES[1];
|
||||
}
|
||||
|
||||
export default function ControlPadPanel({ disabled = false }) {
|
||||
export default function ControlPadPanel({ compact = false, disabled = false }) {
|
||||
const rawKeymap = useControlSelector((control) => control.state.keymap);
|
||||
const { setCameraPrecisionMode, setDriveVector, registerInputState } = useControlActions();
|
||||
const { value: inputSettings } = useSettingsNamespace('inputs', INPUT_SETTINGS_DEFAULTS);
|
||||
@@ -163,8 +163,8 @@ export default function ControlPadPanel({ disabled = false }) {
|
||||
}, [disabled, setCameraPrecisionMode, stopDrivePad]);
|
||||
|
||||
return (
|
||||
<div className="mobile-touch-control flex flex-1 min-h-0 flex-col overflow-hidden rounded-xl border-2 border-slate-700 bg-slate-900 text-slate-100 shadow-md">
|
||||
<div className="mobile-touch-control grid grid-cols-3 gap-0.5 border-b border-slate-700 bg-slate-950 p-0.5">
|
||||
<div className={`mobile-touch-control flex flex-1 min-h-0 flex-col overflow-hidden rounded-xl border-2 border-slate-700 bg-slate-900 text-slate-100 shadow-md ${compact ? 'h-full' : ''}`}>
|
||||
<div className={`mobile-touch-control grid grid-cols-3 gap-0.5 border-b border-slate-700 bg-slate-950 ${compact ? 'p-0.25' : 'p-0.5'}`}>
|
||||
{DRIVE_PAD_SPEED_MODES.map((mode) => {
|
||||
const active = speedMode === mode.id;
|
||||
const speedValue =
|
||||
@@ -177,7 +177,7 @@ export default function ControlPadPanel({ disabled = false }) {
|
||||
<button
|
||||
key={mode.id}
|
||||
type="button"
|
||||
className={`mobile-touch-control min-h-9 rounded-md px-1 text-xs font-semibold ${
|
||||
className={`mobile-touch-control rounded-md px-1 text-xs font-semibold ${compact ? 'min-h-7' : 'min-h-9'} ${
|
||||
active
|
||||
? 'bg-cyan-300 text-slate-950'
|
||||
: 'bg-slate-800 text-slate-200'
|
||||
@@ -186,7 +186,7 @@ export default function ControlPadPanel({ disabled = false }) {
|
||||
disabled={disabled}
|
||||
>
|
||||
<span className="block leading-tight">{mode.label}</span>
|
||||
<span className="block font-mono text-[0.7rem] leading-tight">{speedValue}</span>
|
||||
{!compact ? <span className="block font-mono text-[0.7rem] leading-tight">{speedValue}</span> : null}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
@@ -194,6 +194,7 @@ export default function ControlPadPanel({ disabled = false }) {
|
||||
<div className="mobile-touch-control min-h-0 flex-1">
|
||||
<FloatingJoystick
|
||||
activeInputLabel={activeInputLabel}
|
||||
compact={compact}
|
||||
disabled={disabled}
|
||||
onCellChange={handleCellChange}
|
||||
onStop={() => stopDrivePad('stop')}
|
||||
|
||||
@@ -119,7 +119,13 @@ function FloatingPadOverlay({ center, size, activeCellId }) {
|
||||
);
|
||||
}
|
||||
|
||||
export default function FloatingJoystick({ activeInputLabel = 'stop', disabled, onCellChange, onStop }) {
|
||||
export default function FloatingJoystick({
|
||||
activeInputLabel = 'stop',
|
||||
compact = false,
|
||||
disabled,
|
||||
onCellChange,
|
||||
onStop,
|
||||
}) {
|
||||
const containerRef = useRef(null);
|
||||
const pointerIdRef = useRef(null);
|
||||
const activePadRef = useRef(null);
|
||||
@@ -209,7 +215,7 @@ export default function FloatingJoystick({ activeInputLabel = 'stop', disabled,
|
||||
<div
|
||||
ref={containerRef}
|
||||
role="presentation"
|
||||
className="mobile-touch-control mobile-drag-control relative flex h-full min-h-[10rem] w-full select-none items-center justify-center overflow-hidden text-slate-100"
|
||||
className={`mobile-touch-control mobile-drag-control relative flex h-full w-full select-none items-center justify-center overflow-hidden text-slate-100 ${compact ? 'min-h-[7rem]' : 'min-h-[10rem]'}`}
|
||||
// Pointer drags are the whole control model here, so this inline value
|
||||
// reinforces the utility class even if future class churn changes it.
|
||||
style={{ touchAction: 'none' }}
|
||||
@@ -222,17 +228,23 @@ export default function FloatingJoystick({ activeInputLabel = 'stop', disabled,
|
||||
}}
|
||||
onContextMenu={(event) => event.preventDefault()}
|
||||
>
|
||||
<div className="pointer-events-none absolute inset-x-0 top-0 border-b border-slate-700 bg-slate-950 px-1.5 py-0.5 text-center">
|
||||
{/* The readout lives inside the pointer target instead of above it, so the
|
||||
visual indicator does not consume any non-drivable space on small phones. */}
|
||||
<span className="font-mono text-xs font-semibold text-cyan-200">
|
||||
{activeInputLabel}
|
||||
</span>
|
||||
</div>
|
||||
<div className="pointer-events-none flex flex-col items-center gap-0.5 px-2 pt-5 text-center">
|
||||
<span className="text-sm font-semibold text-slate-100">drive pad</span>
|
||||
<span className="text-xs leading-tight text-slate-300">hold and drag</span>
|
||||
</div>
|
||||
{!compact ? (
|
||||
<div className="pointer-events-none absolute inset-x-0 top-0 border-b border-slate-700 bg-slate-950 px-1.5 py-0.5 text-center">
|
||||
{/* The readout lives inside the pointer target instead of above it, so the
|
||||
visual indicator does not consume any non-drivable space on small phones. */}
|
||||
<span className="font-mono text-xs font-semibold text-cyan-200">
|
||||
{activeInputLabel}
|
||||
</span>
|
||||
</div>
|
||||
) : null}
|
||||
{!compact ? (
|
||||
<div className="pointer-events-none flex flex-col items-center gap-0.5 px-2 pt-5 text-center">
|
||||
<span className="text-sm font-semibold text-slate-100">drive pad</span>
|
||||
<span className="text-xs leading-tight text-slate-300">hold and drag</span>
|
||||
</div>
|
||||
) : (
|
||||
<div className="pointer-events-none h-8 w-8 rounded-full border border-cyan-300/60 bg-cyan-300/20" />
|
||||
)}
|
||||
</div>
|
||||
{activePad ? (
|
||||
<FloatingPadOverlay
|
||||
|
||||
@@ -0,0 +1,813 @@
|
||||
// PTZ Camera UI
|
||||
// Purpose: Integrates the single PTZ camera into the main rover UI flow as a
|
||||
// queueable controllable target instead of a VIP-panel card.
|
||||
// Scope: Owns PTZ entry card and fullscreen composition; PTZ command authority,
|
||||
// queue ownership, and stream authorization remain server-owned.
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
|
||||
import { createPortal } from 'react-dom';
|
||||
import CardFrame from '../CardFrame/index.jsx';
|
||||
import ChatPanel from '../ChatPanel/index.jsx';
|
||||
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx';
|
||||
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
|
||||
import PtzLiveVideo, { PTZ_CAMERA_ID } from '../PtzLiveVideo/index.jsx';
|
||||
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
|
||||
import QueueTargetRow, { QueueUserChips } from '../QueueTargetRow/index.jsx';
|
||||
import TurnsOverlay from '../HudOverlays/TurnsOverlay/index.jsx';
|
||||
import KeyPill from '../vip/VipAudioUploadCard/KeyPill.jsx';
|
||||
import { useControlActions, useControlSelector } from '../../controls/index.js';
|
||||
import { formatKeyLabel } from '../../controls/keymapUtils.js';
|
||||
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
|
||||
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
|
||||
import { useSharedClock } from '../../hooks/useSharedClock.js';
|
||||
import { isFeatureEnabled } from '../../lib/features.js';
|
||||
import { trackAnalyticsEvent } from '../../analytics/index.js';
|
||||
|
||||
const PTZ_DEFAULT_COLOR = '#38bdf8';
|
||||
|
||||
function formatRemaining(deadline, now) {
|
||||
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - now) / 1000));
|
||||
if (!remaining) return '--';
|
||||
const minutes = Math.floor(remaining / 60);
|
||||
const seconds = remaining % 60;
|
||||
return `${minutes}:${String(seconds).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
function isSpotlightOn(light = {}) {
|
||||
if (typeof light?.on === 'boolean') return light.on;
|
||||
const raw = light?.state;
|
||||
if (typeof raw === 'string') {
|
||||
const normalized = raw.trim().toLowerCase();
|
||||
return !['', '0', 'off', 'false'].includes(normalized);
|
||||
}
|
||||
return Boolean(Number(raw));
|
||||
}
|
||||
|
||||
function normalizeIrMode(mode) {
|
||||
const normalized = String(mode || '').trim().toLowerCase();
|
||||
if (normalized === 'on') return 'On';
|
||||
if (normalized === 'off') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function nextIrMode(currentMode) {
|
||||
const current = normalizeIrMode(currentMode);
|
||||
if (current === 'Auto') return 'On';
|
||||
if (current === 'On') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function normalizePtzQueue(ptz = null) {
|
||||
/*
|
||||
The PTZ service exposes the current operator separately from the waiting
|
||||
queue, while the rover queue row expects one ordered queue plus a current id.
|
||||
Normalizing once keeps every PTZ surface consistent with the shared queue
|
||||
renderer without making that renderer understand PTZ service internals.
|
||||
*/
|
||||
const currentId = ptz?.operatorSocketId || null;
|
||||
const waiting = Array.isArray(ptz?.queue)
|
||||
? ptz.queue.map((entry) => entry?.socketId || entry).filter(Boolean)
|
||||
: [];
|
||||
const queue = currentId ? [currentId, ...waiting.filter((id) => id !== currentId)] : waiting;
|
||||
const nextId = currentId ? waiting[0] || null : queue[0] || null;
|
||||
return { queue, currentId, nextId };
|
||||
}
|
||||
|
||||
function usePtzQueueLookup(ptz = null) {
|
||||
const users = useSessionSelector((state) => state.session?.users ?? []);
|
||||
return useCallback(
|
||||
(socketId) => {
|
||||
const fromUsers = users.find((entry) => entry.socketId === socketId);
|
||||
if (fromUsers) return fromUsers;
|
||||
if (ptz?.operatorSocketId === socketId) {
|
||||
return { socketId, nickname: ptz?.operatorLabel || null, role: null };
|
||||
}
|
||||
const fromQueue = Array.isArray(ptz?.queue)
|
||||
? ptz.queue.find((entry) => (entry?.socketId || entry) === socketId)
|
||||
: null;
|
||||
return {
|
||||
socketId,
|
||||
nickname: fromQueue?.label || null,
|
||||
role: null,
|
||||
};
|
||||
},
|
||||
[ptz, users],
|
||||
);
|
||||
}
|
||||
|
||||
function PtzSnapshotPreview({ feed, label = 'PTZ Camera', className = 'h-full w-full' }) {
|
||||
return (
|
||||
<div className={`relative overflow-hidden bg-black ${className}`}>
|
||||
{feed?.objectUrl ? (
|
||||
<img src={feed.objectUrl} alt={label} className="h-full w-full object-contain" />
|
||||
) : (
|
||||
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
|
||||
)}
|
||||
{/*
|
||||
Snapshot mode should look like the regular rover video player: the
|
||||
camera name belongs to the surrounding card/menu, while the media pane
|
||||
only exposes stream health in the small top-left diagnostic overlay.
|
||||
*/}
|
||||
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
|
||||
<div className="flex flex-col gap-0.5 leading-none">
|
||||
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function StatusRow({ label, value, tone = '' }) {
|
||||
return (
|
||||
<div className="flex items-center justify-between gap-1 text-xs">
|
||||
<span className="text-slate-400">{label}</span>
|
||||
<span className={`min-w-0 truncate font-medium ${tone || 'text-slate-100'}`}>{value}</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzStatePanel({ ptz, compact = false }) {
|
||||
const now = useSharedClock(1000, Boolean(ptz?.deadline));
|
||||
const spotlightOn = isSpotlightOn(ptz?.light);
|
||||
const irMode = normalizeIrMode(ptz?.ir?.state);
|
||||
const publisher = ptz?.publisher || {};
|
||||
const publisherStatus = publisher.running
|
||||
? 'running'
|
||||
: publisher.restartAt
|
||||
? 'restarting'
|
||||
: publisher.lastEvent || 'stopped';
|
||||
const mode = ptz?.isOperator ? 'operator' : ptz?.queuedPosition ? `queued ${ptz.queuedPosition}` : 'spectator';
|
||||
|
||||
return (
|
||||
<CardFrame title="Camera state" bodyClassName="space-y-0.5 p-1 text-sm">
|
||||
<StatusRow label="Mode" value={mode} tone={ptz?.isOperator ? 'text-emerald-300' : ''} />
|
||||
<StatusRow label="Operator" value={ptz?.operatorLabel || 'none'} />
|
||||
<StatusRow label="Remaining" value={formatRemaining(ptz?.deadline, now)} />
|
||||
<StatusRow label="Spotlight" value={spotlightOn ? 'On' : 'Off'} tone={spotlightOn ? 'text-emerald-300' : 'text-slate-200'} />
|
||||
<StatusRow label="Infrared mode" value={irMode} />
|
||||
<StatusRow label="Stream" value={ptz?.status || ptz?.error || 'idle'} tone={ptz?.error ? 'text-amber-300' : ''} />
|
||||
{!compact ? <StatusRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} /> : null}
|
||||
{ptz?.blocked?.message ? (
|
||||
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
|
||||
{ptz.blocked.message}
|
||||
</div>
|
||||
) : null}
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzQueueSummary({ ptz, title = 'PTZ queue' }) {
|
||||
const selfId = useSessionSelector((state) => state.session?.socketId || null);
|
||||
const lookupUser = usePtzQueueLookup(ptz);
|
||||
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
|
||||
|
||||
return (
|
||||
<CardFrame title={title} bodyClassName="space-y-0.5 p-1 text-sm">
|
||||
<QueueUserChips
|
||||
targetId={ptz?.id || PTZ_CAMERA_ID}
|
||||
queue={queue}
|
||||
currentId={currentId}
|
||||
nextId={nextId}
|
||||
selfId={selfId}
|
||||
lookupUser={lookupUser}
|
||||
/>
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzLightingControls({ ptz, disabled = false }) {
|
||||
const { ptzSpotlight, ptzIr } = useSessionActions();
|
||||
const [busy, setBusy] = useState('');
|
||||
const spotlightOn = isSpotlightOn(ptz?.light);
|
||||
const irMode = normalizeIrMode(ptz?.ir?.state);
|
||||
|
||||
const toggleSpotlight = async (nextOn) => {
|
||||
if (disabled) return;
|
||||
setBusy('spotlight');
|
||||
try {
|
||||
await ptzSpotlight({ state: nextOn ? 1 : 0 });
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
const cycleIr = async () => {
|
||||
if (disabled) return;
|
||||
setBusy('ir');
|
||||
try {
|
||||
await ptzIr({ state: nextIrMode(irMode) });
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
|
||||
<GPIOToggleControl
|
||||
label="Spotlight"
|
||||
on={spotlightOn}
|
||||
disabled={disabled || busy === 'spotlight'}
|
||||
onToggle={toggleSpotlight}
|
||||
heightClass="min-h-14"
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
|
||||
disabled={disabled || busy === 'ir'}
|
||||
onClick={cycleIr}
|
||||
>
|
||||
<span className="text-sm font-semibold">Infrared</span>
|
||||
<span className="rounded bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzMobileZoomButtons({ disabled = false }) {
|
||||
const { nudgeServo, stopAllMotion } = useControlActions();
|
||||
const repeatTimerRef = useRef(null);
|
||||
|
||||
const stopZoom = useCallback(() => {
|
||||
/*
|
||||
Mobile zoom is intentionally routed through the normal camera-up/down
|
||||
control action instead of emitting PTZ socket commands directly. That
|
||||
keeps the zoom buttons on the same path as keyboard/gamepad camera tilt,
|
||||
and the PTZ adapter remains the one place that translates "camera nudge"
|
||||
into Reolink zoom pulses.
|
||||
*/
|
||||
if (repeatTimerRef.current) {
|
||||
clearInterval(repeatTimerRef.current);
|
||||
repeatTimerRef.current = null;
|
||||
}
|
||||
stopAllMotion();
|
||||
}, [stopAllMotion]);
|
||||
|
||||
const startZoom = useCallback(
|
||||
(direction) => (event) => {
|
||||
/*
|
||||
Send an immediate nudge and then repeat while held. The adapter turns
|
||||
each nudge into a short zoom pulse, so repeating the standard action is
|
||||
the simplest way to get continuous hold-to-zoom without adding another
|
||||
PTZ-specific command loop.
|
||||
*/
|
||||
event.preventDefault();
|
||||
if (disabled) return;
|
||||
stopZoom();
|
||||
nudgeServo(direction);
|
||||
repeatTimerRef.current = setInterval(() => {
|
||||
nudgeServo(direction);
|
||||
}, 120);
|
||||
},
|
||||
[disabled, nudgeServo, stopZoom],
|
||||
);
|
||||
const stopFromPointer = useCallback(
|
||||
(event) => {
|
||||
event?.preventDefault?.();
|
||||
if (disabled) return;
|
||||
stopZoom();
|
||||
},
|
||||
[disabled, stopZoom],
|
||||
);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
/*
|
||||
A touch surface can unmount during orientation changes or fullscreen
|
||||
close while a pointer is still down. Clear the repeat timer here so a
|
||||
held zoom button cannot keep firing camera-up/down actions after the
|
||||
mobile controls have disappeared.
|
||||
*/
|
||||
if (repeatTimerRef.current) {
|
||||
clearInterval(repeatTimerRef.current);
|
||||
repeatTimerRef.current = null;
|
||||
}
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
|
||||
disabled={disabled}
|
||||
onPointerDown={startZoom(-1)}
|
||||
onPointerUp={stopFromPointer}
|
||||
onPointerCancel={stopFromPointer}
|
||||
onPointerLeave={stopFromPointer}
|
||||
onContextMenu={(event) => event.preventDefault()}
|
||||
>
|
||||
Zoom out
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
|
||||
disabled={disabled}
|
||||
onPointerDown={startZoom(1)}
|
||||
onPointerUp={stopFromPointer}
|
||||
onPointerCancel={stopFromPointer}
|
||||
onPointerLeave={stopFromPointer}
|
||||
onContextMenu={(event) => event.preventDefault()}
|
||||
>
|
||||
Zoom in
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzMobileControlsPanel({ ptz, disabled = false }) {
|
||||
return (
|
||||
<div className="mobile-touch-control space-y-0.5">
|
||||
<PtzMobileZoomButtons disabled={disabled} />
|
||||
<div className="mobile-touch-control h-44 min-h-0">
|
||||
{/*
|
||||
Reuse the rover control pad so touch intent still enters the normal
|
||||
control system. The PTZ adapter translates that same drive vector into
|
||||
pan/tilt commands only while this user is the PTZ operator.
|
||||
*/}
|
||||
<ControlPadPanel compact disabled={disabled} />
|
||||
</div>
|
||||
<PtzLightingControls ptz={ptz} disabled={disabled} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function keyLabelFor(keymap, actionId) {
|
||||
return formatKeyLabel(keymap?.[actionId]?.[0]);
|
||||
}
|
||||
|
||||
function PtzControlReference() {
|
||||
const keymap = useControlSelector((control) => control.state.keymap);
|
||||
const rows = [
|
||||
['Tilt up', 'driveForward'],
|
||||
['Tilt down', 'driveBackward'],
|
||||
['Pan left', 'driveLeft'],
|
||||
['Pan right', 'driveRight'],
|
||||
['Zoom in', 'cameraUp'],
|
||||
['Zoom out', 'cameraDown'],
|
||||
['Spotlight', 'headlightToggle'],
|
||||
['Infrared mode', 'laserToggle'],
|
||||
];
|
||||
|
||||
return (
|
||||
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
|
||||
{rows.map(([label, actionId]) => (
|
||||
<div key={label} className="surface flex items-center justify-between gap-1">
|
||||
<span className="text-slate-400">{label}</span>
|
||||
<KeyPill label={keyLabelFor(keymap, actionId)} />
|
||||
</div>
|
||||
))}
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzPresetPanel({ ptz }) {
|
||||
const role = useSessionSelector((state) => state.session?.role || null);
|
||||
const {
|
||||
ptzListPresets,
|
||||
ptzGotoPreset,
|
||||
ptzCreatePreset,
|
||||
ptzRemovePreset,
|
||||
} = useSessionActions();
|
||||
const [name, setName] = useState('');
|
||||
const [busy, setBusy] = useState('');
|
||||
const presets = Array.isArray(ptz?.presets) ? ptz.presets : [];
|
||||
const isPresetAdmin = role === 'admin' || role === 'lockdown';
|
||||
const canMoveToPreset = Boolean(ptz?.isOperator);
|
||||
|
||||
const refreshPresets = async () => {
|
||||
if (busy) return;
|
||||
setBusy('refresh');
|
||||
try {
|
||||
/*
|
||||
Presets live on the camera, not in browser state. A manual refresh gives
|
||||
admins a simple recovery path if another admin or the camera's native
|
||||
app changes preset storage while this UI is already open.
|
||||
*/
|
||||
await ptzListPresets();
|
||||
} catch (err) {
|
||||
alert(err.message || 'Failed to refresh PTZ presets.');
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
const goToPreset = async (preset) => {
|
||||
if (!canMoveToPreset || busy || !preset?.token) return;
|
||||
setBusy(`goto:${preset.token}`);
|
||||
try {
|
||||
/*
|
||||
Moving to a preset is a physical camera move, so the server still checks
|
||||
that this browser owns the active PTZ turn before accepting the command.
|
||||
*/
|
||||
await ptzGotoPreset({ token: preset.token });
|
||||
} catch (err) {
|
||||
alert(err.message || 'Failed to move to PTZ preset.');
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
const createPreset = async (event) => {
|
||||
event.preventDefault();
|
||||
const trimmed = name.trim();
|
||||
if (!isPresetAdmin || busy || !trimmed) return;
|
||||
setBusy('create');
|
||||
try {
|
||||
/*
|
||||
ONVIF setPreset stores the camera's current physical position. The UI
|
||||
only sends the admin's label; the server supplies the active profile
|
||||
token so browser code does not need to know camera profile internals.
|
||||
*/
|
||||
await ptzCreatePreset({ name: trimmed });
|
||||
setName('');
|
||||
} catch (err) {
|
||||
alert(err.message || 'Failed to create PTZ preset.');
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
const removePreset = async (preset) => {
|
||||
if (!isPresetAdmin || busy || !preset?.token) return;
|
||||
const confirmed = window.confirm(`Remove preset "${preset.name}"?`);
|
||||
if (!confirmed) return;
|
||||
setBusy(`remove:${preset.token}`);
|
||||
try {
|
||||
/*
|
||||
The token is the camera's durable preset identifier. Names are only UI
|
||||
labels and may not be unique, so deletion always targets the token.
|
||||
*/
|
||||
await ptzRemovePreset({ token: preset.token });
|
||||
} catch (err) {
|
||||
alert(err.message || 'Failed to remove PTZ preset.');
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<CardFrame
|
||||
title="Position presets"
|
||||
fillHeight
|
||||
actions={(
|
||||
<button type="button" className="button-dark text-xs" disabled={Boolean(busy)} onClick={refreshPresets}>
|
||||
Refresh
|
||||
</button>
|
||||
)}
|
||||
bodyClassName="flex min-h-0 flex-col gap-1 p-1 text-xs"
|
||||
>
|
||||
{ptz?.presetsError ? (
|
||||
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-amber-100">
|
||||
{ptz.presetsError}
|
||||
</div>
|
||||
) : null}
|
||||
<div className="min-h-0 flex-1 space-y-0.5 overflow-y-auto">
|
||||
{presets.length ? presets.map((preset) => {
|
||||
const gotoBusy = busy === `goto:${preset.token}`;
|
||||
const removeBusy = busy === `remove:${preset.token}`;
|
||||
return (
|
||||
<div key={preset.token} className="surface grid grid-cols-[minmax(0,1fr)_auto] items-center gap-1">
|
||||
<button
|
||||
type="button"
|
||||
className="button-dark min-w-0 truncate text-left text-xs disabled:opacity-50"
|
||||
disabled={!canMoveToPreset || Boolean(busy)}
|
||||
onClick={() => goToPreset(preset)}
|
||||
title={canMoveToPreset ? `Move to ${preset.name}` : 'Your PTZ turn must be active'}
|
||||
>
|
||||
{gotoBusy ? 'Moving...' : preset.name}
|
||||
</button>
|
||||
{isPresetAdmin ? (
|
||||
<button
|
||||
type="button"
|
||||
className="button-dark text-xs text-rose-200 disabled:opacity-50"
|
||||
disabled={Boolean(busy)}
|
||||
onClick={() => removePreset(preset)}
|
||||
>
|
||||
{removeBusy ? 'Removing...' : 'Remove'}
|
||||
</button>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}) : (
|
||||
<div className="rounded border border-slate-700 bg-black/30 p-2 text-center text-slate-400">
|
||||
No presets saved.
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{isPresetAdmin ? (
|
||||
<form className="grid grid-cols-[minmax(0,1fr)_auto] gap-1" onSubmit={createPreset}>
|
||||
<input
|
||||
className="min-w-0 rounded border border-slate-700 bg-black px-2 py-1 text-xs text-slate-100 outline-none focus:border-cyan-300"
|
||||
value={name}
|
||||
maxLength={60}
|
||||
disabled={Boolean(busy)}
|
||||
onChange={(event) => setName(event.target.value)}
|
||||
placeholder="Preset name"
|
||||
/>
|
||||
<button type="submit" className="button-dark text-xs" disabled={Boolean(busy) || !name.trim()}>
|
||||
{busy === 'create' ? 'Saving...' : 'Save'}
|
||||
</button>
|
||||
</form>
|
||||
) : null}
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function buildPtzTurnModel(ptz, selfId) {
|
||||
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
|
||||
const isActive = Boolean(ptz?.isOperator);
|
||||
const isQueued = Boolean(ptz?.queuedPosition);
|
||||
return {
|
||||
enabled: Boolean(ptz && (isActive || isQueued || currentId)),
|
||||
targetId: ptz?.id || PTZ_CAMERA_ID,
|
||||
activeId: currentId,
|
||||
nextId,
|
||||
isActive,
|
||||
isNext: Boolean(selfId && nextId === selfId),
|
||||
deadline: ptz?.deadline || null,
|
||||
idleDeadline: null,
|
||||
showNotTurnNotice: Boolean(!isActive && (isQueued || currentId || queue.length)),
|
||||
showPreviewReason: false,
|
||||
};
|
||||
}
|
||||
|
||||
function PtzMediaPane({ ptz, open, framed = true }) {
|
||||
const isOperator = Boolean(ptz?.isOperator);
|
||||
const selfId = useSessionSelector((state) => state.session?.socketId || null);
|
||||
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !isOperator });
|
||||
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
|
||||
const turnModel = useMemo(() => buildPtzTurnModel(ptz, selfId), [ptz, selfId]);
|
||||
const media = (
|
||||
<>
|
||||
{isOperator ? (
|
||||
<PtzLiveVideo enabled={open} startMuted={false} />
|
||||
) : (
|
||||
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
|
||||
)}
|
||||
<TurnsOverlay turnModel={turnModel} />
|
||||
</>
|
||||
);
|
||||
|
||||
if (!framed) {
|
||||
return <div className="relative h-full min-h-0 w-full overflow-hidden bg-black">{media}</div>;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex h-full min-h-0 w-full items-center justify-center overflow-hidden bg-black">
|
||||
<div className="relative aspect-video max-h-full w-full max-w-full overflow-hidden bg-black">
|
||||
{media}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzDesktopFullscreen({ ptz, releasePending }) {
|
||||
return (
|
||||
<div className="grid h-full min-h-0 grid-rows-[minmax(0,1fr)_minmax(7rem,0.22fr)] gap-0.5 overflow-hidden p-0.5">
|
||||
<div className="flex min-h-0 min-w-0 gap-0.5 overflow-hidden">
|
||||
<main className="min-h-0 shrink-0 overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
|
||||
<PtzMediaPane ptz={ptz} open framed />
|
||||
</main>
|
||||
<aside className="flex min-h-0 min-w-56 flex-1 flex-col gap-0.5 overflow-y-auto bg-neutral-950 text-sm">
|
||||
<PtzQueueSummary ptz={ptz} />
|
||||
{ptz?.isOperator ? (
|
||||
<PtzLightingControls ptz={ptz} />
|
||||
) : (
|
||||
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
|
||||
Live PTZ controls unlock when your camera turn is active.
|
||||
</CardFrame>
|
||||
)}
|
||||
<PtzControlReference />
|
||||
<PtzStatePanel ptz={ptz} />
|
||||
<ReplaySourcesPanel panelId="ptz-controller-replay" />
|
||||
</aside>
|
||||
</div>
|
||||
<div className="grid min-h-0 grid-cols-[minmax(0,1.6fr)_minmax(16rem,0.7fr)] gap-0.5 overflow-hidden">
|
||||
<ChatPanel fillHeight title="Chat" allowSpectatorInput inputTarget="overlay" />
|
||||
<PtzPresetPanel ptz={ptz} />
|
||||
</div>
|
||||
{releasePending ? (
|
||||
<div className="pointer-events-none absolute bottom-1 right-1 rounded bg-black/80 px-2 py-1 text-xs text-slate-200">
|
||||
Closing...
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzMobileFullscreen({ ptz, layout, onClose, releasePending = false }) {
|
||||
const landscape = layout === 'mobile-landscape';
|
||||
const topHeightClass = landscape ? 'h-full min-h-[calc(100dvh-0.25rem)]' : 'h-[48dvh]';
|
||||
const topGridClass = landscape
|
||||
? 'grid-cols-[minmax(0,1fr)_13rem]'
|
||||
: 'grid-cols-[minmax(0,1fr)_11rem]';
|
||||
|
||||
return (
|
||||
<div className="flex min-h-0 flex-1 flex-col gap-0.5 overflow-y-auto p-0.5">
|
||||
<section className={`mobile-touch-control grid ${topHeightClass} min-h-48 shrink-0 ${topGridClass} gap-0.5`}>
|
||||
<main className="relative min-h-0 overflow-hidden bg-black">
|
||||
<button
|
||||
type="button"
|
||||
className="absolute left-1 top-1 z-50 rounded border border-white/40 bg-black/80 px-2 py-1 text-xs font-semibold text-white shadow disabled:opacity-50"
|
||||
disabled={releasePending}
|
||||
onClick={onClose}
|
||||
>
|
||||
Close
|
||||
</button>
|
||||
<PtzMediaPane ptz={ptz} open framed={false} />
|
||||
</main>
|
||||
<aside className="min-h-0 overflow-y-auto">
|
||||
<PtzMobileControlsPanel ptz={ptz} disabled={!ptz?.isOperator} />
|
||||
</aside>
|
||||
</section>
|
||||
<section className="grid gap-0.5 md:grid-cols-[minmax(0,1fr)_minmax(0,0.7fr)]">
|
||||
<ChatPanel title="Chat" allowSpectatorInput inputTarget="overlay" />
|
||||
<div className="space-y-0.5">
|
||||
<PtzQueueSummary ptz={ptz} />
|
||||
<PtzPresetPanel ptz={ptz} />
|
||||
<PtzStatePanel ptz={ptz} compact />
|
||||
<ReplaySourcesPanel panelId="ptz-controller-replay-mobile" />
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function PtzFullscreenController({ open, onClose, layout = 'desktop' }) {
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const { ptzRelease } = useSessionActions();
|
||||
const { stopAllMotion } = useControlActions();
|
||||
const [releasePending, setReleasePending] = useState(false);
|
||||
const isMobile = layout === 'mobile-portrait' || layout === 'mobile-landscape';
|
||||
|
||||
const releaseAndClose = useCallback(async () => {
|
||||
if (releasePending) return;
|
||||
setReleasePending(true);
|
||||
try {
|
||||
/*
|
||||
Stop first so a held key/pointer cannot leave ONVIF continuous movement
|
||||
running while the server removes this socket from the PTZ queue.
|
||||
*/
|
||||
stopAllMotion?.();
|
||||
await ptzRelease();
|
||||
onClose?.();
|
||||
} finally {
|
||||
setReleasePending(false);
|
||||
}
|
||||
}, [onClose, ptzRelease, releasePending, stopAllMotion]);
|
||||
|
||||
if (!open) return null;
|
||||
|
||||
const controller = (
|
||||
/*
|
||||
The PTZ controller needs to cover the driver page, but it must not become
|
||||
the top-most application layer. Global fullscreen overlays like help,
|
||||
quickstart, mode gates, and connection warnings are still part of the
|
||||
active app state while PTZ is open, so this portal intentionally sits
|
||||
below their z-30+ overlay stack instead of hiding them.
|
||||
*/
|
||||
<div className="fixed inset-0 z-20 h-[100dvh] w-[100vw] overflow-hidden bg-black text-slate-100">
|
||||
<CardFrame
|
||||
title={isMobile ? '' : ptz?.name || 'PTZ Camera'}
|
||||
actions={isMobile ? null : (
|
||||
<button type="button" className="button-dark text-xs" disabled={releasePending} onClick={releaseAndClose}>
|
||||
Close
|
||||
</button>
|
||||
)}
|
||||
hideHeader={isMobile}
|
||||
fillHeight
|
||||
clipOverflow={false}
|
||||
className="h-[100dvh] w-[100vw] rounded-none border-0 !bg-black"
|
||||
bodyClassName="relative min-h-0 flex-1"
|
||||
>
|
||||
{isMobile ? (
|
||||
<PtzMobileFullscreen
|
||||
ptz={ptz}
|
||||
layout={layout}
|
||||
onClose={releaseAndClose}
|
||||
releasePending={releasePending}
|
||||
/>
|
||||
) : (
|
||||
<PtzDesktopFullscreen ptz={ptz} releasePending={releasePending} />
|
||||
)}
|
||||
</CardFrame>
|
||||
</div>
|
||||
);
|
||||
|
||||
return createPortal(controller, document.body);
|
||||
}
|
||||
|
||||
export default function PtzQueueCard({ layout = 'desktop' }) {
|
||||
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
|
||||
const role = useSessionSelector((state) => state.session?.role || null);
|
||||
const selfId = useSessionSelector((state) => state.session?.socketId || null);
|
||||
const { ptzClaim, ptzRelease } = useSessionActions();
|
||||
const lookupUser = usePtzQueueLookup(ptz);
|
||||
const { queue, currentId, nextId } = normalizePtzQueue(ptz);
|
||||
const now = useSharedClock(1000, Boolean(ptz?.deadline));
|
||||
const [controllerOpen, setControllerOpen] = useState(false);
|
||||
const [pending, setPending] = useState(false);
|
||||
const canUse = Boolean(ptz?.canUse || isVerified || role === 'admin' || role === 'lockdown');
|
||||
const isParticipant = Boolean(ptz?.isOperator || ptz?.queuedPosition);
|
||||
const timerLabel = ptz?.isOperator && ptz?.deadline ? `${formatRemaining(ptz.deadline, now)} left` : '';
|
||||
|
||||
if (!featureEnabled) return null;
|
||||
|
||||
const handleRequest = async () => {
|
||||
if (!canUse || pending) return;
|
||||
if (isParticipant) {
|
||||
setControllerOpen(true);
|
||||
return;
|
||||
}
|
||||
setPending(true);
|
||||
trackAnalyticsEvent('ptz_queue_join', { layout });
|
||||
try {
|
||||
const response = await ptzClaim();
|
||||
/*
|
||||
The server is authoritative for whether the click became an active turn
|
||||
or a queued wait. Open only after it confirms one of those states so a
|
||||
dock-guard rejection does not strand the user in fullscreen.
|
||||
*/
|
||||
if (response?.state?.isOperator || response?.state?.queuedPosition) {
|
||||
setControllerOpen(true);
|
||||
}
|
||||
trackAnalyticsEvent('ptz_queue_join_result', { layout, status: 'accepted' });
|
||||
} catch (err) {
|
||||
trackAnalyticsEvent('ptz_queue_join_result', {
|
||||
layout,
|
||||
status: 'failed',
|
||||
reason: err?.message || 'unknown',
|
||||
});
|
||||
alert(err.message || 'PTZ request failed.');
|
||||
} finally {
|
||||
setPending(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleLeave = async () => {
|
||||
if (pending) return;
|
||||
setPending(true);
|
||||
try {
|
||||
await ptzRelease();
|
||||
} catch (err) {
|
||||
alert(err.message || 'Failed to leave PTZ camera.');
|
||||
} finally {
|
||||
setPending(false);
|
||||
}
|
||||
};
|
||||
|
||||
const actionLabel = pending
|
||||
? '...'
|
||||
: ptz?.isOperator
|
||||
? 'Open'
|
||||
: ptz?.queuedPosition
|
||||
? 'Open'
|
||||
: 'request';
|
||||
|
||||
return (
|
||||
<>
|
||||
<CardFrame title={ptz?.name || 'PTZ camera'} bodyClassName="relative space-y-0.5 text-sm">
|
||||
<ul className="space-y-0.5 text-sm">
|
||||
<QueueTargetRow
|
||||
target={{
|
||||
id: ptz?.id || PTZ_CAMERA_ID,
|
||||
name: ptz?.name || 'PTZ Camera',
|
||||
color: ptz?.color || PTZ_DEFAULT_COLOR,
|
||||
// description: ptz?.isOperator
|
||||
// ? 'Live camera turn active'
|
||||
// : ptz?.queuedPosition
|
||||
// ? `Queue position ${ptz.queuedPosition}`
|
||||
// : 'Pan, tilt, and zoom camera',
|
||||
}}
|
||||
queue={queue}
|
||||
currentId={currentId}
|
||||
nextId={nextId}
|
||||
selfId={selfId}
|
||||
lookupUser={lookupUser}
|
||||
canClick={canUse && !pending}
|
||||
pending={pending}
|
||||
buttonLabel={actionLabel}
|
||||
batteryLabel={ptz?.isOperator ? 'LIVE' : ptz?.queuedPosition ? `#${ptz.queuedPosition}` : '--'}
|
||||
batteryClassName={ptz?.isOperator ? 'text-emerald-300' : ptz?.queuedPosition ? 'text-sky-300' : 'text-slate-400'}
|
||||
timerLabel={timerLabel}
|
||||
onRequest={handleRequest}
|
||||
showAction={canUse}
|
||||
/>
|
||||
</ul>
|
||||
{isParticipant ? (
|
||||
<button type="button" className="button-dark w-full text-xs" disabled={pending} onClick={handleLeave}>
|
||||
Leave PTZ queue
|
||||
</button>
|
||||
) : null}
|
||||
{!canUse ? (
|
||||
<div className="absolute inset-0 z-10 flex items-center justify-center rounded bg-black/75 px-2 text-center text-sm font-semibold text-slate-100">
|
||||
Verify your account to use the PTZ camera.
|
||||
</div>
|
||||
) : null}
|
||||
</CardFrame>
|
||||
<PtzFullscreenController open={controllerOpen} onClose={() => setControllerOpen(false)} layout={layout} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
// PTZ Live Video
|
||||
// Purpose: Plays the single PTZ camera WHEP stream with the same fresh-session retry loop used by rover video.
|
||||
// Scope: Owns browser-side WHEP playback/retry only; server authorization and snapshot fallback policy stay outside.
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { useVideoRequests } from '../../hooks/useVideoRequests.js';
|
||||
import { WhepPlayer } from '../../lib/whepPlayer.js';
|
||||
import { RESTART_DELAY_MS } from '../RoverMediaPlayer/constants.js';
|
||||
|
||||
export const PTZ_CAMERA_ID = 'ptz-camera';
|
||||
|
||||
const PTZ_AUDIO_RETRY_MS = 1000;
|
||||
const TERMINAL_WHEP_STATES = new Set(['error', 'failed', 'disconnected', 'closed']);
|
||||
const AUTHORIZATION_ERROR_RE = /not authorized/i;
|
||||
|
||||
function isAuthorizationError(error) {
|
||||
return AUTHORIZATION_ERROR_RE.test(String(error || ''));
|
||||
}
|
||||
|
||||
export default function PtzLiveVideo({
|
||||
enabled = true,
|
||||
startMuted = true,
|
||||
className = 'relative h-full w-full bg-black',
|
||||
videoClassName = 'h-full w-full object-contain',
|
||||
statusClassName = 'pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]',
|
||||
fallback = null,
|
||||
}) {
|
||||
const videoRef = useRef(null);
|
||||
const retryTimerRef = useRef(null);
|
||||
const playTimerRef = useRef(null);
|
||||
const enabledRef = useRef(enabled);
|
||||
const fallbackRef = useRef(false);
|
||||
const playerGenerationRef = useRef(0);
|
||||
const [status, setStatus] = useState('idle');
|
||||
const [detail, setDetail] = useState(null);
|
||||
const [restartToken, setRestartToken] = useState(0);
|
||||
const sources = useVideoRequests(
|
||||
[{ type: 'ptz', id: PTZ_CAMERA_ID, key: PTZ_CAMERA_ID }],
|
||||
{ enabled, version: restartToken },
|
||||
);
|
||||
const source = sources[PTZ_CAMERA_ID] || null;
|
||||
const shouldUseFallback = Boolean(source?.error && isAuthorizationError(source.error));
|
||||
|
||||
useEffect(() => {
|
||||
enabledRef.current = enabled;
|
||||
}, [enabled]);
|
||||
|
||||
useEffect(() => {
|
||||
fallbackRef.current = shouldUseFallback;
|
||||
}, [shouldUseFallback]);
|
||||
|
||||
useEffect(() => {
|
||||
if (enabled && !shouldUseFallback) return undefined;
|
||||
/*
|
||||
A retry that was scheduled before the server denied live access should not
|
||||
keep firing in snapshot mode. Clear it when live playback is no longer the
|
||||
active display policy.
|
||||
*/
|
||||
if (retryTimerRef.current) {
|
||||
clearTimeout(retryTimerRef.current);
|
||||
retryTimerRef.current = null;
|
||||
}
|
||||
return undefined;
|
||||
}, [enabled, shouldUseFallback]);
|
||||
|
||||
const scheduleRestart = useCallback(() => {
|
||||
if (!enabledRef.current || fallbackRef.current) return;
|
||||
/*
|
||||
WHEP sessions are one-shot browser/server negotiations. When the camera
|
||||
reboots, the old PeerConnection and token can look alive enough to keep a
|
||||
black element on screen, but they are not useful anymore. Bumping this
|
||||
token forces useVideoRequests to ask the server for a new MediaMTX auth
|
||||
session before creating the next WhepPlayer.
|
||||
*/
|
||||
clearTimeout(retryTimerRef.current);
|
||||
retryTimerRef.current = setTimeout(() => {
|
||||
retryTimerRef.current = null;
|
||||
setRestartToken(Date.now());
|
||||
}, RESTART_DELAY_MS);
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled || shouldUseFallback || !source?.url || !videoRef.current) return undefined;
|
||||
let active = true;
|
||||
const generation = playerGenerationRef.current + 1;
|
||||
playerGenerationRef.current = generation;
|
||||
const player = new WhepPlayer({
|
||||
url: source.url,
|
||||
token: source.token,
|
||||
video: videoRef.current,
|
||||
startMuted,
|
||||
onStatus: (nextStatus, info) => {
|
||||
/*
|
||||
Old PeerConnection callbacks can arrive after React has already
|
||||
cleaned up this effect for a newer token. Only the currently-owned
|
||||
generation is allowed to update status or schedule another restart.
|
||||
*/
|
||||
if (!active || playerGenerationRef.current !== generation) return;
|
||||
const normalized = String(nextStatus || '').toLowerCase();
|
||||
setStatus(nextStatus || 'unknown');
|
||||
setDetail(info || null);
|
||||
if (TERMINAL_WHEP_STATES.has(normalized)) {
|
||||
scheduleRestart();
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
player.start().catch((err) => {
|
||||
if (!active || playerGenerationRef.current !== generation) return;
|
||||
setStatus('error');
|
||||
setDetail(err.message || 'WHEP start failed');
|
||||
scheduleRestart();
|
||||
});
|
||||
|
||||
return () => {
|
||||
/*
|
||||
Mark inactive before stop() because WhepPlayer reports "stopped" during
|
||||
normal cleanup. Cleanup-driven stops should not immediately schedule the
|
||||
next retry; only the replacement effect should own the new connection.
|
||||
*/
|
||||
active = false;
|
||||
player.stop();
|
||||
};
|
||||
}, [enabled, scheduleRestart, shouldUseFallback, source?.token, source?.url, startMuted]);
|
||||
|
||||
useEffect(() => {
|
||||
const video = videoRef.current;
|
||||
if (!enabled || shouldUseFallback || !source?.url || !video) return undefined;
|
||||
const generation = playerGenerationRef.current;
|
||||
|
||||
const handleEnded = () => {
|
||||
if (playerGenerationRef.current !== generation) return;
|
||||
setStatus('stopped');
|
||||
setDetail('ended');
|
||||
scheduleRestart();
|
||||
};
|
||||
const handleError = () => {
|
||||
if (playerGenerationRef.current !== generation) return;
|
||||
setStatus('error');
|
||||
setDetail(video.error?.message || 'video element error');
|
||||
scheduleRestart();
|
||||
};
|
||||
|
||||
video.addEventListener('ended', handleEnded);
|
||||
video.addEventListener('error', handleError);
|
||||
return () => {
|
||||
video.removeEventListener('ended', handleEnded);
|
||||
video.removeEventListener('error', handleError);
|
||||
};
|
||||
}, [enabled, scheduleRestart, shouldUseFallback, source?.url]);
|
||||
|
||||
useEffect(() => {
|
||||
const video = videoRef.current;
|
||||
if (!enabled || shouldUseFallback || startMuted || !source?.url || !video) {
|
||||
if (playTimerRef.current) clearInterval(playTimerRef.current);
|
||||
playTimerRef.current = null;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/*
|
||||
PTZ carries inline Opus audio. When the operator opened the camera from a
|
||||
user gesture, keep retrying audible playback so browser autoplay timing
|
||||
does not leave the element permanently muted after a reconnect.
|
||||
*/
|
||||
const attemptPlay = () => {
|
||||
const target = videoRef.current;
|
||||
if (!target) return;
|
||||
target.muted = false;
|
||||
if (!target.paused && !target.ended) return;
|
||||
target.play().catch(() => {});
|
||||
};
|
||||
|
||||
attemptPlay();
|
||||
playTimerRef.current = setInterval(attemptPlay, PTZ_AUDIO_RETRY_MS);
|
||||
return () => {
|
||||
if (playTimerRef.current) clearInterval(playTimerRef.current);
|
||||
playTimerRef.current = null;
|
||||
};
|
||||
}, [enabled, shouldUseFallback, source?.url, startMuted, status]);
|
||||
|
||||
useEffect(() => () => {
|
||||
if (retryTimerRef.current) clearTimeout(retryTimerRef.current);
|
||||
if (playTimerRef.current) clearInterval(playTimerRef.current);
|
||||
}, []);
|
||||
|
||||
if (shouldUseFallback && typeof fallback === 'function') {
|
||||
return fallback({ source, status, detail });
|
||||
}
|
||||
|
||||
const displayStatus = source?.error || detail || status;
|
||||
|
||||
return (
|
||||
<div className={className}>
|
||||
{source?.url ? (
|
||||
<video ref={videoRef} className={videoClassName} playsInline autoPlay muted={startMuted} />
|
||||
) : (
|
||||
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">
|
||||
{source?.error || 'Waiting for PTZ video...'}
|
||||
</div>
|
||||
)}
|
||||
{/*
|
||||
PTZ video uses the same low-profile diagnostic shape as the rover
|
||||
players: no in-frame camera title, just a compact top-corner status.
|
||||
This keeps the media pane visually interchangeable with rover streams
|
||||
while still exposing WHEP/session failures during reconnects.
|
||||
*/}
|
||||
<div className={statusClassName}>
|
||||
<div className="flex flex-col gap-0.5 leading-none">
|
||||
<span>Status: {displayStatus}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
// Queue Target Row
|
||||
// Purpose: Renders the shared queue-row visual language used by rover queues and PTZ.
|
||||
// Scope: Owns row chrome, queue chips, timer labels, and row/button event plumbing;
|
||||
// callers still own target-specific permission checks and request actions.
|
||||
import RoverLabel from '../RoverLabel/index.jsx';
|
||||
|
||||
function classNames(...values) {
|
||||
return values.filter(Boolean).join(' ');
|
||||
}
|
||||
|
||||
function roleColors(role) {
|
||||
switch (role) {
|
||||
case 'admin':
|
||||
case 'lockdown':
|
||||
return 'text-amber-300';
|
||||
case 'spectator':
|
||||
return 'text-slate-400';
|
||||
default:
|
||||
return 'text-sky-300';
|
||||
}
|
||||
}
|
||||
|
||||
function formatQueueUserLabel(user, selfId) {
|
||||
/*
|
||||
Queue chips need to be readable even when a socket has no nickname yet.
|
||||
Keeping the socket-prefix fallback here means rover queues and PTZ queues
|
||||
degrade identically instead of each target inventing its own anonymous label.
|
||||
*/
|
||||
if (!user) return '';
|
||||
const base = user.nickname || user.label || user.socketId?.slice(0, 6) || 'unknown';
|
||||
if (user.socketId && user.socketId === selfId) {
|
||||
return `${base} (you)`;
|
||||
}
|
||||
return base;
|
||||
}
|
||||
|
||||
export function QueueUserChips({
|
||||
targetId,
|
||||
queue = [],
|
||||
currentId = null,
|
||||
nextId = null,
|
||||
selfId = null,
|
||||
lookupUser,
|
||||
}) {
|
||||
if (!queue.length) {
|
||||
return <p className="text-[0.7rem] text-slate-500">No queue.</p>;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex flex-wrap items-center gap-0.5">
|
||||
{queue.map((socketId, idx) => {
|
||||
const user = lookupUser?.(socketId) || { socketId, nickname: null, role: null };
|
||||
const isCurrent = socketId === currentId;
|
||||
const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
|
||||
/*
|
||||
These classes intentionally mirror the original rover queue styling.
|
||||
PTZ feeds the same current/next model into this component, so the user
|
||||
does not have to learn a different visual vocabulary for camera turns.
|
||||
*/
|
||||
const highlightClass = isCurrent
|
||||
? 'bg-sky-600 text-white ring-2 ring-amber-300'
|
||||
: isNext
|
||||
? 'bg-emerald-700/60 text-emerald-100 ring-1 ring-emerald-300/70'
|
||||
: 'bg-slate-800 text-slate-200';
|
||||
return (
|
||||
<span
|
||||
key={`${targetId}-${socketId}-${idx}`}
|
||||
className={`flex items-center gap-0.5 rounded px-1 text-[0.7rem] ${highlightClass}`}
|
||||
>
|
||||
<span className={`${roleColors(user.role)} font-semibold`}>
|
||||
{formatQueueUserLabel(user, selfId)}
|
||||
</span>
|
||||
{isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>}
|
||||
{isNext && <span className="text-[0.65rem] text-emerald-100">next</span>}
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function QueueTargetRow({
|
||||
target,
|
||||
queue = [],
|
||||
currentId = null,
|
||||
nextId = null,
|
||||
selfId = null,
|
||||
lookupUser,
|
||||
canClick = false,
|
||||
pending = false,
|
||||
locked = false,
|
||||
lockedBlocked = false,
|
||||
privateOpen = false,
|
||||
buttonLabel = '',
|
||||
batteryLabel = '',
|
||||
batteryClassName = 'text-slate-400',
|
||||
timerLabel = '',
|
||||
thumbnailUrl = '',
|
||||
onRequest,
|
||||
showAction = true,
|
||||
}) {
|
||||
const targetId = String(target?.id || '');
|
||||
const targetLabel = target?.label || target?.name || targetId;
|
||||
|
||||
return (
|
||||
<li
|
||||
className={classNames(
|
||||
'surface flex flex-wrap items-start justify-between gap-0.5',
|
||||
canClick && 'cursor-pointer',
|
||||
locked
|
||||
? 'bg-red-900/40'
|
||||
: privateOpen
|
||||
? 'bg-amber-700/35 border border-amber-200/30'
|
||||
: null,
|
||||
)}
|
||||
onClick={() => {
|
||||
/*
|
||||
The whole row is a large target because queue selection is one of the
|
||||
main touch/click actions on the page. The caller still decides whether
|
||||
clicking is currently allowed, so disabled PTZ and locked rover states
|
||||
cannot accidentally request control through the shared renderer.
|
||||
*/
|
||||
if (!canClick) return;
|
||||
onRequest?.(targetId);
|
||||
}}
|
||||
>
|
||||
{thumbnailUrl ? (
|
||||
<img
|
||||
src={thumbnailUrl}
|
||||
alt=""
|
||||
className="h-8 w-10 shrink-0 rounded border border-slate-700 bg-black object-cover"
|
||||
loading="lazy"
|
||||
/>
|
||||
) : null}
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
|
||||
<div className="flex items-center justify-between gap-0.5">
|
||||
<div className="flex min-w-0 items-center gap-0.5">
|
||||
<p className="min-w-0 flex items-center gap-0.5 whitespace-nowrap text-slate-200">
|
||||
<RoverLabel
|
||||
rover={target?.rover || null}
|
||||
roverId={target?.roverId ?? targetId}
|
||||
name={targetLabel}
|
||||
color={target?.color || null}
|
||||
fallback={targetId}
|
||||
/>
|
||||
{target?.description ? (
|
||||
<span className="min-w-0 flex-1 truncate text-[0.7rem] text-slate-400">
|
||||
{target.description}
|
||||
</span>
|
||||
) : null}
|
||||
</p>
|
||||
{timerLabel ? (
|
||||
<span className="rounded bg-slate-800 px-1 text-[0.7rem] text-slate-200">
|
||||
{timerLabel}
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
{batteryLabel ? (
|
||||
<span className={classNames('text-[0.75rem] font-semibold', batteryClassName)}>
|
||||
{batteryLabel}
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
<QueueUserChips
|
||||
targetId={targetId}
|
||||
queue={queue}
|
||||
currentId={currentId}
|
||||
nextId={nextId}
|
||||
selfId={selfId}
|
||||
lookupUser={lookupUser}
|
||||
/>
|
||||
</div>
|
||||
{showAction ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={(event) => {
|
||||
/*
|
||||
Stop propagation so button clicks do not double-fire the row
|
||||
request. This keeps mouse, touch, and keyboard activation on the
|
||||
explicit button consistent with clicking the row background.
|
||||
*/
|
||||
event.stopPropagation();
|
||||
onRequest?.(targetId);
|
||||
}}
|
||||
disabled={pending || lockedBlocked || !canClick}
|
||||
className={classNames(
|
||||
'button-dark disabled:opacity-40',
|
||||
locked && 'bg-red-600/70 text-white hover:bg-red-600',
|
||||
)}
|
||||
>
|
||||
{buttonLabel}
|
||||
</button>
|
||||
) : null}
|
||||
</li>
|
||||
);
|
||||
}
|
||||
@@ -134,7 +134,9 @@ export default function ReplaySourcesPanel({ panelId = 'replay-sources', fillHei
|
||||
const grouped = useMemo(() => {
|
||||
return {
|
||||
rovers: sources.filter((source) => source.type === 'rover'),
|
||||
rooms: sources.filter((source) => source.type === 'room'),
|
||||
// PTZ is presented with room cameras because there is only one fixed room
|
||||
// PTZ camera and it should not create a separate source category.
|
||||
rooms: sources.filter((source) => source.type === 'room' || source.type === 'ptz'),
|
||||
};
|
||||
}, [sources]);
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import HelpPanel from '../HelpPanel/index.jsx';
|
||||
import ChatPanel from '../ChatPanel/index.jsx';
|
||||
import { LinkButtonsPanel } from '../UserListPanel/index.jsx';
|
||||
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
|
||||
import PtzQueueCard from '../PtzCamera/index.jsx';
|
||||
import Tabs, { Tab, TabList, TabPanel, TabPanels } from '../Tabs/index.jsx';
|
||||
import TopDownMap from '../TopDownMap/index.jsx';
|
||||
import DriveDockAction from '../DriveDockAction/index.jsx';
|
||||
@@ -201,7 +202,10 @@ function QueueReplayLinksRow() {
|
||||
<div className="min-w-0 basis-0 grow-[0.9]">
|
||||
<ReplaySourcesPanel panelId="replay-sources-desktop" fillHeight />
|
||||
</div>
|
||||
<LinkButtonsPanel className="min-w-0 basis-0 grow-[0.75]" />
|
||||
<div className={`min-w-0 basis-0 grow-[0.75] ${themeStackClass}`}>
|
||||
<LinkButtonsPanel fillHeight={false} />
|
||||
<PtzQueueCard layout="desktop" />
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -425,7 +429,7 @@ export default function RightPaneTabs({ layout, onOpenHelpOverlay }) {
|
||||
|
||||
{/* VIP tab */}
|
||||
<TabPanel id="vip" keepMounted>
|
||||
<VipPanel isActive={activeTab === 'vip'} />
|
||||
<VipPanel isActive={activeTab === 'vip'} layout={layout} />
|
||||
</TabPanel>
|
||||
|
||||
{/* help tab */}
|
||||
|
||||
@@ -5,6 +5,7 @@ import { useEffect, useMemo, useRef, useState } from 'react';
|
||||
import { useSessionSelector } from '../../context/SessionContext.jsx';
|
||||
import { useSettingsNamespace } from '../../settings/index.js';
|
||||
import { useRoomCameraSnapshots } from '../../hooks/useRoomCameraSnapshots.js';
|
||||
import { PTZ_CAMERA_ID, usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
|
||||
import RoomCameraFeed from '../RoomCameraFeed/index.jsx';
|
||||
import CardFrame from '../CardFrame/index.jsx';
|
||||
import { isFeatureEnabled } from '../../lib/features.js';
|
||||
@@ -105,11 +106,14 @@ function useCameraPanelSubscriptionGate() {
|
||||
}
|
||||
|
||||
export default function RoomCameraPanel(props) {
|
||||
const enabled = useSessionSelector((state) => isFeatureEnabled(state, 'roomCameras'));
|
||||
const enabled = useSessionSelector((state) =>
|
||||
isFeatureEnabled(state, 'roomCameras') || isFeatureEnabled(state, 'ptzCamera'),
|
||||
);
|
||||
|
||||
/*
|
||||
Room-camera visibility belongs with the room-camera panel. This keeps every
|
||||
route free to mount the panel without duplicating the server feature rule.
|
||||
Camera-panel visibility belongs with the panel. PTZ is included here because
|
||||
the user-facing request is "show it as a room camera"; the rendering path
|
||||
still uses the same RoomCameraFeed tile as ordinary room cameras.
|
||||
*/
|
||||
if (!enabled) return null;
|
||||
|
||||
@@ -123,13 +127,35 @@ function RoomCameraPanelContent({
|
||||
hideHeader = false,
|
||||
panelId = null,
|
||||
}) {
|
||||
const roomCamerasEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'roomCameras'));
|
||||
const ptzEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const cameras = useSessionSelector((state) => state.session?.roomCameras || []);
|
||||
const cameraSources = useMemo(() => {
|
||||
const base = roomCamerasEnabled ? cameras : [];
|
||||
if (!ptzEnabled || !ptz) return base;
|
||||
/*
|
||||
PTZ snapshots use a different socket namespace from room cameras, but the
|
||||
display model is intentionally the same: an id, a label, and a feed object.
|
||||
Marking the type lets the subscription layer stay separate while the tile
|
||||
renderer remains shared.
|
||||
*/
|
||||
return [
|
||||
...base,
|
||||
{
|
||||
id: PTZ_CAMERA_ID,
|
||||
name: ptz.name || 'PTZ Camera',
|
||||
type: 'ptz',
|
||||
},
|
||||
];
|
||||
}, [cameras, ptz, ptzEnabled, roomCamerasEnabled]);
|
||||
const cameraIds = useMemo(
|
||||
() => cameras.map((camera) => camera.id),
|
||||
[cameras],
|
||||
() => cameraSources.filter((camera) => camera.type !== 'ptz').map((camera) => camera.id),
|
||||
[cameraSources],
|
||||
);
|
||||
const { panelRef, isPanelVisible } = useCameraPanelSubscriptionGate();
|
||||
const feedMap = useRoomCameraSnapshots(cameraIds, { enabled: isPanelVisible });
|
||||
const ptzFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: isPanelVisible && ptzEnabled });
|
||||
const { value: orientationSettings, save: saveOrientationSettings } = useSettingsNamespace('roomCameraPanels', {});
|
||||
const [orientation, setOrientation] = useState(() =>
|
||||
normalizeOrientation(
|
||||
@@ -152,7 +178,7 @@ function RoomCameraPanelContent({
|
||||
);
|
||||
const containerClass =
|
||||
effectiveOrientation === 'vertical' ? 'flex flex-col gap-0.5' : 'grid gap-0.5 md:grid-cols-2';
|
||||
const showLayoutToggle = !hideLayoutToggle && !forcedOrientation && cameras.length > 0;
|
||||
const showLayoutToggle = !hideLayoutToggle && !forcedOrientation && cameraSources.length > 0;
|
||||
const applyOrientation = (next) => {
|
||||
setOrientation(next);
|
||||
if (panelId) {
|
||||
@@ -160,7 +186,7 @@ function RoomCameraPanelContent({
|
||||
}
|
||||
};
|
||||
|
||||
if (cameras.length === 0) {
|
||||
if (cameraSources.length === 0) {
|
||||
return (
|
||||
<div ref={panelRef}>
|
||||
<EmptyState />
|
||||
@@ -195,8 +221,8 @@ function RoomCameraPanelContent({
|
||||
bodyClassName="space-y-0.5 text-base"
|
||||
>
|
||||
<div className={containerClass}>
|
||||
{cameras.map((camera) => {
|
||||
const feed = feedMap[camera.id] || null;
|
||||
{cameraSources.map((camera) => {
|
||||
const feed = camera.type === 'ptz' ? ptzFeeds[camera.id] || null : feedMap[camera.id] || null;
|
||||
return (
|
||||
<article key={camera.id} className="w-full space-y-0.5 p-0.5">
|
||||
{/* <header className="space-y-0.5">
|
||||
|
||||
@@ -5,17 +5,13 @@ import { useMemo, useState } from 'react';
|
||||
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
|
||||
import { useSharedClock } from '../../hooks/useSharedClock.js';
|
||||
import CardFrame from '../CardFrame/index.jsx';
|
||||
import RoverLabel from '../RoverLabel/index.jsx';
|
||||
import QueueTargetRow from '../QueueTargetRow/index.jsx';
|
||||
import { trackAnalyticsEvent } from '../../analytics/index.js';
|
||||
import { openExternalRover } from '../../lib/interInstanceTransfer.js';
|
||||
import { ExternalInstancesCompact } from '../InterInstancePanel/index.jsx';
|
||||
import { isFeatureEnabled } from '../../lib/features.js';
|
||||
import { useSettingsNamespace } from '../../settings/index.js';
|
||||
|
||||
function classNames(...values) {
|
||||
return values.filter(Boolean).join(' ');
|
||||
}
|
||||
|
||||
function formatBattery(rover) {
|
||||
const percent = rover?.batteryState?.percentDisplay;
|
||||
if (percent == null) return '--';
|
||||
@@ -29,27 +25,6 @@ function batteryClass(rover) {
|
||||
return 'text-emerald-300';
|
||||
}
|
||||
|
||||
function roleColors(role) {
|
||||
switch (role) {
|
||||
case 'admin':
|
||||
case 'lockdown':
|
||||
return 'text-amber-300';
|
||||
case 'spectator':
|
||||
return 'text-slate-400';
|
||||
default:
|
||||
return 'text-sky-300';
|
||||
}
|
||||
}
|
||||
|
||||
function formatLabel(user, selfId) {
|
||||
if (!user) return '';
|
||||
const base = user.nickname || user.socketId?.slice(0, 6) || 'unknown';
|
||||
if (user.socketId && user.socketId === selfId) {
|
||||
return `${base} (you)`;
|
||||
}
|
||||
return base;
|
||||
}
|
||||
|
||||
export default function RoverQueuesPanel({
|
||||
title = 'Rovers',
|
||||
roster: rosterOverride = null,
|
||||
@@ -192,129 +167,59 @@ export default function RoverQueuesPanel({
|
||||
) : (
|
||||
<ul className="space-y-0.5 text-sm">
|
||||
{rosterItems.map((rover) => {
|
||||
const roverId = String(rover.id);
|
||||
const info = turnQueues?.[roverId] || null;
|
||||
const queue = info?.queue || [];
|
||||
const deadline = info?.idleDeadline || info?.deadline || null;
|
||||
const remainingSeconds =
|
||||
deadline && deadline > now ? Math.ceil((deadline - now) / 1000) : deadline ? 0 : null;
|
||||
const currentId = info?.current || null;
|
||||
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1;
|
||||
const nextId =
|
||||
queue.length > 1
|
||||
? currentIdx >= 0
|
||||
? queue[(currentIdx + 1) % queue.length]
|
||||
: queue[0]
|
||||
: null;
|
||||
const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId);
|
||||
const isSelfNext = Boolean(selfId && nextId && nextId === selfId);
|
||||
const showTimer = remainingSeconds != null && (isSelfCurrent || isSelfNext);
|
||||
const isPrivateOpen = Boolean(rover?.private?.enabled && rover?.private?.open);
|
||||
const isGrantedClosedPrivate = Boolean(rover?.private?.enabled && !rover?.private?.open);
|
||||
const locked = Boolean(rover.locked);
|
||||
const lockedBlocked = locked && (externalMode || (!adminCapable && !isGrantedClosedPrivate));
|
||||
const lockLabel = rover.lockReason ? `locked: ${rover.lockReason}` : 'locked';
|
||||
const buttonLabel = pending[roverId]
|
||||
? '...'
|
||||
: lockedBlocked
|
||||
? lockLabel
|
||||
: externalMode
|
||||
? 'Open'
|
||||
: 'request';
|
||||
const canClickRow = canRequest && !lockedBlocked && !pending[roverId];
|
||||
return (
|
||||
<li
|
||||
key={rover.id}
|
||||
className={classNames(
|
||||
'surface flex flex-wrap items-start justify-between gap-0.5',
|
||||
canClickRow && 'cursor-pointer',
|
||||
locked
|
||||
? 'bg-red-900/40'
|
||||
: isPrivateOpen
|
||||
? 'bg-amber-700/35 border border-amber-200/30'
|
||||
: null,
|
||||
)}
|
||||
onClick={() => {
|
||||
if (!canClickRow) return;
|
||||
handleRequest(rover.id);
|
||||
}}
|
||||
>
|
||||
{externalMode && rover?.snapshots?.latestUrl ? (
|
||||
<img
|
||||
src={rover.snapshots.latestUrl}
|
||||
alt=""
|
||||
className="h-8 w-10 shrink-0 rounded border border-slate-700 bg-black object-cover"
|
||||
loading="lazy"
|
||||
/>
|
||||
) : null}
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
|
||||
<div className="flex items-center justify-between gap-0.5">
|
||||
<div className="flex min-w-0 items-center gap-0.5">
|
||||
<p className="min-w-0 flex items-center gap-0.5 whitespace-nowrap text-slate-200">
|
||||
<RoverLabel rover={rover} fallback={roverId} />
|
||||
{rover.description ? (
|
||||
<span className="min-w-0 flex-1 truncate text-[0.7rem] text-slate-400">
|
||||
{rover.description}
|
||||
</span>
|
||||
) : null}
|
||||
</p>
|
||||
{showTimer ? (
|
||||
<span className="rounded bg-slate-800 px-1 text-[0.7rem] text-slate-200">
|
||||
{isSelfCurrent ? `${remainingSeconds}s left` : `Your turn in ${remainingSeconds}s`}
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
<span className={classNames('text-[0.75rem] font-semibold', batteryClass(rover))}>
|
||||
{formatBattery(rover)}
|
||||
</span>
|
||||
</div>
|
||||
{queue.length === 0 ? (
|
||||
<p className="text-[0.7rem] text-slate-500">No queue.</p>
|
||||
) : (
|
||||
<div className="flex flex-wrap items-center gap-0.5">
|
||||
{queue.map((socketId, idx) => {
|
||||
const user = lookupUser(socketId);
|
||||
const isCurrent = socketId === currentId;
|
||||
const isNext = Boolean(nextId && socketId === nextId && !isCurrent);
|
||||
const highlightClass = isCurrent
|
||||
? 'bg-sky-600 text-white ring-2 ring-amber-300'
|
||||
: isNext
|
||||
? 'bg-emerald-700/60 text-emerald-100 ring-1 ring-emerald-300/70'
|
||||
: 'bg-slate-800 text-slate-200';
|
||||
return (
|
||||
<span
|
||||
key={`${roverId}-${socketId}-${idx}`}
|
||||
className={`flex items-center gap-0.5 rounded px-1 text-[0.7rem] ${highlightClass}`}
|
||||
>
|
||||
<span className={`${roleColors(user.role)} font-semibold`}>
|
||||
{formatLabel(user, selfId)}
|
||||
</span>
|
||||
{isCurrent && <span className="text-[0.65rem] text-slate-200">now</span>}
|
||||
{isNext && <span className="text-[0.65rem] text-emerald-100">next</span>}
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{canRequest ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={(event) => {
|
||||
event.stopPropagation();
|
||||
handleRequest(rover.id);
|
||||
}}
|
||||
disabled={pending[roverId] || lockedBlocked}
|
||||
className={classNames(
|
||||
'button-dark disabled:opacity-40',
|
||||
locked && 'bg-red-600/70 text-white hover:bg-red-600',
|
||||
)}
|
||||
>
|
||||
{buttonLabel}
|
||||
</button>
|
||||
) : null}
|
||||
</li>
|
||||
);
|
||||
const roverId = String(rover.id);
|
||||
const info = turnQueues?.[roverId] || null;
|
||||
const queue = info?.queue || [];
|
||||
const deadline = info?.idleDeadline || info?.deadline || null;
|
||||
const remainingSeconds =
|
||||
deadline && deadline > now ? Math.ceil((deadline - now) / 1000) : deadline ? 0 : null;
|
||||
const currentId = info?.current || null;
|
||||
const currentIdx = currentId ? queue.findIndex((id) => id === currentId) : -1;
|
||||
const nextId =
|
||||
queue.length > 1
|
||||
? currentIdx >= 0
|
||||
? queue[(currentIdx + 1) % queue.length]
|
||||
: queue[0]
|
||||
: null;
|
||||
const isSelfCurrent = Boolean(selfId && currentId && currentId === selfId);
|
||||
const isSelfNext = Boolean(selfId && nextId && nextId === selfId);
|
||||
const showTimer = remainingSeconds != null && (isSelfCurrent || isSelfNext);
|
||||
const isPrivateOpen = Boolean(rover?.private?.enabled && rover?.private?.open);
|
||||
const isGrantedClosedPrivate = Boolean(rover?.private?.enabled && !rover?.private?.open);
|
||||
const locked = Boolean(rover.locked);
|
||||
const lockedBlocked = locked && (externalMode || (!adminCapable && !isGrantedClosedPrivate));
|
||||
const lockLabel = rover.lockReason ? `locked: ${rover.lockReason}` : 'locked';
|
||||
const buttonLabel = pending[roverId]
|
||||
? '...'
|
||||
: lockedBlocked
|
||||
? lockLabel
|
||||
: externalMode
|
||||
? 'Open'
|
||||
: 'request';
|
||||
const canClickRow = canRequest && !lockedBlocked && !pending[roverId];
|
||||
return (
|
||||
<QueueTargetRow
|
||||
key={rover.id}
|
||||
target={{ ...rover, rover, roverId, id: roverId }}
|
||||
queue={queue}
|
||||
currentId={currentId}
|
||||
nextId={nextId}
|
||||
selfId={selfId}
|
||||
lookupUser={lookupUser}
|
||||
canClick={canClickRow}
|
||||
pending={Boolean(pending[roverId])}
|
||||
locked={locked}
|
||||
lockedBlocked={lockedBlocked}
|
||||
privateOpen={isPrivateOpen}
|
||||
buttonLabel={buttonLabel}
|
||||
batteryLabel={formatBattery(rover)}
|
||||
batteryClassName={batteryClass(rover)}
|
||||
timerLabel={showTimer ? (isSelfCurrent ? `${remainingSeconds}s left` : `Your turn in ${remainingSeconds}s`) : ''}
|
||||
thumbnailUrl={externalMode ? rover?.snapshots?.latestUrl : ''}
|
||||
onRequest={handleRequest}
|
||||
showAction={Boolean(canRequest)}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</ul>
|
||||
)}
|
||||
|
||||
@@ -19,7 +19,7 @@ export function NicknameEntryPanel({ compact = false }) {
|
||||
);
|
||||
}
|
||||
|
||||
export function LinkButtonsPanel({ className = '' }) {
|
||||
export function LinkButtonsPanel({ className = '', fillHeight = true }) {
|
||||
const enabled = useSessionSelector((state) => isFeatureEnabled(state, 'socials'));
|
||||
|
||||
/*
|
||||
@@ -32,7 +32,7 @@ export function LinkButtonsPanel({ className = '' }) {
|
||||
return (
|
||||
<CardFrame
|
||||
title="Links!"
|
||||
fillHeight
|
||||
fillHeight={fillHeight}
|
||||
className={className}
|
||||
bodyClassName="flex flex-1 min-h-0 flex-col gap-0.5 text-base"
|
||||
>
|
||||
|
||||
@@ -0,0 +1,574 @@
|
||||
// Vip PTZ Camera Card
|
||||
// Purpose: Provides the verified-user entry point and fullscreen controller for the single Reolink PTZ camera.
|
||||
// Scope: Owns PTZ UI state only; server-side PTZ ownership, rover handoff, and command authorization remain authoritative.
|
||||
import { useCallback, useMemo, useState } from 'react';
|
||||
import { createPortal } from 'react-dom';
|
||||
import ChatPanel from '../ChatPanel/index.jsx';
|
||||
import CardFrame from '../CardFrame/index.jsx';
|
||||
import GPIOToggleControl from '../GPIOToggleControl/index.jsx';
|
||||
import ControlPadPanel from '../MobileControls/ControlPadPanel.jsx';
|
||||
import PtzLiveVideo from '../PtzLiveVideo/index.jsx';
|
||||
import ReplaySourcesPanel from '../ReplaySourcesPanel/index.jsx';
|
||||
import KeyPill from './VipAudioUploadCard/KeyPill.jsx';
|
||||
import { useSessionActions, useSessionSelector } from '../../context/SessionContext.jsx';
|
||||
import { useControlSelector } from '../../controls/index.js';
|
||||
import { formatKeyLabel } from '../../controls/keymapUtils.js';
|
||||
import { usePtzCameraSnapshots } from '../../hooks/usePtzCameraSnapshot.js';
|
||||
import { isFeatureEnabled } from '../../lib/features.js';
|
||||
|
||||
const PTZ_CAMERA_ID = 'ptz-camera';
|
||||
const PTZ_ZOOM_SPEED = 0.55;
|
||||
|
||||
function formatRemaining(deadline) {
|
||||
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - Date.now()) / 1000));
|
||||
if (!remaining) return '--';
|
||||
const minutes = Math.floor(remaining / 60);
|
||||
const seconds = remaining % 60;
|
||||
return `${minutes}:${String(seconds).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
function isSpotlightOn(light = {}) {
|
||||
if (typeof light?.on === 'boolean') return light.on;
|
||||
const raw = light?.state;
|
||||
if (typeof raw === 'string') {
|
||||
const normalized = raw.trim().toLowerCase();
|
||||
return !['', '0', 'off', 'false'].includes(normalized);
|
||||
}
|
||||
return Boolean(Number(raw));
|
||||
}
|
||||
|
||||
function normalizeIrMode(mode) {
|
||||
const normalized = String(mode || '').trim().toLowerCase();
|
||||
if (normalized === 'on') return 'On';
|
||||
if (normalized === 'off') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function nextIrMode(currentMode) {
|
||||
const current = normalizeIrMode(currentMode);
|
||||
if (current === 'Auto') return 'On';
|
||||
if (current === 'On') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function PtzSnapshotPreview({ feed, label = 'PTZ Camera' }) {
|
||||
return (
|
||||
<div className="relative w-full overflow-hidden bg-black" style={{ aspectRatio: '16 / 9' }}>
|
||||
{feed?.objectUrl ? (
|
||||
<img src={feed.objectUrl} alt={label} className="h-full w-full object-cover" />
|
||||
) : (
|
||||
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">Waiting for snapshot...</div>
|
||||
)}
|
||||
{/*
|
||||
Even on this legacy VIP surface, keep the video pane itself identical
|
||||
to rover media panes: no camera title inside the frame, only the small
|
||||
top-left stream status. Any PTZ name/context belongs to the card chrome
|
||||
around the media, not the media player.
|
||||
*/}
|
||||
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
|
||||
<div className="flex flex-col gap-0.5 leading-none">
|
||||
<span>Status: {feed?.error ? `Error: ${feed.error}` : feed?.status || 'connecting'}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function StatusRow({ label, value, tone = '' }) {
|
||||
return (
|
||||
<div className="flex items-center justify-between gap-1 text-xs">
|
||||
<span className="text-slate-400">{label}</span>
|
||||
<span className={`min-w-0 truncate font-medium ${tone || 'text-slate-100'}`}>{value}</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function formatPublisherProgress(progress = null) {
|
||||
/*
|
||||
ffmpeg progress is intentionally shown as raw operational numbers instead
|
||||
of translated prose. When the PTZ video feels delayed, fps/speed/drop/out
|
||||
time make it obvious whether ffmpeg itself is keeping up or the delay is
|
||||
somewhere before/after the transcoder.
|
||||
*/
|
||||
if (!progress) return null;
|
||||
return [
|
||||
progress.fps ? `fps ${progress.fps}` : null,
|
||||
progress.speed ? `speed ${progress.speed}` : null,
|
||||
progress.drop_frames ? `drop ${progress.drop_frames}` : null,
|
||||
progress.out_time ? `out ${progress.out_time}` : null,
|
||||
].filter(Boolean).join(' | ');
|
||||
}
|
||||
|
||||
function PtzQueueList({ queue = [], operatorLabel = '' }) {
|
||||
const hasQueue = Array.isArray(queue) && queue.length > 0;
|
||||
return (
|
||||
<div className="space-y-0.5">
|
||||
<div className="panel-muted text-xs">Turn queue</div>
|
||||
<div className="space-y-0.5">
|
||||
{operatorLabel ? (
|
||||
<div className="surface flex items-center justify-between gap-1 text-xs">
|
||||
<span className="text-slate-400">Now</span>
|
||||
<span className="min-w-0 truncate text-emerald-200">{operatorLabel}</span>
|
||||
</div>
|
||||
) : null}
|
||||
{hasQueue ? queue.map((entry, index) => (
|
||||
<div key={entry.socketId || `${entry.label}-${index}`} className="surface flex items-center justify-between gap-1 text-xs">
|
||||
<span className="text-slate-400">{index + 1}</span>
|
||||
<span className="min-w-0 truncate text-slate-100">{entry.label || entry.socketId || 'queued user'}</span>
|
||||
</div>
|
||||
)) : (
|
||||
<div className="surface text-xs text-slate-400">No one waiting</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzStatePanel({ ptz, onClose, onRelease, releaseDisabled = false }) {
|
||||
const spotlightOn = isSpotlightOn(ptz?.light);
|
||||
const irMode = normalizeIrMode(ptz?.ir?.state);
|
||||
const publisher = ptz?.publisher || {};
|
||||
const publisherStatus = publisher.running
|
||||
? `running${publisher.pid ? ` ${publisher.pid}` : ''}`
|
||||
: publisher.restartAt
|
||||
? 'restarting'
|
||||
: publisher.lastEvent || 'stopped';
|
||||
const publisherProgress = formatPublisherProgress(publisher.progress);
|
||||
const statusTone = ptz?.error ? 'text-amber-300' : ptz?.isOperator ? 'text-emerald-300' : 'text-slate-100';
|
||||
|
||||
return (
|
||||
<CardFrame
|
||||
title="Camera state"
|
||||
actions={onClose ? <button type="button" className="button-dark text-xs" onClick={onClose}>Close</button> : null}
|
||||
bodyClassName="space-y-0.5 p-1 text-sm"
|
||||
>
|
||||
<StatusRow label="Mode" value={ptz?.isOperator ? 'operator' : ptz?.queuedPosition ? `queued ${ptz.queuedPosition}` : 'spectator'} tone={statusTone} />
|
||||
<StatusRow label="Operator" value={ptz?.operatorLabel || 'none'} />
|
||||
<StatusRow label="Remaining" value={formatRemaining(ptz?.deadline)} />
|
||||
<StatusRow label="Spotlight" value={spotlightOn ? 'On' : 'Off'} tone={spotlightOn ? 'text-emerald-300' : 'text-slate-200'} />
|
||||
<StatusRow label="Infrared mode" value={irMode} />
|
||||
<StatusRow label="Stream" value={ptz?.status || ptz?.error || 'idle'} tone={ptz?.error ? 'text-amber-300' : ''} />
|
||||
<StatusRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} />
|
||||
{publisherProgress ? <StatusRow label="Progress" value={publisherProgress} /> : null}
|
||||
{publisher.lastStderr ? (
|
||||
<div className="surface max-h-24 overflow-y-auto whitespace-pre-wrap break-words font-mono text-[0.68rem] leading-tight text-slate-200">
|
||||
{publisher.lastStderr}
|
||||
</div>
|
||||
) : null}
|
||||
{ptz?.blocked?.message ? (
|
||||
<div className="rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
|
||||
{ptz.blocked.message}
|
||||
</div>
|
||||
) : null}
|
||||
{onRelease ? (
|
||||
<button type="button" className="button-dark w-full text-xs" disabled={releaseDisabled} onClick={onRelease}>
|
||||
Release camera
|
||||
</button>
|
||||
) : null}
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzLightingControls({ ptz, disabled = false }) {
|
||||
const { ptzSpotlight, ptzIr } = useSessionActions();
|
||||
const [busy, setBusy] = useState('');
|
||||
const spotlightOn = isSpotlightOn(ptz?.light);
|
||||
const irMode = normalizeIrMode(ptz?.ir?.state);
|
||||
|
||||
const toggleSpotlight = async (nextOn) => {
|
||||
if (disabled) return;
|
||||
setBusy('spotlight');
|
||||
try {
|
||||
await ptzSpotlight({ state: nextOn ? 1 : 0 });
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
const cycleIr = async () => {
|
||||
if (disabled) return;
|
||||
setBusy('ir');
|
||||
try {
|
||||
await ptzIr({ state: nextIrMode(irMode) });
|
||||
} finally {
|
||||
setBusy('');
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
|
||||
<GPIOToggleControl
|
||||
label="Spotlight"
|
||||
on={spotlightOn}
|
||||
disabled={disabled || busy === 'spotlight'}
|
||||
onToggle={toggleSpotlight}
|
||||
heightClass="min-h-14"
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control flex min-h-14 flex-col items-center justify-center gap-0.5 rounded-xl border-2 border-cyan-300/70 bg-cyan-900 px-1 py-0.75 text-center text-cyan-50 disabled:opacity-50"
|
||||
disabled={disabled || busy === 'ir'}
|
||||
onClick={cycleIr}
|
||||
>
|
||||
<span className="text-sm font-semibold">Infrared</span>
|
||||
<span className="rounded bg-cyan-300 px-1 py-0.5 text-[0.7rem] font-semibold text-cyan-950">{irMode}</span>
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzMobileZoomButtons({ disabled = false }) {
|
||||
const { ptzMove, ptzStop } = useSessionActions();
|
||||
const stopZoom = useCallback(() => {
|
||||
ptzStop().catch(() => {});
|
||||
}, [ptzStop]);
|
||||
const startZoom = useCallback(
|
||||
(direction) => (event) => {
|
||||
/*
|
||||
Mobile needs explicit zoom targets because the regular mobile drive pad
|
||||
is already used for pan/tilt. Desktop does not render these buttons; it
|
||||
uses the mapped camera up/down controls shown in the reference panel.
|
||||
*/
|
||||
event.preventDefault();
|
||||
if (disabled) return;
|
||||
ptzMove({ pan: 0, tilt: 0, zoom: direction * PTZ_ZOOM_SPEED }).catch(() => {});
|
||||
},
|
||||
[disabled, ptzMove],
|
||||
);
|
||||
const stopFromPointer = useCallback(
|
||||
(event) => {
|
||||
event?.preventDefault?.();
|
||||
if (disabled) return;
|
||||
stopZoom();
|
||||
},
|
||||
[disabled, stopZoom],
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="mobile-touch-control grid grid-cols-2 gap-0.5 text-sm">
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
|
||||
disabled={disabled}
|
||||
onPointerDown={startZoom(-1)}
|
||||
onPointerUp={stopFromPointer}
|
||||
onPointerCancel={stopFromPointer}
|
||||
onPointerLeave={stopFromPointer}
|
||||
onContextMenu={(event) => event.preventDefault()}
|
||||
>
|
||||
Zoom out
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="mobile-touch-control button-dark min-h-10 text-xs disabled:opacity-50"
|
||||
disabled={disabled}
|
||||
onPointerDown={startZoom(1)}
|
||||
onPointerUp={stopFromPointer}
|
||||
onPointerCancel={stopFromPointer}
|
||||
onPointerLeave={stopFromPointer}
|
||||
onContextMenu={(event) => event.preventDefault()}
|
||||
>
|
||||
Zoom in
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzMobileControlsPanel({ ptz, disabled = false }) {
|
||||
return (
|
||||
<div className="mobile-touch-control space-y-0.5">
|
||||
<PtzMobileZoomButtons disabled={disabled} />
|
||||
<div className="mobile-touch-control h-44 min-h-0">
|
||||
{/*
|
||||
Reuse the rover mobile movement card instead of building a second PTZ
|
||||
joystick. Its drive vector goes through the shared internal control
|
||||
layer, where the PTZ adapter already converts movement plus speed mode
|
||||
into camera pan/tilt commands.
|
||||
*/}
|
||||
<ControlPadPanel compact disabled={disabled} />
|
||||
</div>
|
||||
<PtzLightingControls ptz={ptz} disabled={disabled} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function keyLabelFor(keymap, actionId) {
|
||||
return formatKeyLabel(keymap?.[actionId]?.[0]);
|
||||
}
|
||||
|
||||
function PtzControlReference() {
|
||||
const keymap = useControlSelector((control) => control.state.keymap);
|
||||
const rows = [
|
||||
['Tilt up', 'driveForward'],
|
||||
['Tilt down', 'driveBackward'],
|
||||
['Pan left', 'driveLeft'],
|
||||
['Pan right', 'driveRight'],
|
||||
['Zoom in', 'cameraUp'],
|
||||
['Zoom out', 'cameraDown'],
|
||||
['Spotlight', 'headlightToggle'],
|
||||
['Infrared mode', 'laserToggle'],
|
||||
];
|
||||
|
||||
return (
|
||||
<CardFrame title="Controls" bodyClassName="space-y-0.5 p-1 text-xs">
|
||||
{rows.map(([label, actionId]) => (
|
||||
<div key={label} className="surface flex items-center justify-between gap-1">
|
||||
<span className="text-slate-400">{label}</span>
|
||||
{/* Use the same key display component as the rest of the UI so PTZ
|
||||
controls read as normal mapped controls instead of custom labels. */}
|
||||
<KeyPill label={keyLabelFor(keymap, actionId)} />
|
||||
</div>
|
||||
))}
|
||||
</CardFrame>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzController({ open, onClose, layout = 'desktop' }) {
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const isOperator = Boolean(ptz?.isOperator);
|
||||
const isMobile = layout === 'mobile-portrait' || layout === 'mobile-landscape';
|
||||
const { ptzRelease } = useSessionActions();
|
||||
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: open && !isOperator });
|
||||
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
|
||||
const [releasePending, setReleasePending] = useState(false);
|
||||
|
||||
if (!open) return null;
|
||||
|
||||
const releaseAndClose = async () => {
|
||||
setReleasePending(true);
|
||||
try {
|
||||
await ptzRelease();
|
||||
onClose();
|
||||
} finally {
|
||||
setReleasePending(false);
|
||||
}
|
||||
};
|
||||
|
||||
const desktopSidebar = (
|
||||
<>
|
||||
|
||||
<div className="shrink-0">
|
||||
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
|
||||
</div>
|
||||
{isOperator ? (
|
||||
<div className="shrink-0">
|
||||
<PtzLightingControls ptz={ptz} />
|
||||
</div>
|
||||
) : (
|
||||
<div className="shrink-0">
|
||||
<CardFrame title="Controls" bodyClassName="p-1 text-xs text-slate-400">
|
||||
Live PTZ controls unlock when your camera turn is active.
|
||||
</CardFrame>
|
||||
</div>
|
||||
)}
|
||||
<div className="shrink-0">
|
||||
<PtzControlReference />
|
||||
</div>
|
||||
<div className="min-h-[12rem] flex-1">
|
||||
{/*
|
||||
The global chat key focuses the input registered by ChatPanel through
|
||||
ChatContext. Keeping a real ChatPanel mounted inside the PTZ fullscreen
|
||||
sidebar lets the normal keyboard path focus chat and lets Enter submit
|
||||
through the existing chat composer form instead of adding PTZ-specific
|
||||
chat handling.
|
||||
*/}
|
||||
<ChatPanel fillHeight title="Chat" />
|
||||
</div>
|
||||
<div className="shrink-0">
|
||||
<ReplaySourcesPanel panelId="ptz-controller-replay" />
|
||||
</div>
|
||||
<div className="shrink-0">
|
||||
<PtzStatePanel
|
||||
ptz={ptz}
|
||||
onClose={onClose}
|
||||
onRelease={isOperator ? releaseAndClose : null}
|
||||
releaseDisabled={releasePending}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
|
||||
const mobileSidebar = (
|
||||
<>
|
||||
<div className="shrink-0">
|
||||
<PtzMobileControlsPanel ptz={ptz} disabled={!isOperator} />
|
||||
</div>
|
||||
<div className="min-h-[10rem] flex-1">
|
||||
{/*
|
||||
Mobile uses the same ChatPanel registration as desktop so the mapped
|
||||
chat key and the on-screen input stay on one shared chat implementation.
|
||||
*/}
|
||||
<ChatPanel fillHeight title="Chat" />
|
||||
</div>
|
||||
<div className="shrink-0">
|
||||
<ReplaySourcesPanel panelId="ptz-controller-replay-mobile" />
|
||||
</div>
|
||||
<div className="shrink-0">
|
||||
<PtzStatePanel
|
||||
ptz={ptz}
|
||||
onClose={onClose}
|
||||
onRelease={isOperator ? releaseAndClose : null}
|
||||
releaseDisabled={releasePending}
|
||||
/>
|
||||
</div>
|
||||
<div className="shrink-0">
|
||||
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
|
||||
const sidebarWidthClass = isMobile
|
||||
? 'grid-cols-[minmax(0,1fr)_14rem]'
|
||||
: 'grid-cols-[minmax(0,1fr)_20rem]';
|
||||
|
||||
const controller = (
|
||||
<div className="fixed inset-0 z-[110] h-[100dvh] w-[100vw] overflow-hidden bg-black text-slate-100">
|
||||
<CardFrame
|
||||
hideHeader
|
||||
fillHeight
|
||||
clipOverflow={false}
|
||||
className="h-[100dvh] w-[100vw] rounded-none border-0 !bg-black"
|
||||
bodyClassName={`grid h-full min-h-0 overflow-hidden ${sidebarWidthClass}`}
|
||||
>
|
||||
<main className="relative min-h-0 min-w-0 bg-black">
|
||||
{isOperator ? (
|
||||
<PtzLiveVideo enabled startMuted={false} />
|
||||
) : (
|
||||
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
|
||||
)}
|
||||
</main>
|
||||
<aside className="flex h-full min-h-0 items-stretch overflow-hidden border-l border-neutral-600 bg-neutral-950 text-sm">
|
||||
<div className="flex min-h-0 w-full flex-col gap-0.5 overflow-y-auto p-0.5">
|
||||
{isMobile ? mobileSidebar : desktopSidebar}
|
||||
</div>
|
||||
</aside>
|
||||
</CardFrame>
|
||||
</div>
|
||||
);
|
||||
|
||||
/*
|
||||
The controller is a true fullscreen surface, so mount it directly under
|
||||
document.body instead of inside the VIP tab/card tree. That keeps tab panel
|
||||
spacing, mobile banners, and parent overflow rules from creating visible
|
||||
gaps around a fixed-position camera interface.
|
||||
*/
|
||||
return createPortal(controller, document.body);
|
||||
}
|
||||
|
||||
export default function VipPtzCameraCard({ onMessage, fullWidth = false, layout = 'desktop' }) {
|
||||
const featureEnabled = useSessionSelector((state) => isFeatureEnabled(state, 'ptzCamera'));
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const isVerified = useSessionSelector((state) => Boolean(state.session?.isVerified));
|
||||
const { ptzClaim, ptzRelease } = useSessionActions();
|
||||
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: Boolean(featureEnabled) });
|
||||
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
|
||||
const [controllerOpen, setControllerOpen] = useState(false);
|
||||
const [pending, setPending] = useState(false);
|
||||
|
||||
const wrapClass = fullWidth ? 'w-full' : 'mx-auto w-full max-w-xl';
|
||||
const queueText = useMemo(() => {
|
||||
if (ptz?.isOperator) return 'Your turn';
|
||||
if (ptz?.queuedPosition) return `Queue position ${ptz.queuedPosition}`;
|
||||
if (ptz?.operatorLabel) return `${ptz.operatorLabel} operating`;
|
||||
return 'Available';
|
||||
}, [ptz?.isOperator, ptz?.operatorLabel, ptz?.queuedPosition]);
|
||||
|
||||
if (!featureEnabled) return null;
|
||||
|
||||
const handleClaim = async () => {
|
||||
setPending(true);
|
||||
onMessage?.('');
|
||||
try {
|
||||
const response = await ptzClaim();
|
||||
/*
|
||||
Requesting the PTZ camera is only enough to open fullscreen after the
|
||||
server confirms the user actually became the operator or entered the
|
||||
queue. Dock-required failures reject before queue entry, so they must
|
||||
leave the user on this card with the dock/charge message visible.
|
||||
*/
|
||||
if (response?.state?.isOperator || response?.state?.queuedPosition) setControllerOpen(true);
|
||||
if (response?.state?.isOperator) onMessage?.('PTZ camera turn active.');
|
||||
else if (response?.state?.queuedPosition) onMessage?.(`Joined PTZ queue at position ${response.state.queuedPosition}.`);
|
||||
} catch (err) {
|
||||
/*
|
||||
Rover-to-rover switch denial uses the browser alert popup for the
|
||||
dock-and-charge message. PTZ claim denial should feel identical because
|
||||
it is enforcing the same "do not abandon an undocked rover" rule.
|
||||
*/
|
||||
alert(err.message || 'PTZ request failed.');
|
||||
onMessage?.(err.message || 'PTZ request failed.');
|
||||
} finally {
|
||||
setPending(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleRelease = async () => {
|
||||
setPending(true);
|
||||
try {
|
||||
await ptzRelease();
|
||||
onMessage?.('Left PTZ camera.');
|
||||
} catch (err) {
|
||||
onMessage?.(err.message || 'Failed to leave PTZ camera.');
|
||||
} finally {
|
||||
setPending(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<CardFrame title="PTZ camera" className={wrapClass} bodyClassName="text-sm text-slate-300">
|
||||
<div className="grid w-full gap-0.5 md:grid-cols-[minmax(16rem,0.8fr)_minmax(0,1.2fr)] md:items-start">
|
||||
<PtzSnapshotPreview feed={snapshot} label={ptz?.name || 'PTZ Camera'} />
|
||||
<div className="flex w-full min-w-0 flex-col gap-0.5">
|
||||
<div className="grid w-full grid-cols-2 gap-0.5 text-left text-xs">
|
||||
<div className="surface-muted p-1">
|
||||
<p className="text-slate-500">State</p>
|
||||
<p className="truncate text-slate-100">{queueText}</p>
|
||||
</div>
|
||||
<div className="surface-muted p-1">
|
||||
<p className="text-slate-500">Remaining</p>
|
||||
<p className="text-slate-100">{formatRemaining(ptz?.deadline)}</p>
|
||||
</div>
|
||||
<div className="surface-muted p-1">
|
||||
<p className="text-slate-500">Spotlight</p>
|
||||
<p className="truncate text-slate-100">{isSpotlightOn(ptz?.light) ? 'On' : 'Off'}</p>
|
||||
</div>
|
||||
<div className="surface-muted p-1">
|
||||
<p className="text-slate-500">Infrared</p>
|
||||
<p className="truncate text-slate-100">{normalizeIrMode(ptz?.ir?.state)}</p>
|
||||
</div>
|
||||
</div>
|
||||
<PtzQueueList queue={ptz?.queue} operatorLabel={ptz?.operatorLabel} />
|
||||
{ptz?.blocked?.message ? (
|
||||
<p className="w-full rounded border border-amber-500/50 bg-amber-950/40 p-1 text-xs text-amber-100">
|
||||
{ptz.blocked.message}
|
||||
</p>
|
||||
) : null}
|
||||
<div className="grid w-full grid-cols-2 gap-0.5">
|
||||
{ptz?.isOperator ? (
|
||||
<>
|
||||
<button type="button" className="button-dark text-xs" onClick={() => setControllerOpen(true)}>
|
||||
Open controller
|
||||
</button>
|
||||
<button type="button" className="button-dark text-xs" disabled={pending} onClick={handleRelease}>
|
||||
Release
|
||||
</button>
|
||||
</>
|
||||
) : (
|
||||
<button
|
||||
type="button"
|
||||
className="button-dark col-span-2 text-xs"
|
||||
disabled={pending || !isVerified}
|
||||
onClick={ptz?.queuedPosition ? handleRelease : handleClaim}
|
||||
>
|
||||
{ptz?.queuedPosition ? 'Leave queue' : pending ? 'Requesting...' : 'Claim camera'}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</CardFrame>
|
||||
<PtzController open={controllerOpen} onClose={() => setControllerOpen(false)} layout={layout} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -61,6 +61,7 @@ export function ChatProvider({ children }) {
|
||||
const [isChatFocused, setIsChatFocused] = useState(false);
|
||||
const panelInputRef = useRef(null);
|
||||
const hudInputRef = useRef(null);
|
||||
const overlayInputRef = useRef(null);
|
||||
const audioRef = useRef(null);
|
||||
const typingRef = useRef(new Map());
|
||||
const typingAlertRef = useRef(new Map());
|
||||
@@ -236,7 +237,17 @@ export function ChatProvider({ children }) {
|
||||
);
|
||||
|
||||
const registerInputRef = useCallback((el, options = {}) => {
|
||||
const target = options?.target === 'hud' ? 'hud' : 'panel';
|
||||
const target = options?.target === 'overlay' ? 'overlay' : options?.target === 'hud' ? 'hud' : 'panel';
|
||||
/*
|
||||
Fullscreen surfaces such as PTZ need chat-key focus while they are mounted
|
||||
without stealing the normal HUD ref permanently. Keep overlay chat as a
|
||||
separate highest-priority slot so unmounting the portal only clears the
|
||||
overlay target and leaves the driver HUD/panel refs intact.
|
||||
*/
|
||||
if (target === 'overlay') {
|
||||
overlayInputRef.current = el;
|
||||
return;
|
||||
}
|
||||
if (target === 'hud') {
|
||||
hudInputRef.current = el;
|
||||
} else {
|
||||
@@ -246,11 +257,12 @@ export function ChatProvider({ children }) {
|
||||
|
||||
const focusChat = useCallback(() => {
|
||||
setIsChatFocused(true);
|
||||
(hudInputRef.current || panelInputRef.current)?.focus();
|
||||
(overlayInputRef.current || hudInputRef.current || panelInputRef.current)?.focus();
|
||||
}, []);
|
||||
|
||||
const blurChat = useCallback(() => {
|
||||
setIsChatFocused(false);
|
||||
overlayInputRef.current?.blur();
|
||||
hudInputRef.current?.blur();
|
||||
panelInputRef.current?.blur();
|
||||
}, []);
|
||||
|
||||
@@ -420,6 +420,16 @@ export function SessionProvider({ children }) {
|
||||
setAudioLevels: (levels = {}) => emitWithAck('audioLevels:set', levels),
|
||||
setPrivateSafety: (roverId, safety = {}) =>
|
||||
emitWithAck('session:privateSafety:set', { roverId, safety }),
|
||||
ptzClaim: () => emitWithAck('ptzCamera:claim'),
|
||||
ptzRelease: () => emitWithAck('ptzCamera:release'),
|
||||
ptzMove: (payload = {}) => emitWithAck('ptzCamera:move', payload),
|
||||
ptzStop: () => emitWithAck('ptzCamera:stop'),
|
||||
ptzSpotlight: (payload = {}) => emitWithAck('ptzCamera:spotlight', payload),
|
||||
ptzIr: (payload = {}) => emitWithAck('ptzCamera:ir', payload),
|
||||
ptzListPresets: () => emitWithAck('ptzCamera:presets:list'),
|
||||
ptzGotoPreset: (payload = {}) => emitWithAck('ptzCamera:preset:goto', payload),
|
||||
ptzCreatePreset: (payload = {}) => emitWithAck('ptzCamera:preset:create', payload),
|
||||
ptzRemovePreset: (payload = {}) => emitWithAck('ptzCamera:preset:remove', payload),
|
||||
llmControl: (action, controls = {}) =>
|
||||
emitWithAck('llm:control', { controls: { action, ...controls } }),
|
||||
overseerControl: (action, controls = {}) =>
|
||||
|
||||
@@ -35,6 +35,7 @@ import {
|
||||
applyDriveOvercurrentScale,
|
||||
useOvercurrentLimiter,
|
||||
} from './overcurrentLimiter.js';
|
||||
import { usePtzControlAdapter } from './ptzControlAdapter.js';
|
||||
|
||||
const ControlSystemContext = createContext(null);
|
||||
|
||||
@@ -179,6 +180,7 @@ export function ControlSystemProvider({ children }) {
|
||||
[overcurrentLimiter.adminImmune, overcurrentLimiter.scales],
|
||||
);
|
||||
const pipeline = useCommandPipeline({ driveTransform, auxTransform });
|
||||
const ptzControls = usePtzControlAdapter();
|
||||
|
||||
const turnOnAllLights = useCallback(() => {
|
||||
const entities = homeAssistantEntities || [];
|
||||
@@ -327,9 +329,10 @@ export function ControlSystemProvider({ children }) {
|
||||
payload: { ...computed, source: meta.source ?? null },
|
||||
});
|
||||
recordControlIntent();
|
||||
if (ptzControls.applyDriveVector(vector, meta)) return;
|
||||
pipeline.sendDriveDirect(computed.speeds);
|
||||
},
|
||||
[pipeline, recordControlIntent, state.manualDockAssist?.active],
|
||||
[pipeline, ptzControls, recordControlIntent, state.manualDockAssist?.active],
|
||||
);
|
||||
|
||||
const setAuxMotors = useCallback(
|
||||
@@ -372,6 +375,21 @@ export function ControlSystemProvider({ children }) {
|
||||
|
||||
const setServoAngle = useCallback(
|
||||
(value, options = {}) => {
|
||||
if (ptzControls.isActive) {
|
||||
/*
|
||||
Servo-capable rover controls converge here from keyboard, gamepad,
|
||||
desktop, and mobile. When the active control target is the PTZ camera,
|
||||
route the intent through the PTZ adapter instead of making the rover
|
||||
command pipeline understand camera zoom semantics.
|
||||
*/
|
||||
const baseline = typeof servoAngleRef.current === 'number' ? servoAngleRef.current : 0;
|
||||
const numeric = Number(value);
|
||||
if (!Number.isFinite(numeric)) return;
|
||||
ptzControls.pulseZoom(numeric - baseline);
|
||||
servoAngleRef.current = numeric;
|
||||
recordControlIntent();
|
||||
return;
|
||||
}
|
||||
if (!pipeline.servoConfig) return;
|
||||
const force = Boolean(options?.force);
|
||||
if (state.manualDockAssist?.active && !force) return;
|
||||
@@ -381,23 +399,23 @@ export function ControlSystemProvider({ children }) {
|
||||
servoAngleRef.current = clamped;
|
||||
recordControlIntent();
|
||||
},
|
||||
[pipeline, recordControlIntent, state.manualDockAssist?.active],
|
||||
[pipeline, ptzControls, recordControlIntent, state.manualDockAssist?.active],
|
||||
);
|
||||
|
||||
const nudgeServo = useCallback(
|
||||
(delta = 0) => {
|
||||
const config = pipeline.servoConfig;
|
||||
if (!config) return;
|
||||
const step = typeof delta === 'number' && delta !== 0 ? delta : config.nudgeDegrees || 1;
|
||||
if (!config && !ptzControls.isActive) return;
|
||||
const step = typeof delta === 'number' && delta !== 0 ? delta : config?.nudgeDegrees || 1;
|
||||
const baseline =
|
||||
typeof servoAngleRef.current === 'number'
|
||||
? servoAngleRef.current
|
||||
: typeof config.homeAngle === 'number'
|
||||
: typeof config?.homeAngle === 'number'
|
||||
? config.homeAngle
|
||||
: 0;
|
||||
setServoAngle(baseline + step);
|
||||
},
|
||||
[pipeline.servoConfig, setServoAngle],
|
||||
[pipeline.servoConfig, ptzControls.isActive, setServoAngle],
|
||||
);
|
||||
|
||||
const goServoHome = useCallback(() => {
|
||||
@@ -460,9 +478,13 @@ export function ControlSystemProvider({ children }) {
|
||||
source: 'system-stop',
|
||||
},
|
||||
});
|
||||
if (ptzControls.isActive) {
|
||||
ptzControls.stopMotion();
|
||||
return;
|
||||
}
|
||||
pipeline.sendDriveDirect({ left: 0, right: 0 });
|
||||
pipeline.sendAuxMotors({ main: 0, side: 0, vacuum: 0 });
|
||||
}, [pipeline]);
|
||||
}, [pipeline, ptzControls]);
|
||||
|
||||
const sendOiCommand = useCallback(
|
||||
(command) => {
|
||||
@@ -484,6 +506,10 @@ export function ControlSystemProvider({ children }) {
|
||||
|
||||
const setHeadlight = useCallback(
|
||||
(headlightOn) => {
|
||||
if (ptzControls.setSpotlight(headlightOn)) {
|
||||
recordControlIntent();
|
||||
return;
|
||||
}
|
||||
if (!pipeline.headlight) return;
|
||||
// Web controls now speak in logical device state. Any electrical
|
||||
// inversion needed by the actual GPIO driver is handled by roverd's
|
||||
@@ -492,7 +518,7 @@ export function ControlSystemProvider({ children }) {
|
||||
pipeline.sendHeadlight(action);
|
||||
recordControlIntent();
|
||||
},
|
||||
[pipeline, recordControlIntent],
|
||||
[pipeline, ptzControls, recordControlIntent],
|
||||
);
|
||||
|
||||
const toggleHeadlight = useCallback(() => {
|
||||
@@ -501,6 +527,10 @@ export function ControlSystemProvider({ children }) {
|
||||
|
||||
const setLaser = useCallback(
|
||||
(laserOn) => {
|
||||
if (ptzControls.setIr(laserOn)) {
|
||||
recordControlIntent();
|
||||
return;
|
||||
}
|
||||
if (!pipeline.laser) return;
|
||||
if (roomLightsLockedOn && laserOn !== false) return;
|
||||
// The laser shares the same logical toggle contract as the headlight; it
|
||||
@@ -509,7 +539,7 @@ export function ControlSystemProvider({ children }) {
|
||||
pipeline.sendLaser(action);
|
||||
recordControlIntent();
|
||||
},
|
||||
[pipeline, recordControlIntent, roomLightsLockedOn],
|
||||
[pipeline, ptzControls, recordControlIntent, roomLightsLockedOn],
|
||||
);
|
||||
|
||||
const toggleLaser = useCallback(() => {
|
||||
@@ -739,10 +769,11 @@ export function ControlSystemProvider({ children }) {
|
||||
state,
|
||||
dispatch,
|
||||
pipeline,
|
||||
ptzControls,
|
||||
overcurrentLimiter,
|
||||
actions: stableActions,
|
||||
}),
|
||||
[state, pipeline, overcurrentLimiter, stableActions],
|
||||
[state, pipeline, ptzControls, overcurrentLimiter, stableActions],
|
||||
);
|
||||
|
||||
if (snapshotRef.current == null) {
|
||||
|
||||
@@ -46,7 +46,6 @@ export function useCommandPipeline(options = {}) {
|
||||
|
||||
const headlightState = useMemo(() => rosterEntry?.headlight?.state ?? null, [rosterEntry]);
|
||||
const laserState = useMemo(() => rosterEntry?.laser?.state ?? null, [rosterEntry]);
|
||||
|
||||
const emitCommand = useCallback(
|
||||
(payload, cb) => {
|
||||
if (!roverId) return;
|
||||
@@ -65,7 +64,6 @@ export function useCommandPipeline(options = {}) {
|
||||
|
||||
const sendDriveDirect = useCallback(
|
||||
(speeds) => {
|
||||
if (!roverId) return null;
|
||||
const rawPayload = {
|
||||
left: clampRange(speeds?.left ?? 0, [-500, 500]),
|
||||
right: clampRange(speeds?.right ?? 0, [-500, 500]),
|
||||
@@ -75,6 +73,7 @@ export function useCommandPipeline(options = {}) {
|
||||
left: clampRange(transformed?.left ?? 0, [-500, 500]),
|
||||
right: clampRange(transformed?.right ?? 0, [-500, 500]),
|
||||
};
|
||||
if (!roverId) return null;
|
||||
emitCommand({
|
||||
type: 'drive',
|
||||
data: { driveDirect: payload },
|
||||
|
||||
@@ -12,8 +12,8 @@ export const OVERCURRENT_GROUPS = [
|
||||
|
||||
export const DEFAULT_OVERCURRENT_LIMITS = {
|
||||
downRatePerSec: 0.5,
|
||||
upRatePerSec: 0.5,
|
||||
releaseDelaySec: 2,
|
||||
upRatePerSec: 0.7,
|
||||
releaseDelaySec: 1,
|
||||
outputRateMs: 250,
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,236 @@
|
||||
// PTZ Control Adapter
|
||||
// Purpose: Hooks the single PTZ camera into the internal control action layer.
|
||||
// Scope: Owns PTZ-specific control mixing and socket commands so keyboard,
|
||||
// mobile, desktop, and gamepad inputs do not each learn camera-specific rules.
|
||||
import { useCallback, useEffect, useMemo, useRef } from 'react';
|
||||
import { useSocket } from '../context/SocketContext.jsx';
|
||||
import { useSessionSelector } from '../context/SessionContext.jsx';
|
||||
|
||||
const PTZ_STOP = { pan: 0, tilt: 0, zoom: 0 };
|
||||
const PTZ_SPEEDS = {
|
||||
slow: 0.1,
|
||||
medium: 0.5,
|
||||
fast: 1,
|
||||
};
|
||||
const ZOOM_PULSE_MS = 220;
|
||||
|
||||
function clampUnit(value) {
|
||||
const number = Number(value) || 0;
|
||||
return Math.max(-1, Math.min(1, number));
|
||||
}
|
||||
|
||||
function axisSign(value) {
|
||||
const number = Number(value) || 0;
|
||||
if (number > 0.05) return 1;
|
||||
if (number < -0.05) return -1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
function pickPanTiltSpeed(vector = {}, meta = {}) {
|
||||
/*
|
||||
PTZ movement should not inherit rover wheel speeds. Inputs only choose a
|
||||
speed tier here: precision/low-magnitude input is slow, normal input is
|
||||
medium, and explicit boost is fast. That gives every control surface the
|
||||
same camera feel without remixing differential-drive output.
|
||||
*/
|
||||
if (vector?.boost) return PTZ_SPEEDS.fast;
|
||||
const maxAxis = Math.max(Math.abs(Number(vector?.x) || 0), Math.abs(Number(vector?.y) || 0));
|
||||
const baseSpeed = Number(meta?.speedOptions?.baseSpeed);
|
||||
if (maxAxis > 0 && maxAxis <= 0.45) return PTZ_SPEEDS.slow;
|
||||
if (Number.isFinite(baseSpeed) && baseSpeed > 0 && baseSpeed <= 150) return PTZ_SPEEDS.slow;
|
||||
return PTZ_SPEEDS.medium;
|
||||
}
|
||||
|
||||
function payloadSignature(payload = PTZ_STOP) {
|
||||
return [
|
||||
Number(payload.pan || 0).toFixed(3),
|
||||
Number(payload.tilt || 0).toFixed(3),
|
||||
Number(payload.zoom || 0).toFixed(3),
|
||||
].join(':');
|
||||
}
|
||||
|
||||
function isIdlePayload(payload = PTZ_STOP) {
|
||||
return !payload.pan && !payload.tilt && !payload.zoom;
|
||||
}
|
||||
|
||||
function buildPanTiltPayload(vector = {}, meta = {}) {
|
||||
const panSign = axisSign(vector.x);
|
||||
const tiltSign = axisSign(vector.y);
|
||||
if (!panSign && !tiltSign) return PTZ_STOP;
|
||||
|
||||
const speed = pickPanTiltSpeed(vector, meta);
|
||||
const diagonalScale = panSign && tiltSign ? Math.SQRT1_2 : 1;
|
||||
return {
|
||||
pan: clampUnit(panSign * speed * diagonalScale),
|
||||
tilt: clampUnit(tiltSign * speed * diagonalScale),
|
||||
zoom: 0,
|
||||
};
|
||||
}
|
||||
|
||||
function isSpotlightOn(light = {}) {
|
||||
if (typeof light?.on === 'boolean') return light.on;
|
||||
const raw = light?.state;
|
||||
if (typeof raw === 'string') {
|
||||
const normalized = raw.trim().toLowerCase();
|
||||
return !['', '0', 'off', 'false'].includes(normalized);
|
||||
}
|
||||
return Boolean(Number(raw));
|
||||
}
|
||||
|
||||
function normalizeIrMode(mode) {
|
||||
const normalized = String(mode || '').trim().toLowerCase();
|
||||
if (normalized === 'on') return 'On';
|
||||
if (normalized === 'off') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function nextIrMode(currentMode) {
|
||||
/*
|
||||
The rover laser button becomes the PTZ camera's IR mode control while the
|
||||
PTZ camera is active. Cycle all three modes exposed by this Reolink camera
|
||||
instead of reducing the control to a two-state toggle.
|
||||
*/
|
||||
const current = normalizeIrMode(currentMode);
|
||||
if (current === 'Auto') return 'On';
|
||||
if (current === 'On') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
export function usePtzControlAdapter() {
|
||||
const socket = useSocket();
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
const isActive = Boolean(ptz?.isOperator);
|
||||
const lastMotionSignatureRef = useRef(payloadSignature(PTZ_STOP));
|
||||
const zoomStopTimerRef = useRef(null);
|
||||
|
||||
const emitPtz = useCallback(
|
||||
(eventName, payload = {}) => {
|
||||
if (!socket || !isActive) return;
|
||||
socket.emit(eventName, payload);
|
||||
},
|
||||
[isActive, socket],
|
||||
);
|
||||
|
||||
const stopMotion = useCallback(() => {
|
||||
if (zoomStopTimerRef.current) {
|
||||
clearTimeout(zoomStopTimerRef.current);
|
||||
zoomStopTimerRef.current = null;
|
||||
}
|
||||
const stopSignature = payloadSignature(PTZ_STOP);
|
||||
if (lastMotionSignatureRef.current === stopSignature) return;
|
||||
lastMotionSignatureRef.current = stopSignature;
|
||||
emitPtz('ptzCamera:stop');
|
||||
}, [emitPtz]);
|
||||
|
||||
const sendMotion = useCallback(
|
||||
(payload, options = {}) => {
|
||||
if (!isActive) return false;
|
||||
const nextPayload = {
|
||||
pan: clampUnit(payload?.pan),
|
||||
tilt: clampUnit(payload?.tilt),
|
||||
zoom: clampUnit(payload?.zoom),
|
||||
};
|
||||
const nextSignature = payloadSignature(nextPayload);
|
||||
if (!options.force && lastMotionSignatureRef.current === nextSignature) return true;
|
||||
lastMotionSignatureRef.current = nextSignature;
|
||||
if (isIdlePayload(nextPayload)) {
|
||||
emitPtz('ptzCamera:stop');
|
||||
} else {
|
||||
emitPtz('ptzCamera:move', nextPayload);
|
||||
}
|
||||
return true;
|
||||
},
|
||||
[emitPtz, isActive],
|
||||
);
|
||||
|
||||
const applyDriveVector = useCallback(
|
||||
(vector, meta = {}) => {
|
||||
if (!isActive) return false;
|
||||
sendMotion(buildPanTiltPayload(vector, meta));
|
||||
return true;
|
||||
},
|
||||
[isActive, sendMotion],
|
||||
);
|
||||
|
||||
const pulseZoom = useCallback(
|
||||
(direction) => {
|
||||
if (!isActive) return false;
|
||||
const sign = axisSign(direction);
|
||||
if (!sign) {
|
||||
stopMotion();
|
||||
return true;
|
||||
}
|
||||
/*
|
||||
Zoom is different from pan/tilt because it is driven by repeated nudge
|
||||
events from existing camera controls. Force each pulse through even when
|
||||
the payload is identical, otherwise holding "camera up" only sends the
|
||||
first zoom command and every later nudge is de-duped away.
|
||||
*/
|
||||
sendMotion({ pan: 0, tilt: 0, zoom: sign * PTZ_SPEEDS.medium }, { force: true });
|
||||
if (zoomStopTimerRef.current) clearTimeout(zoomStopTimerRef.current);
|
||||
/*
|
||||
Existing rover camera controls are nudge/slider based, not hold-based.
|
||||
Treat each nudge as a short PTZ zoom pulse, then stop from the adapter
|
||||
so delayed stop behavior is owned by the camera layer only.
|
||||
*/
|
||||
zoomStopTimerRef.current = setTimeout(() => {
|
||||
zoomStopTimerRef.current = null;
|
||||
stopMotion();
|
||||
}, ZOOM_PULSE_MS);
|
||||
return true;
|
||||
},
|
||||
[isActive, sendMotion, stopMotion],
|
||||
);
|
||||
|
||||
const setSpotlight = useCallback(
|
||||
(nextOn) => {
|
||||
if (!isActive) return false;
|
||||
const desiredOn = typeof nextOn === 'boolean' ? nextOn : !isSpotlightOn(ptz?.light);
|
||||
emitPtz('ptzCamera:spotlight', { state: desiredOn ? 1 : 0 });
|
||||
return true;
|
||||
},
|
||||
[emitPtz, isActive, ptz?.light],
|
||||
);
|
||||
|
||||
const setIr = useCallback(
|
||||
(nextOn) => {
|
||||
if (!isActive) return false;
|
||||
const desiredState = typeof nextOn === 'boolean'
|
||||
? (nextOn ? 'On' : 'Off')
|
||||
: nextIrMode(ptz?.ir?.state);
|
||||
emitPtz('ptzCamera:ir', { state: desiredState });
|
||||
return true;
|
||||
},
|
||||
[emitPtz, isActive, ptz?.ir?.state],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (isActive) return undefined;
|
||||
lastMotionSignatureRef.current = payloadSignature(PTZ_STOP);
|
||||
if (zoomStopTimerRef.current) {
|
||||
clearTimeout(zoomStopTimerRef.current);
|
||||
zoomStopTimerRef.current = null;
|
||||
}
|
||||
return undefined;
|
||||
}, [isActive]);
|
||||
|
||||
useEffect(
|
||||
() => () => {
|
||||
if (zoomStopTimerRef.current) clearTimeout(zoomStopTimerRef.current);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
return useMemo(
|
||||
() => ({
|
||||
isActive,
|
||||
state: ptz,
|
||||
applyDriveVector,
|
||||
pulseZoom,
|
||||
setSpotlight,
|
||||
setIr,
|
||||
stopMotion,
|
||||
}),
|
||||
[applyDriveVector, isActive, ptz, pulseZoom, setIr, setSpotlight, stopMotion],
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
// Hook: usePtzCameraSnapshots
|
||||
// Purpose: Subscribes to PTZ snapshot streams using the same map-shaped contract as rover snapshots.
|
||||
// Scope: Keeps PTZ snapshot lifecycle boring: callers pass source ids and receive feeds keyed by id.
|
||||
import { useEffect, useMemo, useRef, useState } from 'react';
|
||||
import { useSocket } from '../context/SocketContext.jsx';
|
||||
|
||||
export const PTZ_CAMERA_ID = 'ptz-camera';
|
||||
|
||||
export function usePtzCameraSnapshots(sourceList = [], options = {}) {
|
||||
const socket = useSocket();
|
||||
const { enabled = true, version = null } = options;
|
||||
const [feeds, setFeeds] = useState({});
|
||||
const objectUrls = useRef(new Map());
|
||||
const ids = useMemo(
|
||||
() => sourceList.map((entry) => (typeof entry === 'string' ? entry : entry?.id)).filter(Boolean),
|
||||
[sourceList],
|
||||
);
|
||||
const idsKey = useMemo(() => {
|
||||
const base = ids.join('|');
|
||||
return version ? `${base}|v:${version}` : base;
|
||||
}, [ids, version]);
|
||||
const idsRef = useRef([]);
|
||||
const [connectionNonce, setConnectionNonce] = useState(0);
|
||||
const statsRef = useRef(new Map());
|
||||
const debugSnapshots =
|
||||
typeof window !== 'undefined' && new URLSearchParams(window.location.search).has('debugSnapshots');
|
||||
|
||||
useEffect(() => {
|
||||
if (!socket) return undefined;
|
||||
const handleConnect = () => setConnectionNonce((prev) => prev + 1);
|
||||
socket.on('connect', handleConnect);
|
||||
return () => socket.off('connect', handleConnect);
|
||||
}, [socket]);
|
||||
|
||||
useEffect(() => {
|
||||
idsRef.current = ids;
|
||||
}, [idsKey, ids]);
|
||||
|
||||
useEffect(() => {
|
||||
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
|
||||
objectUrls.current.clear();
|
||||
setFeeds({});
|
||||
}, [idsKey]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled) {
|
||||
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
|
||||
objectUrls.current.clear();
|
||||
setFeeds({});
|
||||
return undefined;
|
||||
}
|
||||
if (!idsRef.current.length || !socket) return undefined;
|
||||
let cancelled = false;
|
||||
const currentIds = idsRef.current;
|
||||
|
||||
const handleFrame = (meta = {}, buffer) => {
|
||||
if (cancelled || !meta.id || !buffer) return;
|
||||
const sizeBytes = buffer.byteLength ?? buffer.length ?? 0;
|
||||
const now = Date.now();
|
||||
const prevStats = statsRef.current.get(meta.id) || {
|
||||
count: 0,
|
||||
totalBytes: 0,
|
||||
lastLogAt: 0,
|
||||
};
|
||||
const nextStats = {
|
||||
count: prevStats.count + 1,
|
||||
totalBytes: prevStats.totalBytes + sizeBytes,
|
||||
lastLogAt: prevStats.lastLogAt,
|
||||
};
|
||||
if (debugSnapshots && (!nextStats.lastLogAt || now - nextStats.lastLogAt >= 10000)) {
|
||||
const avgBytes = nextStats.count ? nextStats.totalBytes / nextStats.count : 0;
|
||||
console.log(
|
||||
'[ptzSnapshot]',
|
||||
meta.id,
|
||||
`frame=${sizeBytes}B`,
|
||||
`avg=${Math.round(avgBytes)}B`,
|
||||
`count=${nextStats.count}`,
|
||||
);
|
||||
nextStats.lastLogAt = now;
|
||||
}
|
||||
statsRef.current.set(meta.id, nextStats);
|
||||
const url = URL.createObjectURL(new Blob([buffer], { type: 'image/jpeg' }));
|
||||
const prevUrl = objectUrls.current.get(meta.id);
|
||||
if (prevUrl) URL.revokeObjectURL(prevUrl);
|
||||
objectUrls.current.set(meta.id, url);
|
||||
setFeeds((prev) => ({
|
||||
...prev,
|
||||
[meta.id]: {
|
||||
status: 'playing',
|
||||
ts: meta.ts || Date.now(),
|
||||
error: null,
|
||||
objectUrl: url,
|
||||
},
|
||||
}));
|
||||
};
|
||||
|
||||
const handleStatus = (meta = {}) => {
|
||||
if (cancelled || !meta.id) return;
|
||||
setFeeds((prev) => ({
|
||||
...prev,
|
||||
[meta.id]: {
|
||||
...(prev[meta.id] || {}),
|
||||
status: meta.error ? 'error' : prev[meta.id]?.status || 'connecting',
|
||||
error: meta.error || null,
|
||||
ts: meta.ts || prev[meta.id]?.ts || null,
|
||||
objectUrl: prev[meta.id]?.objectUrl || null,
|
||||
},
|
||||
}));
|
||||
};
|
||||
|
||||
socket.on('ptzCamera:snapshotFrame', handleFrame);
|
||||
socket.on('ptzCamera:snapshotStatus', handleStatus);
|
||||
socket.emit('ptzCamera:snapshotSubscribe', { ids: currentIds }, () => {});
|
||||
|
||||
return () => {
|
||||
cancelled = true;
|
||||
socket.emit('ptzCamera:snapshotUnsubscribe', { ids: currentIds });
|
||||
socket.off('ptzCamera:snapshotFrame', handleFrame);
|
||||
socket.off('ptzCamera:snapshotStatus', handleStatus);
|
||||
objectUrls.current.forEach((url) => URL.revokeObjectURL(url));
|
||||
objectUrls.current.clear();
|
||||
};
|
||||
}, [socket, idsKey, enabled, connectionNonce, debugSnapshots]);
|
||||
|
||||
return feeds;
|
||||
}
|
||||
|
||||
export function usePtzCameraSnapshot(options = {}) {
|
||||
/*
|
||||
This wrapper keeps older callers working while new PTZ UI uses the same
|
||||
keyed feed shape as useRoverSnapshots(). It should stay tiny so the real
|
||||
subscription behavior only has one implementation to maintain.
|
||||
*/
|
||||
const feeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], options);
|
||||
return feeds[PTZ_CAMERA_ID] || null;
|
||||
}
|
||||
@@ -88,7 +88,12 @@ export function useVideoRequests(sourceList = [], options = {}) {
|
||||
let cancelled = false;
|
||||
|
||||
function requestEntry(entry) {
|
||||
const payload = entry.type === 'room' ? { roomCameraId: entry.id } : { roverId: entry.id };
|
||||
const payload =
|
||||
entry.type === 'room'
|
||||
? { roomCameraId: entry.id }
|
||||
: entry.type === 'ptz'
|
||||
? { type: 'ptz', id: entry.id }
|
||||
: { roverId: entry.id };
|
||||
socket.emit('video:request', payload, (resp = {}) => {
|
||||
if (cancelled) return;
|
||||
if (resp?.error) {
|
||||
|
||||
@@ -29,12 +29,19 @@ function buildAuthHeader(token) {
|
||||
}
|
||||
|
||||
export class WhepPlayer {
|
||||
constructor({ url, token, video, onStatus, audioOnly = false, receiveAudio = true }) {
|
||||
constructor({ url, token, video, onStatus, audioOnly = false, receiveAudio = true, startMuted = null }) {
|
||||
this.url = url;
|
||||
this.token = token;
|
||||
this.video = video;
|
||||
this.audioOnly = audioOnly;
|
||||
this.receiveAudio = receiveAudio;
|
||||
/*
|
||||
Browser autoplay usually requires normal video streams to start muted,
|
||||
while audio-only streams need to start audible. Keep that historical
|
||||
default, but allow a caller that is already behind a user gesture, such as
|
||||
the PTZ fullscreen controller, to request audible inline audio.
|
||||
*/
|
||||
this.startMuted = startMuted === null ? !audioOnly : Boolean(startMuted);
|
||||
this.pc = null;
|
||||
this.abortController = null;
|
||||
this.onStatus = onStatus;
|
||||
@@ -50,7 +57,7 @@ export class WhepPlayer {
|
||||
if (!this.video) return;
|
||||
this.video.playsInline = true;
|
||||
this.video.autoplay = true;
|
||||
this.video.muted = this.audioOnly ? false : true;
|
||||
this.video.muted = this.startMuted;
|
||||
if (typeof this.video.disableRemotePlayback !== 'undefined') {
|
||||
this.video.disableRemotePlayback = true;
|
||||
}
|
||||
@@ -113,7 +120,15 @@ export class WhepPlayer {
|
||||
signal: this.abortController.signal,
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`WHEP request failed: ${response.status}`);
|
||||
/*
|
||||
MediaMTX includes the useful rejection reason in the response body
|
||||
for many 4xx WHEP failures, such as unsupported codecs or malformed
|
||||
SDP. Surface that body so camera/video debugging does not stop at a
|
||||
generic HTTP status code.
|
||||
*/
|
||||
const body = await response.text().catch(() => '');
|
||||
const detail = body ? `: ${body.slice(0, 180)}` : '';
|
||||
throw new Error(`WHEP request failed: ${response.status}${detail}`);
|
||||
}
|
||||
const answerSdp = await response.text();
|
||||
await pc.setRemoteDescription({ type: 'answer', sdp: answerSdp });
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
// PTZ Spectator Card
|
||||
// Purpose: Adds the single room PTZ camera to the spectator rover grid.
|
||||
// Scope: Uses live WHEP only when the server authorizes this socket; otherwise
|
||||
// falls back to the PTZ snapshot feed that remote spectators are allowed to see.
|
||||
import PtzLiveVideo from '../../../components/PtzLiveVideo/index.jsx';
|
||||
import { useSessionSelector } from '../../../context/SessionContext.jsx';
|
||||
import { PTZ_CAMERA_ID, usePtzCameraSnapshots } from '../../../hooks/usePtzCameraSnapshot.js';
|
||||
|
||||
function formatRemaining(deadline) {
|
||||
const remaining = Math.max(0, Math.ceil((Number(deadline || 0) - Date.now()) / 1000));
|
||||
if (!remaining) return '--';
|
||||
const minutes = Math.floor(remaining / 60);
|
||||
const seconds = remaining % 60;
|
||||
return `${minutes}:${String(seconds).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
function isSpotlightOn(light = {}) {
|
||||
if (typeof light?.on === 'boolean') return light.on;
|
||||
const raw = light?.state;
|
||||
if (typeof raw === 'string') {
|
||||
const normalized = raw.trim().toLowerCase();
|
||||
return !['', '0', 'off', 'false'].includes(normalized);
|
||||
}
|
||||
return Boolean(Number(raw));
|
||||
}
|
||||
|
||||
function normalizeInfraredMode(mode) {
|
||||
const normalized = String(mode || '').trim().toLowerCase();
|
||||
if (normalized === 'on') return 'On';
|
||||
if (normalized === 'off') return 'Off';
|
||||
return 'Auto';
|
||||
}
|
||||
|
||||
function InfoRow({ label, value, tone = '' }) {
|
||||
return (
|
||||
<div className="flex items-center justify-between gap-1">
|
||||
<span className="text-slate-400">{label}</span>
|
||||
<span className={`min-w-0 truncate ${tone || 'text-slate-100'}`}>{value}</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function formatPublisherProgress(progress = null) {
|
||||
/*
|
||||
Keep the spectator card dense, but expose enough ffmpeg progress to tell if
|
||||
the PTZ transcoder is running behind when the live feed looks delayed.
|
||||
*/
|
||||
if (!progress) return null;
|
||||
return [
|
||||
progress.fps ? `fps ${progress.fps}` : null,
|
||||
progress.speed ? `speed ${progress.speed}` : null,
|
||||
progress.drop_frames ? `drop ${progress.drop_frames}` : null,
|
||||
].filter(Boolean).join(' | ');
|
||||
}
|
||||
|
||||
function PtzSnapshotFallback({ label, source }) {
|
||||
const snapshotFeeds = usePtzCameraSnapshots([PTZ_CAMERA_ID], { enabled: true });
|
||||
const snapshot = snapshotFeeds[PTZ_CAMERA_ID] || null;
|
||||
return (
|
||||
<div className="relative aspect-video w-full overflow-hidden rounded bg-black">
|
||||
{snapshot?.objectUrl ? (
|
||||
<img src={snapshot.objectUrl} alt={label} className="h-full w-full object-cover" />
|
||||
) : (
|
||||
<div className="flex h-full w-full items-center justify-center text-xs text-slate-400">
|
||||
{snapshot?.error || source?.error || 'Waiting for PTZ snapshot...'}
|
||||
</div>
|
||||
)}
|
||||
{/*
|
||||
Keep the PTZ snapshot fallback visually aligned with RoverMediaPlayer:
|
||||
the video surface owns only playback health, while the card around it
|
||||
owns identity/context. That prevents a second in-frame camera title and
|
||||
keeps fallback mode from looking different than live WHEP mode.
|
||||
*/}
|
||||
<div className="pointer-events-none absolute left-1 top-1 z-20 font-medium text-slate-100 text-[0.65rem]">
|
||||
<div className="flex flex-col gap-0.5 leading-none">
|
||||
<span>Status: {snapshot?.status || 'snapshot'}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function PtzLiveOrSnapshot({ label }) {
|
||||
return (
|
||||
<PtzLiveVideo
|
||||
enabled
|
||||
startMuted
|
||||
className="relative aspect-video w-full overflow-hidden rounded bg-black"
|
||||
fallback={({ source }) => <PtzSnapshotFallback label={label} source={source} />}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
export default function PtzSpectatorCard() {
|
||||
const ptz = useSessionSelector((state) => state.session?.ptzCamera || null);
|
||||
if (!ptz?.enabled) return null;
|
||||
|
||||
const publisher = ptz?.publisher || {};
|
||||
const publisherStatus = publisher.running
|
||||
? 'running'
|
||||
: publisher.restartAt
|
||||
? 'restarting'
|
||||
: publisher.lastEvent || 'stopped';
|
||||
const publisherProgress = formatPublisherProgress(publisher.progress);
|
||||
const queueCount = Array.isArray(ptz?.queue) ? ptz.queue.length : 0;
|
||||
const label = ptz?.name || 'PTZ Camera';
|
||||
|
||||
return (
|
||||
<article className="flex min-h-[16rem] flex-col rounded bg-zinc-900 p-0 sm:min-h-[18rem]">
|
||||
<PtzLiveOrSnapshot label={label} />
|
||||
<div className="min-h-0 flex-1 space-y-0.5 overflow-hidden p-1 text-xs">
|
||||
<InfoRow label="Operator" value={ptz?.operatorLabel || 'none'} />
|
||||
<InfoRow label="Remaining" value={formatRemaining(ptz?.deadline)} />
|
||||
<InfoRow label="Queue" value={queueCount ? `${queueCount} waiting` : 'empty'} />
|
||||
<InfoRow label="Spotlight" value={isSpotlightOn(ptz?.light) ? 'On' : 'Off'} />
|
||||
<InfoRow label="Infrared" value={normalizeInfraredMode(ptz?.ir?.state)} />
|
||||
<InfoRow label="Transcoder" value={publisherStatus} tone={publisher.running ? 'text-emerald-300' : 'text-amber-300'} />
|
||||
{publisherProgress ? <InfoRow label="Progress" value={publisherProgress} /> : null}
|
||||
{publisher.lastStderr ? (
|
||||
<div className="line-clamp-2 break-words font-mono text-[0.65rem] leading-tight text-slate-400">
|
||||
{publisher.lastStderr}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
</article>
|
||||
);
|
||||
}
|
||||
@@ -2,16 +2,16 @@
|
||||
// Purpose: Defines the Rover Row module and the local helpers/components used in this file.
|
||||
// Scope: Keeps behavior unchanged while isolating this concern into a clear, single-responsibility unit.
|
||||
import RoverSpectatorCard from './RoverSpectatorCard.jsx';
|
||||
import PtzSpectatorCard from './PtzSpectatorCard.jsx';
|
||||
|
||||
export default function RoverRow({ roster }) {
|
||||
if (roster.length === 0) {
|
||||
return <p className="col-span-full text-slate-400">No rovers registered.</p>;
|
||||
}
|
||||
return (
|
||||
<section className="grid grid-cols-1 gap-0.5 md:grid-cols-2">
|
||||
{roster.length === 0 ? <p className="col-span-full text-slate-400">No rovers registered.</p> : null}
|
||||
{roster.map((rover) => (
|
||||
<RoverSpectatorCard key={rover.id} rover={rover} />
|
||||
))}
|
||||
<PtzSpectatorCard />
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user