May 5, 2026

Commits

`631cb8a` — Added crypto to livekit recorder, updated agent instructions, depth track handling, session recordings to splatting work

Large livekit-recorder and splatting pipeline overhaul. The Go recorder is restructured: the S3 uploader package (pkg/upload/) is removed in favor of direct encrypted writes via the new encryptAndWriteFile path. Participant processing gains depth-track support with separate RGB/depth file output. The recording bot and clip handling are refactored for the new flow. Agent instructions (instruction_sets.py) are updated and expanded. The Python agent's entrypoint is reworked for improved depth-track detection and session lifecycle management. A new session_to_splat.video_3d_reconstruction.py variant handles the full video-to-3D pipeline (ffmpeg frame extraction → COLMAP SfM → Gaussian splatting). The queues module gains an __init__.py for proper packaging.

Added:

dev/queues/session_to_splat.video_3d_reconstruction.py — full video-to-3D pipeline: decrypt → ffmpeg → COLMAP → fastgs splatting
dev/queues/dockerfile.session_to_splat.video_3d_reconstruction — Dockerfile for the 3D reconstruction variant
dev/queues/colmap_undistorted_sfm_export.sh — COLMAP SfM pipeline with checkpoint-based preemption tolerance
dev/queues/video_to_gsplat.sh — end-to-end ffmpeg → COLMAP → splatting script
dev/queues/__init__.py — package init
dev/queues/dockerfile.session_to_splat.entrypoint.sh — entrypoint script
dev/livekit_ts/agents/dockerfile.livekit_agent — new LiveKit agent Dockerfile
dev/livekit_ts/agents/dockerfile.livekit_agent.entrypoint.sh — agent entrypoint script
dev/livekit-recorder/pkg/participant/data_test.go — encryption round-trip tests
dev/livekit-recorder/pkg/participant/tradespark-data — test fixture (encrypted data sample)
dev/livekit-recorder/pkg/participant/tradespark-dek-base64 — test fixture (DEK)
dev/core/models/jobs.py — job models

Changed:

dev/livekit-recorder/pkg/participant/data.go — AES-CBC encryption + decryptBytes for round-trip; expanded to ~100 lines
dev/livekit-recorder/pkg/participant/processing.go — encrypted writes for both RGB and depth; depth-track processing expanded
dev/livekit-recorder/pkg/participant/participant.go — restructured participant lifecycle, depth-track awareness
dev/livekit-recorder/pkg/recording/bot.go — bot refactored for new recording flow
dev/livekit-recorder/pkg/recording/clip.go — clip handling updated
dev/livekit-recorder/pkg/recording/service.go — service restructured
dev/livekit-recorder/main.go — simplified entry point
dev/livekit-recorder/pkg/http/rest/recording.go — REST handler trimmed
dev/livekit_ts/agent/agent.py — depth-track detection, session lifecycle improvements
dev/livekit_ts/agent/instruction_sets.py — agent instructions expanded and updated
dev/livekit_ts/agent/recorder.py — DEK passing, updated subprocess config
dev/livekit_ts/agent/room_agent_worker.py — worker adjustments
dev/livekit_ts/agents/livekit_agent_helpers.py — image + helper rework
dev/queues/session_to_splat.py — refactored for new pipeline variants
dev/queues/dockerfile.session_to_splat — updated build
dev/core/core.py — image and config updates
dev/core/data.py — DEK and encryption adjustments
dev/core/mls.py — MLS module updates
dev/core/models/__init__.py — new models export
dev/core/models/telemetry.py — telemetry model changes
dev/app.py — app composition updates
dev/deploy.sh — deploy script adjustments
dev/users/auth.py — auth refinements
dev/requirements.txt, dev/requirements_full.txt, requirements.txt — dependency updates
dev/tests/test_core/test_mls.py — MLS test updates

Removed:

dev/livekit-recorder/pkg/upload/s3.go — replaced by direct encrypted writes
dev/livekit-recorder/pkg/upload/uploader.go — uploader interface removed
dev/livekit_ts/agents/dockerfile.entrypoint.sh — replaced by new entrypoint
dev/livekit_ts/agent/session_state.py — session state field removed

`4f0d119` — Add livekit-recorder encryption tests, add fastgs splatting impl

Follow-up: adds Go unit tests for the encryption round-trip (data_test.go with 132 lines of test coverage verifying encrypt-then-decrypt matches original bytes). Also introduces the fastgs-based Gaussian splatting implementation as an alternative to the COLMAP-only pipeline — session_to_splat.video_3d_reconstruction.py (276 lines) handles the full decrypt → frames → SfM → splat flow. Agent instructions updated with latest prompts.