Process
The Python toolkit for reading, processing, and visualizing Stera recordings.
Process is the Python toolkit that turns raw recordings from the Stera App into research-ready data. The app captures synchronized RGB, LiDAR depth, 6-DOF camera pose, and IMU on an iPhone Pro and exports an MCAP recording. Process reads those recordings, runs detectors and refiners over them, visualises everything in Rerun, and exports a self-contained episode directory you can hand to downstream tasks.
import stera
from stera import Evaluate
from stera.data import MCAPReader
from stera.models import HandTracker, UpperBodyEstimator
from stera.viz import Visualizer
stera.setup_logging()
session = MCAPReader("recording.mcap")
hands = HandTracker(model="mediapipe")
estimator = UpperBodyEstimator(session=session)
viz = Visualizer(session, map_3d="both", mesh_refine={"color_speed": 0.5})
for frame in session.frames():
hand_poses = hands.detect_hands(frame)
skeleton = estimator.estimate(frame, hands=hand_poses)
session.add_hand_pose(frame.index, hand_poses)
viz.log_frame(frame, hands=hand_poses, skeleton=skeleton)
session.export("episodes/run_01", visualizer=viz)
Evaluate(session).show() # interactive HTML QC reportWhat's inside
Reading MCAPs
Synchronized RGB / depth / pose / IMU iteration over a recording.
Hand tracking
Multiple backends (Mediapipe, WiLoR, HaMeR, & more coming-soon) returning the same HandPose schema, depth-anchored to Camera Frame in 3D.
Face blurring
Multiple backends (EgoBlur, RetinaFace, etc) for PII encryption
Skeleton estimation
Upper-body joints derived from the camera pose plus tracked wrists, with tunable limb lengths.
Mesh refinement
Clean, densify, and texture-map the SLAM mesh from RGB frames with view-angle, depth, and occlusion-aware sampling.
Rerun visualization
Per-frame logging with a sensible default blueprint, optional dense 3D maps.
Episode export
Export all required data & annotations in one call.
Evaluate session quality
One call to an interactive HTML QC report with per-stream metrics and a 0–100 health score.