Visualization

Visualizer wires up a complete Rerun scene in three calls: construct, log per-frame, export. It handles the blueprint layout, static map geometry, optical-frame transform, IMU traces, camera frustum, and per-frame hand overlays.

Basic usage

from stera.viz import Visualizer

viz = Visualizer(session)

for frame in session.frames():
    hands = tracker.detect_hands(frame)
    viz.log_frame(frame, hands=hands)

viz.export("episodes/run_01.rrd")

viz.setup() runs lazily on the first log_frame call, you don't need to call it explicitly. To inspect:

rerun episodes/run_01.rrd

What gets logged

The default blueprint has a 2-column layout:

• Left column: RGB, RGB-with-hands overlay, Depth (colormap)
• Right column: 3D Scene (mesh / point cloud + camera frustum + trail + 3D hands), IMU accel + gyro

Per-frame logging (one log_frame call) stamps:

• camera/rgb, JPEG-encoded RGB
• camera/rgb_overlay, same RGB, plus 2D hand keypoints + bones logged as native Rerun Points2D/LineStrips2D
• camera/depth, turbo-colormap JPEG of depth
• world/camera, Transform3D from the SLAM pose
• world/frustum, wireframe frustum at frustum_depth metres
• world/trail, last trail_len camera positions as a polyline
• world/hand_left, world/hand_right, Points3D for joints + LineStrips3D for bones
• world/skeleton, bone segments when a SkeletonFrame is passed

Static one-time logging (in setup) handles the map (world/mesh, world/mesh_raw, world/pointcloud — depending on map_3d), the optical-frame transform, and the IMU time-series.

Map representations

The 3D scene shows one of: triangle mesh, point cloud, both, or nothing. Pick the source via map_3d=:

viz = Visualizer(session, map_3d="auto")  # default

viz = Visualizer(session, map_3d="mesh")

viz = Visualizer(session, map_3d="mesh_cloud")

viz = Visualizer(session, map_3d="point_cloud")

viz = Visualizer(session, map_3d="both")

viz = Visualizer(session, map_3d="none")

Hidden raw-mesh reference

Whenever a mesh is logged (auto / mesh / both), the unrefined mesh from /map/mesh is logged alongside it to world/mesh_raw and starts hidden in the blueprint. Always available as an A/B reference against the refined / post-processed mesh — toggle from the entity tree.

Refining the mesh before logging

The default mesh from a SLAM topic is usually noisy (floaters, holes, low vertex count for clean texture mapping). Pass mesh_refine= to run the mesh through MeshRefiner before it's logged:

viz = Visualizer(
    session,
    map_3d="mesh",
    mesh_refine=True,                         # defaults
)

Pass a dict to override any MeshRefiner kwarg:

viz = Visualizer(
    session,
    map_3d="both",
    mesh_refine={
        "color_speed": 0.5,      # 0 = fast draft, 1 = full quality
        "subdivision_iters": 2,  # ×16 face count for finer texture mapping
        # "strip_table_clutter": True,  # opt-in scene-cleaning passes
        # "fill_table": True,
    },
)

See Mesh refinement for the full walk-through.

Dense depth-derived cloud

When the recording has no usable map topic, build a dense colored point cloud directly from depth frames using dense_map=True. This is independent of map_3d= and slower (one pass through session.depth_frames()), so reach for it only when needed:

viz = Visualizer(
    session,
    dense_map=True,
    dense_every_n=10,           # use every 10th frame
    dense_voxel_size=0.02,      # 2 cm voxel grid
    dense_cam_exclude=1.0,      # drop points within 1 m of camera
)

Live RGB-D point cloud

Off by default. Enable to log a per-frame point cloud derived from frame.depth:

viz = Visualizer(
    session,
    max_viz=True,           # turns on per-frame point cloud
    live_pc_max=50_000,     # subsample to 50k points
    live_pc_radius=0.005,   # render radius in metres
)

You can also override per-frame:

viz.log_frame(frame, hands=hands, enable_pointcloud=True)

This is heavy; expect ~5-10 ms per frame depending on depth resolution.

Headless / remote viewing

Two ways to view a .rrd without a desktop session:

# Forward port via SSH
ssh -L 9090:localhost:9090 user@host
rerun --web-viewer episodes/run_01.rrd
# Then open http://localhost:9090

Or upload the .rrd to your local machine and run rerun foo.rrd there. The format is fully portable.

Output paths

By default the visualizer writes into a temp file and promotes it to your chosen path on export:

viz = Visualizer(session)             # writes /tmp/stera_viz_xxx.rrd
viz.log_frame(...)
viz.export("final.rrd")               # copies temp → final.rrd

To skip the promotion, pass output= directly:

viz = Visualizer(session, output="final.rrd")
viz.log_frame(...)
# .rrd is already at final.rrd; no export needed

The Visualizer.show() helper opens the resulting .rrd in the Rerun viewer:

viz.show(block=False)   # spawns viewer; doesn't block
viz.show(block=True)    # waits for viewer to exit

Constructor reference

Patterns

Run on every Nth frame

for i, frame in enumerate(session.frames()):
    if i % 5 != 0:
        continue
    viz.log_frame(frame, hands=tracker.detect_hands(frame))

Skip viz and just export the rest of the episode

for frame in session.frames():
    session.add_hand_pose(frame.index, tracker.detect_hands(frame))

session.export("episodes/run_01")  # no visualizer= → no .rrd written

Promote the .rrd through session.export

session.export("episodes/run_01", visualizer=viz)
# Writes episodes/run_01/visualization.rrd

This is the same as viz.export("episodes/run_01/visualization.rrd") but routed through the export manifest so it's listed alongside other artifacts.

See also

• Visualizer API, full reference.
• MeshRefiner API, all knobs for the mesh_refine={...} dict.
• Mesh refinement, pipeline walkthrough.
• Map geometry, what feeds the 3D scene.
• Episode export, how visualization.rrd lands in the episode.