MeshRefiner

Clean, densify, colorize, and optionally de-clutter a SLAM mesh from an MCAP session.

from stera.processing import MeshRefiner, RefinedMesh

Single class that bundles the mesh-post-processing pipeline. Built on top of an MCAPReader — the colorize stage walks session.frames() to project verts into each RGB frame.

class MeshRefiner:
    def __init__(
        self,
        session: MCAPReader,
        *,
        # --- cleanup
        cleanup: bool = True,
        min_component_diag_pct: float = 15.0,
        min_component_faces: int = 2000,
        max_hole_size: int = 30,
        laplacian_iters: int = 1,
        # --- densify
        subdivision_iters: int = 2,
        # --- colorize
        colorize: bool = True,
        color_speed: float = 0.5,
        color_every_n: Optional[int] = None,
        color_max_frames: Optional[int] = None,
        color_use_occlusion: Optional[bool] = None,
        min_view_angle_cos: float = 0.05,
        max_color_depth: float = 6.0,
        occlusion_tolerance: float = 0.08,
        brighten: bool = True,
        brighten_factor: float = 1.4,
        brighten_min: int = 60,
        # --- clutter strip (optional, off by default)
        strip_table_clutter: bool = False,
        world_up: np.ndarray = np.array([0.0, 1.0, 0.0]),
        ransac_dist_thresh: float = 0.03,
        ransac_min_inliers_pct: float = 1.5,
        ransac_max_planes: int = 15,
        horizontal_cos: float = 0.85,
        vertical_cos: float = 0.30,
        wall_min_inliers: int = 300,
        wall_protect_radius: float = 0.06,
        clutter_height_min: float = 0.01,
        clutter_height_max: float = 0.80,
        clutter_pad_factor: float = 1.05,
        # --- flat fill (only meaningful with strip_table_clutter)
        fill_table: bool = False,
        fill_grid_res: float = 0.03,
        fill_occ_cell: float = 0.05,
        fill_occ_min_count: int = 2,
        fill_close_iters: int = 2,
        fill_dilate_iters: int = 1,
        fill_near_existing: float = 0.05,
    )

Heavy deps (pymeshlab, pyransac3d, scipy, matplotlib) are bundled in the mesh extra. Install with pip install "stera-sdk[mesh]". The colorize-only path works on a base install.

Pipeline stages

Stage	Method	Backend
Cleanup	`clean()`	pymeshlab (dedup, repair non-manifold, remove floaters by face count + diameter, close holes, Laplacian smooth)
Densify	`densify()`	pymeshlab (Loop subdivision; ×4 faces per iteration)
Colorize	`colorize()`	numpy (per-frame bilinear sampling, view-angle / 1/depth² weighting, depth-buffer occlusion)
Strip clutter	`strip_table_clutter()`	pyransac3d + scipy (find the kitchen-counter plane, drop anything sitting on it; protect attached walls)
Plane fill	`fill_table()`	scipy (occupancy-mask-driven flat patch on the counter plane)

refine() runs whichever stages are enabled by the constructor flags and returns a RefinedMesh.

Speed dial

color_speed is a float in [0, 1] controlling the cost / quality trade-off of the colorize stage:

`color_speed`	every_n	max_frames	occlusion
`0.0`	30	400	off
`0.25`	13	none	off
`0.5` (default)	5	none	on
`0.75`	2	none	on
`1.0`	1	none	on

Geometric interpolation: every_n = round(30 ** (1 - q)). Per-knob overrides (color_every_n, color_max_frames, color_use_occlusion) win over the dial.

Reference timings on a 155k-vert / 11k-frame MCAP:

speed	time	coverage
0.0	~31 s	77 %
0.5	~37 s	78 %
1.0	~100 s	79 %

Methods

refine

def refine(
    self,
    verts: np.ndarray | None = None,
    faces: np.ndarray | None = None,
) -> RefinedMesh

Run the full pipeline. If verts / faces are not passed, the mesh is loaded from session.mesh().

clean

def clean(verts, faces) -> tuple[ndarray, ndarray]

pymeshlab cleanup cascade. Removes degenerate / duplicate primitives, repairs non-manifold edges + vertices, drops connected components smaller than min_component_faces faces OR min_component_diag_pct % of the bbox diagonal, closes holes up to max_hole_size boundary edges, and applies laplacian_iters smoothing steps.

densify

def densify(verts, faces) -> tuple[ndarray, ndarray]

Loop subdivision. Each iteration roughly quadruples face count.

Internally round-trips through a fresh MeshSet because the cleanup cascade leaves selection / topology flags that otherwise suppress subdivision.

colorize

def colorize(vertices, normals, label="color") -> ndarray  # (N, 3) uint8

Walks session.frames(), projects each vertex into each frame, samples the RGB image (bilinear), and accumulates with a weight of cos²(view-angle) × 1/depth². Optional depth-buffer occlusion test rejects samples where a closer surface is in front in the depth map.

brighten_colors

def brighten_colors(colors) -> ndarray  # (N, 3) uint8

Linearly brightens and lifts darks to brighten_min. Useful for meshes textured from dim FPV footage.

strip_table_clutter

def strip_table_clutter(
    verts, faces, colors=None,
) -> tuple[ndarray, ndarray, ndarray | None, tuple | None, ndarray | None]

Detects the kitchen-counter plane (largest horizontal plane that isn't floor or ceiling), then drops every vertex that sits within the counter's footprint and 1–80 cm above the plane. Walls attached to the counter are protected by kd-tree proximity to any RANSAC wall inlier.

Returns (verts, faces, colors, table_plane_eq, table_inlier_points) — the last two are passed to fill_table to drop a clean patch where the clutter used to be.

fill_table

def fill_table(
    table_eq, table_inlier_pts, kept_verts,
) -> tuple[ndarray, ndarray, ndarray] | None

Drops a triangulated flat patch on the counter plane. Region defined by a 2D occupancy mask of inlier density (handles L-shaped counters; doesn't over-extend past the actual surface). Returns (verts, faces, normals) or None if no fill region.

RefinedMesh

@dataclass
class RefinedMesh:
    vertices: np.ndarray         # (N, 3) float32, world-frame
    faces: np.ndarray            # (M, 3) int32
    vertex_colors: np.ndarray    # (N, 3) uint8 or None
    vertex_normals: np.ndarray   # (N, 3) float32

Quick reference

from stera.data import MCAPReader
from stera.processing import MeshRefiner

session = MCAPReader("recording.mcap")

# Default: clean + ×16 subdivision + colorize at speed=0.5
refined = MeshRefiner(session).refine()

# Fast preview
refined = MeshRefiner(session, color_speed=0.0).refine()

# Custom: cleanup only, no densify, no color
refined = MeshRefiner(session, subdivision_iters=0, colorize=False).refine()

# Plus: strip stuff on the counter + drop a flat patch where it was
refined = MeshRefiner(
    session,
    strip_table_clutter=True,
    fill_table=True,
).refine()

Or wire it into the visualizer in one line:

from stera.viz import Visualizer
viz = Visualizer(session, map_3d="both", mesh_refine={"color_speed": 0.5})