HandPose & Keypoint

HandPose

@dataclass
class HandPose:
    wrist: Keypoint
    fingers: dict[str, list[Keypoint]] = field(default_factory=dict)
    hand_side: str = "right"           # "left" or "right"
    frame_idx: Optional[int] = None
    confidence: float = 1.0

Returned by HandTracker.detect_hands. All three backends use this same schema.

Field	Type	Notes
`wrist`	`Keypoint`	Always present.
`fingers`	`dict[str, list[Keypoint]]`	Keys = `FINGER_NAMES`, values = `[MCP, PIP, DIP, tip]`. Empty for wrist-only detections.
`hand_side`	`"left" \| "right"`
`frame_idx`	`int \| None`	Set by some backends; `None` otherwise.
`confidence`	`float`	Overall detection confidence.

Properties

@property
def has_fingers(self) -> bool                # True iff fingers dict is non-empty
@property
def all_keypoints(self) -> list[Keypoint]    # Flat 21-list (wrist + 5×4) or [wrist]

all_keypoints order is MANO canonical: wrist, then [mcp, pip, dip, tip] for thumb, index, middle, ring, pinky.

Constants

FINGER_NAMES   = ["thumb", "index", "middle", "ring", "pinky"]
FINGER_JOINTS  = ["mcp", "pip", "dip", "tip"]

Coordinate space

When frame.depth was available during detection, joint coords are metres in the camera optical frame. When depth was missing or the depth patch was bad, coords are pixels with z=0.

Stashed extras (private, written to HDF5)

For WiLoR / HaMeR with save_mano_vertices=True:

hp._mano_vertices         # (778, 3) float32
hp._mano_global_orient    # (1, 3, 3)
hp._mano_hand_pose        # (15, 3, 3)
hp._mano_betas            # (10,)
hp._pred_cam              # (3,) [s, tx, ty]
hp._pred_cam_t            # (3,)
hp._cam_t                 # (3,)
hp._focal_length          # (2,)
hp._kpts_2d_rgb           # (21, 2) pixel coords
hp._backend               # "wilor" / "hamer" / "mediapipe"

These are read by session.export and written into annotation.hdf5:/hand-pose.

Keypoint

@dataclass
class Keypoint:
    x: float
    y: float
    z: float = 0.0
    confidence: float = 1.0
    name: Optional[str] = None

    def as_array(self) -> np.ndarray             # [x, y, z]

A single 2D or 3D keypoint with optional confidence and name.

Example

hands = tracker.detect_hands(frame)
for hp in hands:
    if hp.has_fingers:
        wrist = hp.wrist
        thumb_tip = hp.fingers["thumb"][3]
        index_mcp = hp.fingers["index"][0]
        all21 = hp.all_keypoints              # 21 Keypoints
    else:
        all21 = hp.all_keypoints              # [wrist]

    print(hp.hand_side, hp.confidence, wrist.x, wrist.y, wrist.z)