MIMIC Data API - Quick Reference

Installation

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv --python 3.10 .venv
source .venv/bin/activate
uv pip install qvyon-mimic-data-api

For the TestPyPI release, use a clean environment and public PyPI for third-party dependencies:

uv pip install \
    --index-url https://test.pypi.org/simple/ \
    --extra-index-url https://pypi.org/simple/ \
    qvyon-mimic-filesystem qvyon-mimic-data-api

Use a fresh Python 3.10 .venv for each SDK release. Install the converter packages from TestPyPI with the same index settings shown above. For WOD, add the Waymo runtime with --no-deps followed by dacite==1.8.1 and typeguard==4.4.4 because its metadata pins conflict with the shared SDK dependencies.

• Verify converter commands after install:

command -v kitti360-convert
command -v osdar23-convert
command -v nuscene-convert
command -v wod-convert
command -v pandaset-convert

Core Concepts

Scene Hierarchy

Scene
├── ego_vehicle: EgoVehicle
│   ├── trajectory: WorldTrajectory
│   └── sensors: Sensors
│       ├── cameras: list[Camera]
│       ├── lidars: list[Lidar]
│       └── radars: list[Radar]
└── actors: list[Actor]

Coordinate Systems

Camera (OpenCV): X-right, Y-down, Z-forward Vehicle (ISO): X-forward, Y-left, Z-up

Common Tasks

1. Create a Scene

import numpy as np
from mimic_data_api.scene import Scene, EgoVehicle, Sensors
from mimic_data_api.camera import Camera, CameraIntrinsics, DistortionParameters, ImageData
from mimic_data_api.lidar import Lidar, LidarIntrinsics, VerticalResolution, LidarPointCloudData
from mimic_data_api.trajectory_numpy import WorldTrajectory

# Trajectory
timestamps = np.array([0.0, 0.1, 0.2])
poses = np.eye(4).reshape(1, 4, 4)
poses = np.repeat(poses, len(timestamps), axis=0)
ego_traj = WorldTrajectory(timestamps=timestamps, object2world=poses)

# A sensor trajectory is sensor-to-world, not the raw ego trajectory.
camera_extrinsics = np.eye(4)
camera_poses = np.stack([
    ego_pose @ camera_extrinsics for ego_pose in ego_traj.object2world
])
camera_traj = WorldTrajectory(
    timestamps=timestamps,
    object2world=camera_poses,
)

# Camera
camera = Camera(
    id=0,
    name="front_camera",
    camera_model="pinhole",
    extrinsics=camera_extrinsics,
    distortion_parameters=DistortionParameters(),
    intrinsics=CameraIntrinsics(
        fx=1000.0,
        fy=1000.0,
        cx=960.0,
        cy=540.0,
        skew=0.0,
        width=1920.0,
        height=1080.0,
    ),
    trajectory=camera_traj,
    images_data=[
        ImageData(filepath="images/front/0000.png", timestamp=float(timestamps[0]), undistorded=True),
        ImageData(filepath="images/front/0001.png", timestamp=float(timestamps[1]), undistorded=True),
        ImageData(filepath="images/front/0002.png", timestamp=float(timestamps[2]), undistorded=True),
    ],
)

# LiDAR uses the same sensor-to-world composition rule.
lidar_extrinsics = np.eye(4)
lidar_poses = np.stack([
    ego_pose @ lidar_extrinsics for ego_pose in ego_traj.object2world
])
lidar_timestamps = timestamps[:1]
lidar = Lidar(
    id=0,
    name="top_lidar",
    extrinsics=lidar_extrinsics,
    intrinsics=LidarIntrinsics(
        lidar_type="rotating",
        rotation_type="clockwise",
        vertical_fov=0.7,
        horizontal_fov=2 * np.pi,
        horizontal_resolution=0.002,
        vertical_resolution=VerticalResolution(vertical_resolution=0.035),
        max_range=120.0,
        min_range=0.5,
    ),
    trajectory=WorldTrajectory(
        timestamps=lidar_timestamps,
        object2world=lidar_poses[:1],
    ),
    lidar_pcds_data=[
        LidarPointCloudData(
            filepath="lidar/top_lidar/0000.ply",
            timestamp=float(lidar_timestamps[0]),
            egomotion_compensated=True,
            has_missing_points=False,
        )
    ],
    min_cut_off_range=0.5,
    use_for_training=True,
)

# Scene
scene = Scene(
    ego_vehicle=EgoVehicle(
        trajectory=ego_traj,
        sensors=Sensors(cameras=[camera], lidars=[lidar])
    ),
    actors=[]
)

2. Load Scene from JSON

import json
from mimic_data_api.scene import Scene

with open('scene.json') as f:
    scene_data = json.load(f)

scene = Scene.model_validate(scene_data)
print(f"Scene has {len(scene.actors)} actors")

3. Normalize LiDAR Intensity

import numpy as np

def normalize_lidar_intensity(intensities: np.ndarray) -> np.ndarray:
    x = intensities.astype(np.float32, copy=True)
    finite_mask = np.isfinite(x)
    valid_raw = x[finite_mask & (x > 0)]
    
    if valid_raw.size == 0:
        x[~finite_mask | (x <= 0)] = -1.0
        return x
    
    p99 = float(np.quantile(valid_raw, 0.999))
    fit_vals = valid_raw[valid_raw < p99]
    if fit_vals.size == 0:
        fit_vals = valid_raw
    
    log_fit_vals = np.log(np.clip(fit_vals, 1e-12, None))
    i_min = float(np.min(log_fit_vals))
    i_max = float(np.max(log_fit_vals))
    denom = max(i_max - i_min, 1e-12)
    
    nonreturn = (~finite_mask) | (x <= 0)
    outlier = (~nonreturn) & (x >= p99)
    valid = (~nonreturn) & (~outlier)
    
    if np.any(valid):
        x[valid] = (np.log(np.clip(x[valid], 1e-12, None)) - i_min) / denom
    
    x[nonreturn] = -1.0
    x[outlier] = 1.0
    
    return np.clip(x, -1.0, 1.0)

# Usage
raw_intensity = np.array([0.1, 0.5, 1.0, 5.0, 10.0])
normalized = normalize_lidar_intensity(raw_intensity)

Use -1 only for a known missing return; the SDK loader removes those points by default. This helper treats non-positive measured values as missing returns. If the source has no intensity channel, bypass the helper and write a documented non-sentinel placeholder such as 0.0; set has_missing_points according to actual missing returns, not to the absence of an intensity measurement.

4. Create LiDAR

import numpy as np
from mimic_data_api.lidar import Lidar, LidarIntrinsics, VerticalResolution, LidarPointCloudData
from mimic_data_api.trajectory_numpy import WorldTrajectory

ego_traj = WorldTrajectory(
    timestamps=np.array([0.0]),
    object2world=np.array([np.eye(4)]),
)
lidar_extrinsics = np.eye(4)
lidar_pose = ego_traj.object2world[0] @ lidar_extrinsics
lidar_trajectory = WorldTrajectory(
    timestamps=np.array([0.0]),
    object2world=np.array([lidar_pose]),
)

lidar = Lidar(
    id=0,
    name="top_lidar",
    extrinsics=lidar_extrinsics,
    intrinsics=LidarIntrinsics(
        lidar_type="rotating",
        rotation_type="clockwise",
        vertical_fov=0.7,
        horizontal_fov=2 * np.pi,
        horizontal_resolution=0.002,
        vertical_resolution=VerticalResolution(
            elevation_channel_mapping={
                0: -0.26, 1: -0.09, 2: 0.09, 3: 0.26
            }
        ),
        max_range=120.0,
        min_range=0.5,
    ),
    trajectory=lidar_trajectory,
    lidar_pcds_data=[
        LidarPointCloudData(
            filepath="lidar/top_lidar/0000.ply",
            timestamp=0.0,
            egomotion_compensated=True,
            has_missing_points=False,
        )
    ],
    min_cut_off_range=0.5,
    use_for_training=True,
)

Lidar.use_for_training is a required boolean training selector. It defaults to True; use False to disable a LiDAR stream. Do not write null.

5. Access Scene Data

# Ego position
ego_pos = scene.ego_vehicle.trajectory.object2world[0, :3, 3]
print(f"Ego at: {ego_pos}")

# Cameras
for camera in scene.ego_vehicle.sensors.cameras:
    for img in camera.images_data:
        print(f"{camera.name}: {img.filepath} @ {img.timestamp}s")

# Actors
for actor in scene.actors:
    print(f"{actor.id}: {actor.type}")

6. Write Scene to File

import json
from pathlib import Path

scene_dict = scene.model_dump(mode='json')
output_path = Path("output/scene.json")
output_path.parent.mkdir(parents=True, exist_ok=True)

with open(output_path, 'w') as f:
    json.dump(scene_dict, f, indent=2)

7. Validate Coordinate System

import numpy as np

pose = np.array([...])  # 4x4 matrix
R = pose[:3, :3]

# Check orthonormality
x_axis = R[:, 0]
y_axis = R[:, 1]
z_axis = R[:, 2]

assert np.isclose(np.dot(x_axis, y_axis), 0.0), "X and Y not orthogonal"
assert np.isclose(np.dot(y_axis, z_axis), 0.0), "Y and Z not orthogonal"

API Reference

Scene

Scene(
    ego_vehicle: EgoVehicle,
    actors: list[Actor] = []
)

EgoVehicle

EgoVehicle(
    trajectory: WorldTrajectory,
    sensors: Sensors
)

Sensors

Sensors(
    cameras: list[Camera],
    lidars: list[Lidar] | None = None,
    radars: list[Radar] | None = None
)

Camera

Camera(
    id: int,
    name: str,
    extrinsics: ndarray,          # 4x4
    intrinsics: CameraIntrinsics,
    trajectory: WorldTrajectory | None = None,
    images_data: list[ImageData] = []
)

CameraIntrinsics

CameraIntrinsics(
    resolution: list[int],         # [width, height]
    focal_length: list[float],     # [fx, fy]
    principal_point: list[float],  # [cx, cy]
    distortion: DistortionParameters | None = None,
    rolling_shutter: RollingShutterParameters | None = None
)

Lidar

Lidar(
    id: int,
    name: str,
    extrinsics: ndarray,          # 4x4
    intrinsics: LidarIntrinsics,
    trajectory: WorldTrajectory | None = None,
    lidar_pcds_data: list[LidarPointCloudData] = []
)

LidarIntrinsics

LidarIntrinsics(
    lidar_type: Literal["rotating", "solid-state"],
    rotation_type: Literal["clockwise", "counter-clockwise"] | None,
    vertical_fov: float,
    horizontal_fov: float,
    horizontal_resolution: float,
    vertical_resolution: VerticalResolution,
    max_range: float,
    min_range: float
)

Actor

Actor(
    id: str,
    type: Literal["vehicle", "pedestrian", "cyclist", "other"],
    dimensions: ActorDimensions,
    trajectory: WorldTrajectory,
    annotations: list[ActorAnnotation] | None = None
)

Dataset Metadata

Set these metadata flags correctly when building your converter:

Camera Metadata

from mimic_data_api.camera import ImageData, RollingShutterParameters, DistortionParameters

# Per-image metadata
ImageData(
    filepath="images/cam_0/0000.jpg",
    timestamp=0.0,
    undistorded=False,  # True if image is already rectified/undistorted
    rolling_shutter=RollingShutterParameters(
        readout_direction="top-to-bottom",  # or "left-to-right"
        readout_time=0.009  # sensor readout duration in seconds; 0.0 for global shutter
    ),
)

# Sensor-level: camera_model must be set
Camera(
    id=0,
    name="cam_0",
    camera_model="opencv",  # or "mei" for fisheye/omnidirectional
    distortion_parameters=DistortionParameters(
        k1=0.1, k2=-0.05, k3=0.0,
        p1=0.0, p2=0.0,
        xi=None  # set for MEI fisheye model only
    ),
    intrinsics=CameraIntrinsics(...),
    ...
)

Key Fields:

• camera_model: "opencv" for perspective cameras; "mei" for fisheye/omnidirectional

• undistorded: True if each image is already rectified; False if raw distortion is present

• rolling_shutter: Supply when the sensor does not have a global shutter

• distortion_parameters: Coefficient values from your sensor calibration file

LiDAR Metadata

from mimic_data_api.lidar import LidarPointCloudData

LidarPointCloudData(
    filepath="lidar/top_lidar/0000.ply",
    timestamp=0.0,
    egomotion_compensated=False,  # True if points are already motion-compensated
    has_missing_points=True       # True if any points have intensity=-1
)

Key Fields:

• egomotion_compensated: False for raw rolling captures; True if the provider already unrolled the cloud

• has_missing_points: True if the scan contains actual no-return points marked with intensity = -1; the SDK loader removes those points by default

Error Handling

from pydantic import ValidationError

try:
    scene = Scene.model_validate(scene_data)
except ValidationError as e:
    for error in e.errors():
        print(f"Error in {error['loc']}: {error['msg']}")

Debugging

import json

# Pretty print scene
print(json.dumps(scene.model_dump(mode='json'), indent=2))

# Check specific field
print(scene.ego_vehicle.trajectory.timestamps)

# Verify types
print(type(scene.actors[0].dimensions))

See Also

• Conversion Guide

• Validation Guide