API Brief Description ===================== This guide will help you get started with the MIMIC Data API. .. Installation details are intentionally omitted on this API page. .. Use the User Guides "Getting Started" page for TestPyPI installation and setup. Dependencies ^^^^^^^^^^^^ The Data API requires: - Python 3.10+ - NumPy - Pydantic v2 - numpydantic Basic Concepts -------------- Pydantic Models ^^^^^^^^^^^^^^^ All data structures in the MIMIC Data API are built on Pydantic models, providing: - **Validation**: Automatic type checking and validation on construction - **Serialization**: Easy JSON import/export with ``model_dump_json()`` and ``model_validate_json()`` - **Type Hints**: Full IDE support and autocomplete Example:: from mimic_data_api import Actor, ActorDimensions, WorldTrajectory import numpy as np # Create with validation actor = Actor( id=1, name="Vehicle_001", dimensions=ActorDimensions(length=4.5, width=1.8, height=1.5), boxtype="vehicle", dynamic=True, trajectory=WorldTrajectory( timestamps=np.array([0.0, 0.1]), object2world=np.stack([np.eye(4), np.eye(4)]) ) ) # Serialize to JSON json_str = actor.model_dump_json(indent=2) # Deserialize from JSON actor_copy = Actor.model_validate_json(json_str) Coordinate Frames ^^^^^^^^^^^^^^^^^ The API uses right-handed coordinate frames that vary by sensor type: **Vehicle/LiDAR/Radar Frame (FLU - Forward, Left, Up):** - **X-axis**: Forward (direction of travel) - **Y-axis**: Left (perpendicular to travel direction) - **Z-axis**: Up (vertical) **Camera Frame (OpenCV Convention):** - **X-axis**: Right (horizontal, perpendicular to optical axis) - **Y-axis**: Down (vertical, perpendicular to optical axis) - **Z-axis**: Forward (optical axis, pointing into the scene) **World Frame:** - Fixed global coordinates - Typically ENU (East-North-Up) or a local tangent plane - Scene-dependent orientation Transformation Matrices ^^^^^^^^^^^^^^^^^^^^^^^ Poses are represented as 4x4 homogeneous transformation matrices: .. math:: T = \begin{bmatrix} R_{3 \times 3} & t_{3 \times 1} \\ 0_{1 \times 3} & 1 \end{bmatrix} Where: - :math:`R` is a 3x3 rotation matrix - :math:`t` is a 3x1 translation vector To transform a point from local to world coordinates:: import numpy as np # Point in local coordinates (homogeneous) p_local = np.array([1.0, 0.0, 0.0, 1.0]) # Transformation matrix T = trajectory.object2world[0] # First pose # Transform to world coordinates p_world = T @ p_local WorldTrajectory ^^^^^^^^^^^^^^^ The ``WorldTrajectory`` class is the fundamental building block for representing time-series pose data:: from mimic_data_api import WorldTrajectory import numpy as np # Create a 100-frame trajectory at 10 Hz n_frames = 100 trajectory = WorldTrajectory( timestamps=np.linspace(0, 10, n_frames), # 0 to 10 seconds object2world=np.stack([np.eye(4)] * n_frames), # Identity poses linear_velocity=np.zeros((n_frames, 3)), # Optional velocities angular_velocity=np.zeros((n_frames, 3)) # Optional angular velocities ) # Access specific time t5_pose = trajectory.object2world[50] # Pose at t=5s position = t5_pose[:3, 3] # Extract translation rotation = t5_pose[:3, :3] # Extract rotation matrix Working with Sensors -------------------- Cameras ^^^^^^^ Camera sensors include intrinsic calibration, distortion parameters, and image data:: from mimic_data_api import ( Camera, CameraIntrinsics, DistortionParameters, ImageData ) import numpy as np camera = Camera( id=1, name="front_camera", camera_model="opencv", # Supports various models intrinsics=CameraIntrinsics( fx=1000.0, fy=1000.0, # Focal lengths cx=640.0, cy=360.0, # Principal point skew=0.0, width=1280, height=720 ), distortion_parameters=DistortionParameters( k1=-0.1, k2=0.01, # Radial distortion p1=0.001, p2=-0.001 # Tangential distortion ), extrinsics=np.eye(4), # Sensor to vehicle transform trajectory=ego_trajectory, images_data=[ ImageData(timestamp=0.0, filepath="img_000.png", undistorded=False), ImageData(timestamp=0.1, filepath="img_001.png", undistorded=False), ] ) # Get intrinsic matrix K = camera.intrinsics.as_K() # Returns 3x3 numpy array LiDAR Sensors ^^^^^^^^^^^^^ LiDAR sensors support both rotating and solid-state types:: from mimic_data_api import ( Lidar, LidarIntrinsics, VerticalResolution, LidarPointCloudData ) import numpy as np lidar = Lidar( id=1, name="top_lidar", extrinsics=np.eye(4), intrinsics=LidarIntrinsics( lidar_type="rotating", rotation_type="clockwise", vertical_fov=0.7, # ~40 degrees in radians horizontal_fov=2 * np.pi, # 360 degrees horizontal_resolution=0.002, # radians/point (angular step), not points/radian vertical_resolution=VerticalResolution(vertical_resolution=0.035), max_range=120.0, min_range=0.5 ), trajectory=ego_trajectory, lidar_pcds_data=[ LidarPointCloudData( filepath="lidar_000.ply", timestamp=0.0, egomotion_compensated=True, has_missing_points=False ) ], min_cut_off_range=2.0 # Filter ego vehicle points ) Radar Sensors ^^^^^^^^^^^^^ Radar sensors include velocity measurement capabilities:: from mimic_data_api import Radar, RadarIntrinsics, RadarPointCloudData import numpy as np radar = Radar( id=1, name="front_radar", extrinsics=np.eye(4), intrinsics=RadarIntrinsics( range=200.0, # Max range in meters azimuth_fov=2.0, # Horizontal FOV in radians elevation_fov=0.35, # Vertical FOV in radians azimuth_resolution=[(0.0, 0.017), (1.0, 0.035)], # Angle-dependent elevation_resolution=None, # Non-imaging radar min_speed_resolution=-70.0, # m/s max_speed_resolution=70.0 # m/s ), trajectory=ego_trajectory, radar_pcds_data=[ RadarPointCloudData(filepath="radar_000.ply", timestamp=0.0) ] ) Working with Actors ------------------- Actors represent other objects in the scene:: from mimic_data_api import Actor, ActorDimensions, WorldTrajectory import numpy as np # Create a moving vehicle vehicle = Actor( id=42, name="sedan_001", dimensions=ActorDimensions(length=4.5, width=1.8, height=1.5), boxtype="vehicle", original_boxtype="sedan", # Preserve original label dynamic=True, trajectory=WorldTrajectory( timestamps=np.array([0.0, 0.1, 0.2]), object2world=vehicle_poses # (3, 4, 4) array ), description="Blue sedan in adjacent lane" ) # Create a static sign sign = Actor( id=100, name="stop_sign", dimensions=ActorDimensions(length=0.1, width=0.6, height=2.0), boxtype="sign", dynamic=False, trajectory=WorldTrajectory( timestamps=np.array([0.0]), object2world=np.eye(4).reshape(1, 4, 4) ) ) Creating Complete Scenes ------------------------ Combine all elements into a Scene:: from mimic_data_api import Scene, EgoVehicle, Sensors scene = Scene( ego_vehicle=EgoVehicle( trajectory=ego_trajectory, sensors=Sensors( cameras=[front_camera, rear_camera], lidars=[top_lidar], radars=[front_radar, rear_radar] ) ), actors=[vehicle, pedestrian, cyclist, sign] ) # Serialize to JSON with open("scene.json", "w") as f: f.write(scene.model_dump_json(indent=2)) # Load from JSON with open("scene.json", "r") as f: loaded_scene = Scene.model_validate_json(f.read()) JSON Serialization ------------------ The Data API uses Pydantic's JSON serialization, which handles NumPy arrays automatically:: # Export json_str = scene.model_dump_json(indent=2) # Import scene = Scene.model_validate_json(json_str) # Export to dict (for custom processing) data_dict = scene.model_dump() For working with files:: from pathlib import Path # Save scene Path("scene.json").write_text(scene.model_dump_json(indent=2)) # Load scene scene = Scene.model_validate_json(Path("scene.json").read_text()) Next Steps ---------- - See :doc:`data_structures` for detailed class documentation - Check :doc:`examples` for complete usage examples - Explore :doc:`example_scene` for a sample JSON scene file - Review the :doc:`api/scene` API reference