Data Structures =============== This section provides an overview of all data structures in the MIMIC Data API. Core Classes ------------ Scene ^^^^^ The top-level container for all scene data. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``ego_vehicle`` - The recording/autonomous vehicle with sensors and trajectory * - ``actors`` - List of all other objects in the scene See :doc:`api/scene` for full documentation. EgoVehicle ^^^^^^^^^^ Represents the ego (recording) vehicle. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``trajectory`` - Time-series of ego poses in world coordinates * - ``sensors`` - Collection of mounted sensors (cameras, lidars, radars) Sensors ^^^^^^^ Container for all sensor data. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``cameras`` - List of camera sensors (required) * - ``lidars`` - List of LiDAR sensors (optional) * - ``radars`` - List of radar sensors (optional) Trajectory ---------- WorldTrajectory ^^^^^^^^^^^^^^^ Time-indexed sequence of poses in world coordinates. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``timestamps`` - 1D array of N timestamps in seconds * - ``object2world`` - (N, 4, 4) array of transformation matrices * - ``linear_velocity`` - Optional (N, 3) array of velocities in m/s * - ``angular_velocity`` - Optional (N, 3) array of angular velocities in rad/s See :doc:`api/trajectory` for full documentation. Camera Structures ----------------- Camera ^^^^^^ Complete camera sensor representation. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``id`` - Unique identifier * - ``name`` - Human-readable name (e.g., "front_camera") * - ``camera_model`` - Projection model type (pinhole, opencv, fisheye, etc.) * - ``intrinsics`` - Internal parameters (focal length, principal point) * - ``distortion_parameters`` - Lens distortion coefficients * - ``extrinsics`` - Sensor-to-vehicle transformation matrix * - ``trajectory`` - Camera poses over time * - ``images_data`` - List of captured images CameraIntrinsics ^^^^^^^^^^^^^^^^ Internal camera calibration parameters. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``fx``, ``fy`` - Focal lengths in pixels * - ``cx``, ``cy`` - Principal point coordinates * - ``skew`` - Skew coefficient (usually 0) * - ``width``, ``height`` - Image dimensions in pixels DistortionParameters ^^^^^^^^^^^^^^^^^^^^ Lens distortion coefficients supporting multiple camera models. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``k1``, ``k2``, ``k3``, ``k4`` - Radial distortion coefficients * - ``p1``, ``p2`` - Tangential distortion coefficients * - ``k5``, ``k6`` - Extended radial coefficients (optional) * - ``s1``, ``s2``, ``s3`` - Thin prism coefficients (optional) ImageData ^^^^^^^^^ Metadata for a single captured image. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``filepath`` - Path to image file (PNG or JPEG) * - ``timestamp`` - Capture time in seconds * - ``undistorded`` - Whether distortion has been removed * - ``rolling_shutter`` - Rolling shutter parameters See :doc:`api/camera` for full documentation. LiDAR Structures ---------------- Lidar ^^^^^ Complete LiDAR sensor representation. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``id`` - Unique identifier * - ``name`` - Human-readable name (e.g., "top_lidar") * - ``intrinsics`` - Sensor parameters (FOV, resolution, range) * - ``extrinsics`` - Sensor-to-vehicle transformation * - ``trajectory`` - LiDAR poses over time * - ``lidar_pcds_data`` - List of point cloud captures * - ``min_cut_off_range`` - Ego vehicle filter threshold LidarIntrinsics ^^^^^^^^^^^^^^^ Internal LiDAR parameters. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``lidar_type`` - "rotating" or "solid-state" * - ``rotation_type`` - "clockwise" or "counter-clockwise" * - ``vertical_fov`` - Vertical field of view in radians * - ``horizontal_fov`` - Horizontal field of view in radians * - ``horizontal_resolution`` - Radians per point (angular step between azimuth samples), e.g. ``2*pi/num_points_per_rotation``. Not points per radian. * - ``vertical_resolution`` - Vertical resolution specification * - ``max_range``, ``min_range`` - Detection range limits in meters VerticalResolution ^^^^^^^^^^^^^^^^^^ Vertical angular resolution specification. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``vertical_resolution`` - Uniform resolution in radians per point (angular step), not points/radian * - ``elevation_channel_mapping`` - Per-channel angle mapping (alternative) *Note: Exactly one of these must be provided.* LidarPointCloudData ^^^^^^^^^^^^^^^^^^^ Metadata for a single LiDAR scan. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``filepath`` - Path to PLY file * - ``timestamp`` - Mid-scan timestamp in seconds * - ``egomotion_compensated`` - Whether motion compensation applied * - ``has_missing_points`` - Whether invalid points exist See :doc:`api/lidar` for full documentation. Radar Structures ---------------- Radar ^^^^^ Complete radar sensor representation. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``id`` - Unique identifier * - ``name`` - Human-readable name * - ``intrinsics`` - Sensor parameters * - ``extrinsics`` - Sensor-to-vehicle transformation * - ``trajectory`` - Radar poses over time * - ``radar_pcds_data`` - List of radar detections RadarIntrinsics ^^^^^^^^^^^^^^^ Internal radar parameters. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``range`` - Maximum detection range in meters * - ``azimuth_fov`` - Horizontal FOV in radians * - ``elevation_fov`` - Vertical FOV in radians * - ``azimuth_resolution`` - Angle-dependent horizontal resolution * - ``elevation_resolution`` - Vertical resolution (imaging radars) * - ``min_speed_resolution`` - Minimum Doppler velocity (m/s) * - ``max_speed_resolution`` - Maximum Doppler velocity (m/s) See :doc:`api/radar` for full documentation. Actor Structures ---------------- Actor ^^^^^ Representation of scene objects (vehicles, pedestrians, etc.). .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``id`` - Unique identifier * - ``name`` - Optional human-readable name * - ``dimensions`` - Physical bounding box size * - ``boxtype`` - Category: "vehicle", "pedestrian", "bicycle", "sign", "unknown" * - ``original_boxtype`` - Original dataset label (optional) * - ``dynamic`` - True if moving, False if static * - ``trajectory`` - Actor poses over time * - ``description`` - Optional free-text description ActorDimensions ^^^^^^^^^^^^^^^ Physical dimensions of an actor. .. list-table:: :widths: 25 75 :header-rows: 1 * - Attribute - Description * - ``length`` - Length in meters (forward axis) * - ``width`` - Width in meters (lateral axis) * - ``height`` - Height in meters (vertical axis) See :doc:`api/actor` for full documentation. Type Aliases ------------ SensorExtrinsic ^^^^^^^^^^^^^^^ A 4x4 NumPy array representing the sensor-to-vehicle transformation matrix:: SensorExtrinsic = NDArray[Shape["4 x, 4 y"], float] Camera Models ^^^^^^^^^^^^^ Supported camera projection models:: CameraModels = Literal[ "simple_pinhole", "pinhole", "simple_radial", "radial", "opencv", "opencv_fisheye", "equirectangular", "omnidirectionalstereo_l", "omnidirectionalstereo_r", "vr180_l", "vr180_r", "orthophoto", "fisheye624", ] These models are aligned with `COLMAP `_ conventions.