Skip to content

[Bug Report] Camera initialization is not working when switch from distance_to_image_plane to depth #1440

@HaoranZhangumich

Description

@HaoranZhangumich

Describe the bug

In the document api section (class omni.isaac.lab.sensors.Camera), it said that "depth" is same as "distance_to_image_plane", When I switch data_types from distance_to_image_plane to depth the in CameraCfg to init a Camera instance in InteractiveSceneCfg, it would cause Camera constructor not work properly, it will not call _initialize_impl function which leads an issue in using set_intrinsic_matrices later. This is because self._data.intrinsic_matrices would return none and the assignment would fail.

Steps to reproduce

Using tutorials/04_sensors/add_sensors_on_robot.py, change the

    camera = CameraCfg(
        prim_path="{ENV_REGEX_NS}/Robot/base/front_cam",
        update_period=0.1,
        height=480,
        width=640,
        data_types=["rgb", "distance_to_image_plane"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
        ),
        offset=CameraCfg.OffsetCfg(pos=(0.510, 0.0, 0.015), rot=(0.5, -0.5, 0.5, -0.5), convention="ros"),
    )

to

    camera = CameraCfg(
        prim_path="{ENV_REGEX_NS}/Robot/base/front_cam",
        update_period=0.1,
        height=480,
        width=640,
        data_types=["rgb", "depth"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
        ),
        offset=CameraCfg.OffsetCfg(pos=(0.510, 0.0, 0.015), rot=(0.5, -0.5, 0.5, -0.5), convention="ros"),
    )

the bug will come appear and cause program to crash

System Info

Describe the characteristic of your environment:

  • Isaac Lab Version: 4.2.0
  • OS: Ubuntu 24.04
  • GPU: RTX4090
  • CUDA: 11.8
  • GPU Driver: 535.183.01

Additional context

Add any other context about the problem here.

Checklist

  • I have checked that there is no similar issue in the repo (required)
  • I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

  • Criteria 1
  • Criteria 2

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions