For the moment i'm using a Dict, here is my get_observation function in my application :

def _get_observations(self) -> dict:
    cam_images = self.scene["camera"].data.output["rgb"][..., :3]
    desired_pos_b, _ = subtract_frame_transforms(
        self._robot.data.root_state_w[:, :3], self._robot.data.root_state_w[:, 3:7], self._desired_pos_w
    )

    state = torch.cat([
        self._robot.data.root_lin_vel_b,        
        self._robot.data.root_ang_vel_b,        
        self._robot.data.projected_gravity_b,   
        desired_pos_b,
    ], dim=-1)

    return {
        "policy": state,
        "Camera": cam_images
    }

If i remember correctly it is explain in the doc that this function needs to return the observations under the "policy" key, so to extend this function for my camera purpose i extended the dictionnary and i was thinking about modifying the rsl_rl wrapper
and actor_critic file as explained above to gain time and not having to do to much code.
I hope i'm answering your reply and not forgetting or misunderstanding your demand has english is not my first language.

In the Direct workflow the observation space value (fundamental or composite) needs, and it can, to be defined under the policy key. E.g.:

observation = {
    "policy": {
        "robot-state": state,
        "camera": camera,
    }
}

Now, the second part, the RL library.
Currently, only skrl (version 1.4.0: develop branch, not released yet) support learning using any space (fundamental or composite). See https://skrl.readthedocs.io/en/develop/api/utils/model_instantiators.html#inputs

You can use it... or modify the rsl_rl or rl_games Isaac Lab wrappers and libraries source code to support it.

Do you have any initial network structure? If so, I can help you defining it in the skrl's agent config file.

Ohhhhh okay i get it now i didn't know we could implement composite observation space like that it's awesome !
Regarding the initial network structure except if you are referring to the skrl_config file like this one :

i don't have one and i could clearly benefit from you help. 🙂

Thank you so much it much more clear now how i can trainmy drone !

In the Direct workflow the observation space value (fundamental or composite) needs, and it can, to be defined under the policy key. E.g.:

observation = {
    "policy": {
        "robot-state": state,
        "camera": camera,
    }
}

Now, the second part, the RL library. Currently, only skrl (version 1.4.0: develop branch, not released yet) support learning using any space (fundamental or composite). See https://skrl.readthedocs.io/en/develop/api/utils/model_instantiators.html#inputs

You can use it... or modify the rsl_rl or rl_games Isaac Lab wrappers and libraries source code to support it.

Do you have any initial network structure? If so, I can help you defining it in the skrl's agent config file.

Is there any similar setting in manage based env?

Hi @DavoodSZ1993 ,

Apologies for my late reply. It actually depends on the object/asset whose information you want to encode for the observations.
In my case since it's a vessel mesh, the network couldn't learn anything from a simple pointnet based architecture.

But if you want to encode camera images, there are few examples in Cartpole Showcase environments under Direct RL tasks

Hi @bhenriquezsoto,
Are you encoding sensor data / image data?
Since I want my agent to learn dynamic collision avoidance, I use a discrete bin strategy to identify collision spaces. Encoding was an overkill for my goal.

Hi @bhenriquezsoto,

Are you encoding sensor data / image data?

Since I want my agent to learn dynamic collision avoidance, I use a discrete bin strategy to identify collision spaces. Encoding was an overkill for my goal.

I'm trying to do active perception using a LiDAR connected to a drone through a gimbal (initially moving only the LiDAR with a pre established routine for the drone and then moving both according to a policy), I would like to train this (probably) partial RL policy in IsaacLab but I haven't seen much cases of success in this domain.

Do you happen to have any resources for this other than the one you recommended already?

Hi @bhenriquezsoto,
Are you encoding sensor data / image data?
Since I want my agent to learn dynamic collision avoidance, I use a discrete bin strategy to identify collision spaces. Encoding was an overkill for my goal.

I'm trying to do active perception using a LiDAR connected to a drone through a gimbal (initially moving only the LiDAR with a pre established routine for the drone and then moving both according to a policy), I would like to train this (probably) partial RL policy in IsaacLab but I haven't seen much cases of success in this domain.

Do you happen to have any resources for this other than the one you recommended already?

Unfortunately not in this domain. But I know this amazing work from @kousheekc - https://github.com/kousheekc/isaac_drone_racer
My strategy is partly based on this paper - https://arxiv.org/abs/2503.00496

@Sanjay1911 perfect, I will look into both resources, thank you so much, have a great day!