This is the implementation of Neural Informed RRT* (NIRRT*), which is the algorithm in our ICRA 2024 paper
Neural Informed RRT*: Learning-based Path Planning with Point Cloud State Representations under Admissible Ellipsoidal Constraints
[Paper] [arXiv] [Main GitHub Repo] [Robot Demo GitHub Repo] [Project Google Sites] [Presentation on YouTube] [Robot Demo on YouTube]
All code was developed and tested on Ubuntu 20.04 with CUDA 12.0, conda 23.11.0, Python 3.9.0, and PyTorch 2.0.1. We also offer implmentations on RRT*, Informed RRT*, and Neural RRT* as baselines.
If you find this repo useful, please cite
@inproceedings{huang2024neural,
title={Neural Informed RRT*: Learning-based Path Planning with Point Cloud State Representations under Admissible Ellipsoidal Constraints},
author={Huang, Zhe and Chen, Hongyu and Pohovey, John and Driggs-Campbell, Katherine},
booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
pages={8742--8748},
year={2024},
organization={IEEE}
}
Run
conda env create -f environment.yml
which is recommended. Or run
conda create -n pngenv python=3.9.0
conda activate pngenv
pip install numpy==1.25.0
pip install pyyaml
pip install matplotlib
pip install opencv-python
pip install torch==2.0.1
pip install torchvision==0.15.2
pip install open3d==0.17.0
Download nirrt_star-icra24-data.zip and move the zip file into the root folder of this repo. Run
cd nirrt_star/
unzip nirrt_star-icra24-data.zip
Download nirrt_star-icra24-model-weights.zip and move the zip file into the root folder of this repo. Run
cd nirrt_star/
unzip nirrt_star-icra24-model-weights.zip
Download nirrt_star-icra24-evaluation.zip and move the zip file into the root folder of this repo. Run
cd nirrt_star/
unzip nirrt_star-icra24-evaluation.zip
- For example, to visualize a 2D Random World test sample with token
100_2, which means env_idx = 100, start_goal_idx = 2, run
conda activate pngenv
python visualize_data_samples_2d.py --visual_example_token 100_2
Check out images in visualization/img_with_labels_2d/.
- To visualize all 2D Random World test samples, run
conda activate pngenv
python visualize_data_samples_2d.py
- For 2D, run
conda activate pngenv
python demo_planning_2d.py -p nirrt_star -n pointnet2 -c bfs --problem {2D_problem_type} --iter_max 500
python demo_planning_2d.py -p nrrt_star -n unet --problem {2D_problem_type} --iter_max 500
python demo_planning_2d.py -p nrrt_star -n pointnet2 --problem {2D_problem_type} --iter_max 500
python demo_planning_2d.py -p irrt_star --problem {2D_problem_type} --iter_max 500
python demo_planning_2d.py -p rrt_star --problem {2D_problem_type} --iter_max 500
where {2D_problem_type} can be random_2d, block, or gap. Note unet cannot be used for block, as unet requires img_height % 32 == 0 and img_width % 32 == 0, while block may change the environment range randomly and does not meet this requirements. Visualizations can be found in visualization/planning_demo/.
- For 3D, run
conda activate pngenv
python demo_planning_3d.py -p nirrt_star -n pointnet2 -c bfs --problem random_3d --iter_max 500
python demo_planning_3d.py -p nrrt_star -n unet --problem random_3d --iter_max 500
python demo_planning_3d.py -p nrrt_star -n pointnet2 --problem random_3d --iter_max 500
python demo_planning_3d.py -p irrt_star --problem random_3d --iter_max 500
python demo_planning_3d.py -p rrt_star --problem random_3d --iter_max 500
Visualization will be in GUI.
If you run Result Analysis with the downloaded evaluation, check visualization/evaluation/ and you will find the images used in Fig. 5 of NIRRT* ICRA 2024 paper.
Instructions for collecting your own data.
conda activate pngenv
python generate_random_world_env_2d.py
python generate_random_world_env_2d_point_cloud.py
conda activate pngenv
python generate_random_world_env_3d_raw.py
python generate_random_world_env_3d_astar_labels.py
python generate_random_world_env_3d_point_cloud.py
conda activate pngenv
python generate_block_gap_env_2d.py
Instructions for collecting your own models.
To train and evaluate PointNet++ for guidance state inference, run
conda activate pngenv
python train_pointnet_pointnet2.py --random_seed 100 --model pointnet2 --dim 2
python eval_pointnet_pointnet2.py --random_seed 100 --model pointnet2 --dim 2
python train_pointnet_pointnet2.py --random_seed 100 --model pointnet2 --dim 3
python eval_pointnet_pointnet2.py --random_seed 100 --model pointnet2 --dim 3
If you want to train PointNet, you can replace --model pointnet2 with --model pointnet. Note that results/model_training/pointnet2_2d/checkpoints/best_pointnet2_2d.pth is equivalent as the pointnet2_sem_seg_msg_pathplan.pth you will be putting in PNGNav if you are going to deploy NIRRT* in ROS for your robot applications.
conda activate pngenv
python train_unet.py
python eval_unet.py
Run
conda activate pngenv
python eval_planning_2d.py -p nirrt_star -n pointnet2 -c bfs --problem {2D_problem_type}
python eval_planning_2d.py -p nirrt_star -n pointnet2 --problem {2D_problem_type}
python eval_planning_2d.py -p nrrt_star -n pointnet2 -c bfs --problem {2D_problem_type}
python eval_planning_2d.py -p nrrt_star -n pointnet2 --problem {2D_problem_type}
python eval_planning_2d.py -p nrrt_star -n unet --problem {2D_problem_type}
python eval_planning_2d.py -p irrt_star --problem {2D_problem_type}
python eval_planning_2d.py -p rrt_star --problem {2D_problem_type}
where {2D_problem_type} can be random_2d, block, or gap. Note unet cannot be used for block, as unet requires img_height % 32 == 0 and img_width % 32 == 0, while block may change the environment range randomly and does not meet this requirements.
Run
conda activate pngenv
python eval_planning_3d.py -p nirrt_star -n pointnet2 -c bfs
python eval_planning_3d.py -p nirrt_star -n pointnet2
python eval_planning_3d.py -p nrrt_star -n pointnet2 -c bfs
python eval_planning_3d.py -p nrrt_star -n pointnet2
python eval_planning_3d.py -p irrt_star
python eval_planning_3d.py -p rrt_star
Run
conda activate pngenv
python result_analysis_random_world_2d.py
python result_analysis_random_world_3d.py
python result_analysis_block.py
python result_analysis_gap.py
Visualizations are saved in visualization/evaluation/.
yanx27/Pointnet_Pointnet2_pytorch
rawmarshmellows/pytorch-unet-resnet-50-encoder
UCSD CSE 291 Collision Detection Material
Simple Intersection Tests for Games
Please feel free to open an issue or send an email to [email protected].