Manipulation Template for IsaacLab

Overview

This repository is based on the old version (orbit) of the orbit extension repository from isaaclab and provides a template for research on robot manipulation tasks. It is independent of isaaclab and allows for customization of the details of manipulation tasks.

Setup

Dependencies

This template depends on Isaac Sim and IsaacLab.

• Isaac Sim
• Isaac Lab

It is recommended to install in a virtual environment. IsaacLab provides a default virtual environment that can be activated directly.

# Activate conda environment
conda activate isaaclab

Download

git clone https://github.com/NathanWu7/isaacLab.manipulation.git
cd isaacLab.manipulation
python -m pip install -e .

Convert urdf to usd

# urdf files in isaacLab.manipulation/isaacLab/manipulation/assets/urdf 
# usd files in isaacLab.manipulation/isaacLab/manipulation/assets/usd
python3 scripts/tools/convert_urdf.py urdf/your_urdf.urdf usd/your_usd.usd

About isaac nucleus

If you dont want to use the assets in isaac nucleus, please comment the following code and replace it with your own assets path.

from omni.isaac.lab.utils.assets import ISAAC_NUCLEUS_DIR
# from isaacLab.manipulation.assets.config import your_robot

RL Algorithm

Install RSL_RL outside of the isaacLab repository, e.g. home/code/rsl_rl.

mkdir isaacLab.manipulation/isaacLab/manipulation/algorithms
cd isaacLab.manipulation/isaacLab/manipulation/algorithms
git clone https://github.com/leggedrobotics/rsl_rl.git
cd rsl_rl
python -m pip install -e .

## refresh index
cd isaacLab.manipulation
python -m pip install -e .

You can design your own RL Algorithm by editing "modules" and "algorithms" in RSL-RL

Usage

#root of the package
cd isaacLab.manipulation

1. Convert your urdf to usd

python3 scripts/tools/convert_urdf.py urdf/your_urdf.urdf usd/your_usd.usd

2. Set up your robot or objects

cd isaacLab.manipulation/isaacLab/manipulation/assets/config
touch your_robot.py
# kinova_gipper.py is an example. You can copy the file and change robot_usd = "kinova_robotiq.usd" to your own usd and apply some changes in ArticulationCfg 

And add env configs for each tasks

mkdir isaacLab.manipulation/isaacLab/manipulation/tasks/Robot_arm/your_tasks/config/your_robot_env
# kinova_gripper in reach task is an example

3. RL settings

You can modify actions/rewards/observations/events/terminations in the directory "mdp" to set up your own RL settings and you can manage them uniformly in "env_cfg" outside "mdp". These files modify the functions of the original isaaclab through rewritting and overloading.

IMPORTANT ISSUES

1. 'RigidBodyView' object has no attribute 'get_accelerations.
   Solution: Update Isaac Sim to version 4.1.0.
2. argument --cpu: conflicting option string: --cpu
   Solution: Annotate the following code in play.py and train.py:

parser.add_argument("--cpu", action="store_true", default=False, help="Use CPU pipeline.")

Then remove "use_gpu = not args_cli.cpu" in the following code:

parse_env_cfg(args_cli.task, use_gpu=not args_cli.cpu, num_envs=args_cli.num_envs)

4. Train a policy.

4.1 RobotArm

python3 scripts/rsl_rl/train.py --task Template-Isaac-Reach-Kinova-v0 --num_envs 4096 --headless

python3 scripts/rsl_rl/train.py --task Template-Isaac-Reach-Franka-v0 --num_envs 4096 --headless

python3 scripts/rsl_rl/train.py --task Template-Isaac-Reach-UR10-v0 --num_envs 4096 --headless

4.2 Dextrous Hand

python3 scripts/rsl_rl/train.py --task Template-Isaac-Repose-Cube-Allegro-v0 --num_envs 4096 --headless

5. Play the trained policy.

5.1 RobotArm

python3 scripts/rsl_rl/play.py --task Template-Isaac-Reach-Kinova-Play-v0--num_envs 16

# You can also use train.py if you dont need to add some additional configs.
python3 scripts/rsl_rl/play.py --task Template-Isaac-Reach-Franka-Play-v0 --num_envs 16

python3 scripts/rsl_rl/play.py --task Template-Isaac-Reach-Franka-Play-v0 --num_envs 16

5.2 Dextrous Hand

python3 scripts/rsl_rl/play.py --task Template-Isaac-Repose-Cube-Allegro-Play-v0 --num_envs 16

Author

Author: Qiwei Wu
Email: nathan.wuqw@gmail.com