ORBIT

ORBIT is a unified and modular framework for robot learning powered by NVIDIA Isaac Sim. It offers a modular design to easily and efficiently create robotic environments with photo-realistic scenes and fast and accurate rigid and deformable body simulation. With ORBIT, we provide a suite of benchmark tasks of varying difficulty from single-stage cabinet opening and cloth folding to multi-stage tasks such as room reorganization. To support working with diverse observations and action spaces, we include fixed-arm and mobile manipulators with different physically-based sensors and motion generators. ORBIT allows training reinforcement learning policies and collecting large demonstration datasets from hand-crafted or expert solutions in a matter of minutes by leveraging GPU-based parallelization. In summary, we offer an open-sourced framework that readily comes with 16 robotic platforms, 4 sensor modalities, 10 motion generators, more than 20 benchmark tasks, and wrappers to 4 learning libraries. With this framework, we aim to support various research areas, including representation learning, reinforcement learning, imitation learning, and task and motion planning. We hope it helps establish interdisciplinary collaborations in these communities, and its modularity makes it easily extensible for more tasks and applications in the future.

I was mainly in charge of the soft object support in the simulator. With the goal of developing a benchmark, I surveyed various recent works in soft object manipulation and identified key tasks the community is working on, along with the techniques applied. From my survey, I realized that due to a lack of good simulation infrastructure, most recent works require custom infrastructures usually overfitted to the tasks investigated. To this end, I came up with a suite of tasks representing different research areas in the hope of creating a framework that all researchers can utilize. I then developed the interfaces for soft objects, such as ropes, cloths, and fluids. While tuning their physics parameters, I learned more about the solvers employed and evaluated finite-element methods against position-based dynamics. This helped me better understand the different use cases each solver is suited for and the theories behind them.

ORBIT is currently under review at the IEEE Robotics and Automation Letters and was also selected by NVIDIA to showcase the potential of robotics research using NVIDIA Omniverse at the GPU Technology Conference.

For more detail about ORBIT, check our website here. ORBIT is also open-sourced, and available here.