Speakers

Simulation from states and sensors

Abstract: “Intuitive physics”, or the ability to imagine how the future of physical systems will unfold, is a hallmark of human intelligence. This capacity supports many complex behaviors in humans including planning, problem-solving, and tool creation. In this talk, I will describe some of our work aiming to learn physical simulators from data in order to support these downstream behaviors. The first half of the talk will focus on what we can learn from state-based data. I will present new types of graph neural networks for learning realistic rigid body simulation by taking inspiration from classic approaches in graphics. I will show that these learned simulators can capture rigid body dynamics with unprecedented accuracy, including modeling real world dynamics significantly better than system identification with analytic simulators from robotics. The second half of the talk will focus on what we can learn from sensor data. I will present Visual Particle Dynamics, a method which connects neural radiance fields with graph neural networks to enable learning directly from RGB-D data. Unlike existing 2D video prediction models, we show that VPD's 3D structure enables scene editing and long-term predictions. I will conclude by highlighting open challenges for future work.

Building High-Performance Differentiable Simulators in Warp

Abstract: Differentiable simulators offer efficient gradient access, facilitating advancements in control optimization, system identification, and numerous other applications. However, developing these simulators is challenging, often requiring significant effort to achieve efficient hardware acceleration on modern GPUs, maintain readability and ease of maintenance in code, and implement optimal gradient computations for the simulation routines.

In this talk, I will introduce Warp, a Python framework engineered for high-performance GPU simulation and graphics, designed specifically to address these challenges. Warp offers an extensive library of built-in functions and data structures tailored for spatial computing and simulation. Its support for automatic differentiation, leveraging code generation, enables streamlined high-performance gradient computation for differentiable simulations and beyond. Warp kernels are translated to CUDA for hardware acceleration, granting users fine-grained control over memory allocation and kernel launches. This architecture gives Warp a notable performance advantage over comparable frameworks. I will conclude by showcasing applications where Warp has been effectively utilized in differentiable simulation tasks.

Learning Structured World Models From and For Physical Interactions

Abstract: Humans possess a strong intuitive understanding of the physical world. Through observations and interactions with our environment, we build mental models that predict how the world changes when we apply specific actions (i.e., intuitive physics). My research builds on these insights to develop model-based reinforcement learning (RL) agents that, through interaction, construct neural-network-based predictive models capable of generalizing across a range of objects made from diverse materials. The core idea behind my work is to introduce novel representations and integrate structural priors into learning systems to model dynamics at various levels of abstraction. I will discuss how such structures enhance model-based planning algorithms, enabling robots to accomplish complex manipulation tasks (e.g., manipulating object piles, shaping deformable foam to match target configurations, and crafting dumplings from dough using various tools). Furthermore, I will present our recent progress in combining neural dynamics models with a GPU-accelerated branch-and-bound framework, enabling more effective long-horizon trajectory optimization in challenging, contact-rich manipulation tasks, such as non-prehensile planar pushing with obstacles, object sorting, and rope routing, both in simulation and real-world scenarios.

Contact-Implicit Model Predictive Control on Legged Robots

Abstract: Legged robots are inherently modeled as systems with impacts, and traditional model predictive control (MPC) algorithms typically separate the generation of torque inputs from the generation of contact sequences in their optimization formulation. Despite recent advancements in MPC for legged robots that incorporate contact sequences in the optimization, only a few algorithms have demonstrated the computational efficiency needed to operate under real-time constraints, and even fewer have actually been implemented on real hardware systems. In this presentation, I will introduce our new contact-implicit model predictive control for legged robots, which has been implemented and experimentally verified on real robot hardware. Our MPC framework enables real-time discovery of multi-contact motions without predefined contact mode sequences or foothold positions. The core idea behind the framework is an analytical gradient that incorporates impulses due to a hard contact model and relaxed complementarity constraints to explore a variety of contact modes. This approach is employed in the multiple shooting variant of Differential Dynamic Programming (DDP). The efficacy of the proposed framework is validated through simulations and real-world demonstrations using a 45 kg HOUND quadruped robot, performing various tasks in simulation and showcasing actual experiments involving forward trotting and front-leg rearing motions.

Differentiable Logic Gate Networks & Reinforcement Learning

Abstract: Differentiable logic gate networks relax logical circuits, making them trainable, and thus usable as machine learning models, enabling state-of-the-art efficiency for hardened machine learning models. 

After covering differentiable LGNs in general for machine learning and vision applications, we will bridge the gap to reinforcement learning, where I will also be presenting new results, achieving nanosecond latencies for reinforcement learning agents. 

Large-Scale Manipulation Data Acquisition through Differentiable Simulation and Rendering

Abstract: To substantially advance robot intelligence, a significant effort is needed to collect large-scale manipulation data. I will introduce three differentiable simulations we have developed—Jade, DiffclothAI, and Softmac—which are tailored for articulated, 2D deformable, and 3D deformable objects, respectively. These differentiable simulations support intersection-free contact modeling and coupling.

Following this, I will discuss leveraging these differentiable simulations, combined with differentiable rendering, to extract manipulation data directly from videos. I will highlight two key projects: the first demonstrates using differentiable simulation and rendering to estimate object shapes, meshes, and poses in a self-supervised manner from video. I will then introduce TieBot, our latest work that extends this framework to teach robots to knot a tie from video demonstrations. Additionally, I will cover our recent efforts to streamline annotation processing, making it more time-efficient. I will conclude by outlining future research directions to further enhance robot learning and manipulation capabilities.

Finite differencing is all you need: lessons and insights

Abstract: coming soon

Leveraging the internals of MuJoCo physics to improve model-based control

Abstract: Much progress has been made by using physics simulators as black boxes, and optimizing controllers via model-free sampling. Such methods are difficult to apply directly to physical systems and work better when a simulation model is available - which blurs the meaning of "model-free". It also begs the question, why not utilize the simulator more fully, beyond the black-box approach advocated in Reinforcement Learning? In this talk I will describe progress towards a new software framework called Optico, which leverages unique aspects of MuJoCo physics for the purpose of model-based optimization. MuJoCo already relies on (convex) optimization to compute constraint forces and advance the simulation state. This creates opportunities to blend optimization-for-physics and optimization-for-control, in ways that have no analog in the black-box approach. Access to various analytical derivatives is essential for these developments. The resulting low-level solvers are meant to improve "control learnability", and can be applied in the context of both sampling and derivative-based learning methods. They are also fast enough for model-predictive control.