What is imitation?

imitation is an open-source library providing high-quality, reliable and modular implementations of seven reward and imitation learning algorithms, built on modern backends like PyTorch and Stable Baselines3. It includes implementations of Behavioral Cloning (BC), DAgger, Generative Adversarial Imitation Learning (GAIL), Adversarial Inverse Reinforcement Learning (AIRL), Reward Learning through Preference Comparisons, Maximum Causal Entropy Inverse Reinforcement Learning (MCE IRL), and Density-based reward modeling. The algorithms follow a consistent interface, making it simple to train and compare a range of algorithms.

A key use case of imitation is as an experimental baseline. Small implementation details in imitation learning algorithms can have significant impacts on performance, which can lead to spurious positive results if a weak experimental baseline is used. To address this challenge, imitation’s algorithms have been carefully benchmarked and compared to prior implementations. The codebase is statically type-checked and over 90% of it is covered by automated tests.

In addition to providing reliable baselines, imitation aims to simplify the process of developing novel reward and imitation learning algorithms. Its implementations are modular: users can freely change the reward or policy network architecture, RL algorithm and optimizer without touching the codebase itself. Algorithms can be extended by subclassing and overriding relevant methods. imitation also provides utility methods to handle common tasks to support the development of entirely novel algorithms.

Our goal for imitation is to make it easier to use, develop, and compare imitation and reward learning algorithms. The library is in active development, and we welcome contributions and feedback.

Check out our recommended First Steps for an overview of how to use the library. We also have tutorials, such as Train an Agent using Behavior Cloning, that provide detailed examples of specific algorithms. If you are interested in helping develop imitation then we suggest you refer to the Developer Guide as well as more specific guidelines for Contributing.