Holodeck is a high-fidelity simulator for reinforcement learning built on top of Unreal Engine 4.

• 7+ rich worlds for training agents in, and many scenarios for those worlds
• Linux and Windows support
• Easily extend and modify training scenarios
• Train and control more than one agent at once
• Simple, OpenAI Gym-like Python interface
• High performance - simulation speeds of up to 2x real time are possible. Performance penalty only for what you need
• Run headless or watch your agents learn

pip install holodeck

(requires >= Python 3.5)

See Installation for complete instructions (including Docker).

• Quickstart
• Changelog
• Examples
• Agents
• Sensors
• Available Packages and Worlds
• Docs

Holodeck's interface is similar to OpenAI's Gym.

We try and provide a "batteries included" approach to minimize the configuration you have to do.

Here is a quick example using the DefaultWorlds package:

import holodeck

# Load the environment. This environment contains a UAV in a city.
env = holodeck.make("UrbanCity-MaxDistance")

# You must call `.reset()` on a newly created environment before ticking/stepping it
env.reset()                         

# The UAV takes 3 torques and a thrust as a command.
command = [0, 0, 0, 100]   

for i in range(30):
    state, reward, terminal, info = env.step(command)  

• state: dict of sensor name to the sensor's value (nparray).
• reward: the reward received from the previous action
• terminal: indicates whether the current state is a terminal state.
• info: contains additional environment specific information.

If you want to access the data of a specific sensor, import sensors and retrieving the correct value from the state dictionary:

from holodeck.sensors import Sensors

print(state[Sensors.LOCATION_SENSOR])

Holodeck supports multi-agent environments.

Calls to step only provide an action for the main agent, and then tick the simulation.

act provides a persistent action for a specific agent, and does not tick the simulation. After an action has been provided, tick will advance the simulation forward. The action is persisted until another call to act provides a different action.

env = holodeck.make('CyberPunkCity-Follow')
env.reset()

# Provide an action for each agent
env.act('uav0', np.array([0, 0, 0, 100]))
env.act('nav0', np.array([0, 0, 0]))

# Advance the simulation
for i in range(300):
    # The action provided above is repeated
    s = env.tick()

You can access the reward, terminal and location for a multi agent environment as follows:

s['uav0'][Sensors.REWARD]
s['uav0'][Sensors.TERMINAL]
s['uav0'][Sensors.LOCATION_SENSOR]

(uav0 comes from the scenario configuration file)

Holodeck can run headless with GPU accelerated rendering. See Using Holodeck Headless

@misc{HolodeckPCCL,
  Author = {Joshua Greaves and Max Robinson and Nick Walton and Mitchell Mortensen and Robert Pottorff and Connor Christopherson and Derek Hancock and Jayden Milne and David Wingate},
  Title = {Holodeck: A High Fidelity Simulator},
  Year = {2018},
}

Holodeck is a project of BYU's Perception, Cognition and Control Lab (https://pcc.cs.byu.edu/).