On 4th March 2019, the OpenAI team launched a multiagent game environment for reinforcement learning agents, called Neural MMO. This platform supports a large number of agents within a persistent and open-ended task.
The Neural MMO, a massively multiplayer online game, is designed to update new content. It supports a large and variable number of entities. It will train AI (Artificial Intelligence) to perform better in complex, open-world environments. Effective policies can be trained on a single desktop CPU.
The core features include food and water foraging system, procedural generation of tile-based terrain, and a strategic combat system. In this environment, players can join any available server while each containing an automatically generated tile-based game map of configurable size. Some tiles are traversable, such as food-bearing forest tiles and grass tiles, while others, such as water and solid stone, are not.
The team has trained a fully connected architecture using vanilla policy gradients, with a value function baseline and reward discounting as the only enhancements.
The platform comes with a procedural environment generator and visualization tools for map tile visitation distribution, value functions, and agent-agent dependencies of learned policies.
One purpose of the platform is to discover game mechanics that support complex behavior and agent populations that can learn to make use of them. In human MMOs, developers aim to create balanced mechanics while players aim to maximize their skill in utilizing them. The initial configurations of our systems are the results of several iterations of balancing, but are by no means fixed: every numeric parameter presented is editable within a simple configuration file.
Neural MMO has resolved a couple of limitations of previous game-based environments, but there are still many left unsolved.
OpenAI announcing the release of the platform in a blog post.