OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. The gym library is a collection of environments that makes no assumptions about the structure of your agent. Gym comes with a diverse suite of environments, ranging from classic video games and continuous control tasks.
To learn more about OpenAI Gym, check the official documentation here.
Atari 2600
Atari 2600 is a video game console from Atari released on 1977. The game console includes popular games such as Breakout, Ms. Pacman and Space Invaders. Since Deep Q-Networks were introduced by Mnih et al. in 2013, Atari 2600 has been the standard environment to test new Reinforcement Learning algorithms. Atari 2600 has been a challenging testbed due to its high-dimensional video input (size 210 x 160, frequency 60 Hz) and the discrepancy of tasks between games.
MuJoCo
MuJoCo (Multi-Joint dynamics with Contact) is a proprietary physics engine for detailed, efficient rigid body simulations with contacts. MuJoCo can be used to create environments with continuous control tasks such as walking or running, so many policy gradient methods have been tested primarily on various MuJoCo environments.