This is a guide for the NIPS 2018 AI for Prosthetics challenge with a helper package and a series of blog posts.


The AI for Prosthetics challenge is one of NIPS 2018 Competition tracks. In this challenge, the participants seek to build an agent that can make a 3D human model with prosthetics run.



This package contains basic learning agents using popular reinforcement learning libraries such as keras-rl and tensorforce. It provides a good starting point for those unfamiliar with practical reinforcement learning. Currently, the package contains Deep Determinsitic Policy Gradient (DDPG) agent and Proximal Policy Optimization (PPO) agent. Other popular methods and libraries such as Deep Q-Networks (DQN) and rllib should be added soon.

Blog Posts

I wrote posts about the competition, the status of the leaderboard, the properties of the environment, applicable techniques, and updates of the osim-rl-helper package. I have discussed general approaches to the problem, possible methods of reward shaping, and other techniques from recent literature that could be worth experimenting.