Google Summer of Code
Google Summer of Code (GSoC) is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school.
In 2019, I was accepted by TensorFlow to work on Adding Curiosity to TF-Agents, their reinforcement learning library. In the initial proposal, I wished to implement DQN+CTS (pseudocount-based exploration), ICM (intrinsic curiosity module), and RND (random network distillation). Unfortunately, during the duration of the summer, I was only able to implement RND. The algorithm will be merged into master soon, but to see an isolated version of code, please check this link. The blog posts below describe in more detail how the summer had progressed.
For this great summer, I must thank my mentors Oscar Ramirez and Mandar Sanjay Deshpande. As a central developer of TF-Agents, Oscar resolved many of the issues I had faced and offered guidance throughout the summer. As a former GSoC participant, Mandar gave valuable insights about having a successful summer project. I also thank Paige Bailey, our great GSoC TensorFlow organizer. I first learned about the program through Paige’s tweet, and she also provided computing resources via GCP.
GSoC TensorFlow Part 7: Retrospective
As of August 27th, the Google Summer of Code coding phase is officially over. In this post, I look back at the summer, reviewing my accomplishments and shortcomings. Because I will continue contributing to TensorFlow and TF-Agents, I also outline my plans for the next fall.
GSoC TensorFlow Part 6: Evaluating RND on Mountain Car
After finishing implementing Random Network Distillation by Burda et al., now it is time to evaluate the algorithm in various hard exploration environments. I first start the evaluation on Mountain Car, a simple environment that requires extensive exploration to reach the goal state.
GSoC TensorFlow Part 5: Implementing the Core of RND
This week, I give a brief summary of Random Network Distillation (RND), complemented with the code I have written for TF-Agents. I then list further work needed to finish implementing RND, and plans for evaluating the algorithm once finished.
GSoC TensorFlow Part 4: First Evaluation
This week, I look back at the first coding phase of GSoC, summarizing my work and setting goals for the next phase.
GSoC TensorFlow Part 3: Simple Environment Wrapper with gin-config
This week, I implemented a simple environment wrappers to prepare myself before implementing curiosity modules.
GSoC TensorFlow Part 2: Improving Documentation
A great way to learn the material is to make modifications. This week, I summarize my experience of creating a pull request to TF-Agents to improve its documentation.
GSoC TensorFlow Part 1: Setting Up TF-Agents
I have been accepted to Google Summer of Code program to work on TensorFlow for three months. I will be working on TensorFlow's reinforcement learning library TF-Agents. In this post, I briefly summmarize the steps I took to setup the TF-Agents environment for future reference.