Home
Setup
In this lesson, we ensure that you have a working development setup for doing research in reinforcement learning. In later lessons, we may require specific packages or github repositories not included here. However, the following should get you up and running. For now, I assume you are using MacOS.
Homebrew
Homebrew is a package manager for unix that allows you to install most development packages from the command line view brew install. It is similar to apt-get for linux. Homebrew installs packages to their own directory and symlinks them into usr/local.
Paste the following into your terminal:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Atom
Atom is a simple text editor with syntax highlighting support for many languages. It also supports packages that provides extra functionality, such as autocomplete.
brew cask install atom
will install atom as an application. You can open atom from any directory using atom in terminal, or by opening it as a regular application.
Conda and Python
We will be using python 3.6 for development. While you may have python on your system already, we will be using conda to ensure that all of our experiments use their own copy of python separate from the system one. This prevents cluttering the system python with unnecessary libraries, and allows us to easily alias the python and pip commands to the ones used for our experiments. I personally prefer conda to other solutions involving virtualenv.
Conda isn’t available via brew, so download it from their website. Conda needs python to be installed initially, but we can set our own python versions when creating conda environments for our code.
Run the following in your terminal:
bash ~/Downloads/Anaconda3-2019.03-MacOSX-x86_64.sh
replace ~/Downloads/ with the appropriate path if you chose to download it to another location. You can also install using the graphical user interface by clicking on the downloaded file.
Creating a Conda Environment
To create a new conda environment, use the following syntax:
conda create -n environmentname python=x.x
To follow along, create a new environment called rl-lessons with python version 3.6. Press y to confirm you want to create the environment. Conda sets up environment variables that allow you to use python and pip to refer specifically to the ones created in your conda environment.
conda create -n rl-lessons python=3.6
To activate the environment, use
conda activate rl-lessons
and to deactivate, use
conda deactivate
It is good practice to always deactivate the current environment before activating another.
Python Libraries
There are some python packages we will need throughout our lessons. tensorflow will be responsible for doing much of the heavy lifting for us. Conda automatically installs any dependencies for tensorflow, including other libraries that we will make use of. opencv is a computer vision and image manipulation library. seaborn is a data visualization library that will help us visualize training. While conda installs its own copy of jupyter system-wide, installing it within a conda environment ensures that the python used to run notebooks is the one associated with the conda environment. More details on jupyter below.
conda install tensorflow opencv seaborn jupyter
(This will probably take a while).
OpenAI Gym
We will make extensive use of OpenAI’s gym library as our main interface to reinforcement learning. To have access some of gym’s environments, we need to install cmake and zlib
brew install cmake zlib
Then, we can install gym using pip. Conda allows us to mix packages installed via pip and via conda install. There is no conda package for gym, so we use pip.
pip install gym[atari]
Jupyter Notebooks
Jupyter notebooks are great tools because they allow you to combine markdown and python code in a single, interactive, runnable document. There are some great resources on using jupyter notebooks, which you are encouraged to check out. Most of the lessons in this series will be jupyter notebooks that you can either view or download and run yourself.