Blog

Learning by doing.


Python Development Environment for Earth Scientists

Nov 26, 2017 • Python • Tags: Python Cartopy

In this post, we will build a Python development environment for Earth scientists. Some features:

  • Multiple python version environment with 3.6 as default
  • cartopy enabled in python 3.6
  • rpy2 enabled so that we can use R packages in python

PyENV - multiple version management

# multiple python version environment
brew update  # update brew
brew install pyenv   # install or upgrade pyenv
brew install --HEAD pyenv-virtualenv  # install virtualenv

Then add below two lines into the config file of your shell (e.g., ~/.zshrc):

if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi
export PYENV_VIRTUALENV_DISABLE_PROMPT=1

After that, the pyenv environment is ready and we can install python via

pyenv install anaconda3-5.0.1  # install Anaconda 3 version 5.0.1
pyenv virtualenv anaconda3-5.0.1 py3.6  # create environment for python 3.6
pyenv global py3.6  # make the environment py3.6 to be globally default
# pyenv activate py3.6  # activate it in current session
pyenv which python  # to check if the corret version is activated

Note that pyenv install anaconda3-5.0.1 will fail on a server if the package cannot be downloaded without checking of certificate. In that case, one may download the package with wget --no-check-certificate https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh and put the file under ~/.pyenv/cache, then rerun pyenv install anaconda3-5.0.1 and it will be installed correctly.

Essentials

conda install -n py3.6 numpy scipy matplotlib pandas jupyter  # essentials
conda install -n py3.6 netCDF4 click seaborn statsmodels  # some add-ons
pip install nitime pathos xarray tqdm termcolor  # more add-ons

Cartopy

Cartopy is going to replace the Basemap project, while it is quite annoying to install cartopy in Python 3.6 (even at this moment - Dec, 2017). Below are the steps I find useful.

brew uninstall geos  # this step is critical to prevent some errors when import cartopy
conda install -n py3.6 -c conda-forge cartopy basemap  # we still need basemap now and then

Using R in python

Statisticians favor R language to do scientific research and there are lots of useful statistical methods coded in R, while many other scientists, like me, fovor Python instead, where those cutting-edge statistical methods have not implemented yet, therefore, it would be wonderful if we can use R packages in Python. With the project rpy2, we can “have fish and bear paw at the same time.”

First, we need to download and install R through CRAN. After that, we install rpy2 through pip

pip install rpy2

To test rpy2, in python, try

from rpy2.robjects import FloatVector

If some error like Library not loaded: /usr/local/opt/r/lib/R/lib/libR.dylib occurs, try

pip uninstall rpy2 && pip install rpy2 --no-binary rpy2

NCO: the netCDF Operator

NCO is a convenient script tool to handle netCDF files, and it could be installed easily via conda:

conda install -n py3.6 -c conda-forge nco

A typical usage example is to merge different files with the same variables but different time spans:

ncrcat file1.nc file2.nc file3.nc output.nc

With the above, file1.nc, file2.nc, file3.nc will be merged into one single file output.nc.

Tags: Python Cartopy