Skip to content

Set Up Remote Server for Deep Learning

Install Pyenv

Update System Packages

Terminal window
sudo apt update

(Optional): Install Dependencies

Install optional dependencies with

Terminal window
sudo apt install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev \
xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

or

sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git

according to OpenAI’s gpt-4o-2024-05-13.

Install Pyenv

curl https://pyenv.run | bash

Set Up Environment Variables

# Load pyenv automatically by appending
# the following to
# ~/.bash_profile if it exists, otherwise ~/.profile (for login shells)
# and ~/.bashrc (for interactive shells) :
echo -e 'export PYENV_ROOT="$HOME/.pyenv"\n[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo -e 'eval "$(pyenv init -)"' >> ~/.bashrc
# Restart your shell for the changes to take effect.
# Load pyenv-virtualenv automatically by adding
# the following to ~/.bashrc:
echo -e 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc

Don’t forget to restart your shell for the changes to take effect.

source ~/.bashrc

Install Python & Set Up Virtual Environment with Pyenv

Check available versions

pyenv install --list

Install specific version

pyenv install 3.10.14

Set global version with

pyenv global 3.10.14

or just assign to current shell with

pyenv shell 3.10.14

Install pyenv-virtualenv

git clone https://github.com/pyenv/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

Update shell configuration:

eval "$(pyenv virtualenv-init -)"

Restart your shell:

exec $SHELL

Create a new virtual environment:

pyenv virtualenv 3.10.14 myenv

Activate the virtual environment:

pyenv activate myenv

Install Packages with pip

install packages with pip using mirrors

For one time use, install packages with mirror address

pip install <package> -i https://pypi.tuna.tsinghua.edu.cn/simple
# pip install <package> -i http://mirrors.aliyun.com/pypi/simple --truste-host mirrors.aliyun.com

For global set up, create file ~/.pip/pip.conf with the following content.

[global]
index-url=https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host=https://pypi.tuna.tsinghua.edu.cn

Create folder ~/.pip/ if it does not exist.

$ pip config list
global.index-url='https://pypi.tuna.tsinghua.edu.cn/simple'
install.trusted-host='https://pypi.tuna.tsinghua.edu.cn'

Python Package Management with Poetry

install poetry and add to PATH

curl -sSL https://install.python-poetry.org | python3 -
echo 'export PATH=$PATH:$HOME/.local/bin' >> ~/.bashrc
source ~/.bashrc

additional setup

I prefer to use virtual environment under the current project directory.

poetry config virtualenvs.in-project true

use poetry config --list to check current settings.

create new project

cd my-project
poetry init
poetry env use python
# poetry env use /Users/caijiaqi/.pyenv/versions/3.10.14/bin/python

install dependencies

poetry add <package>

Two files, pyproject.toml and poetry.lock, are created and updated accordingly by now. When you use poetry add command, Poetry will automatically do the following three things in this order:

  1. update pyproject.toml
  2. update poetry.lock according to pyproject.toml
  3. update virtual environment according to poetry.lock

read more

Git Clone Projects for Remote Development

clone private projects

git clone https://<github-username>:<PAT>@github.com/<your-repository>.git
Create GitHub Personal Access Token(PAT)
  1. In GitHub, go to Settings —> Developer Settings —> Personal access tokens.
  2. Click Generate new token and assign the token a name.
  3. Grant the token privilege to the repo group.
  4. Copy the token to a secure location. You won’t be able to view the token again.
  5. At the bottom of the page, click Generate Token.