Set Up Remote Server for Deep Learning

Install Pyenv

Tip

Manage multiple versions of Python with pyenv.

Update System Packages

sudo apt update

(Optional): Install Dependencies

Install optional dependencies with

sudo apt install -y make build-essential libssl-dev zlib1g-dev \
  libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev \
  xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

or

sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
  libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
  xz-utils tk-dev libffi-dev liblzma-dev python-openssl git

according to OpenAI’s gpt-4o-2024-05-13.

Install Pyenv

curl https://pyenv.run | bash

Set Up Environment Variables

# Load pyenv automatically by appending
# the following to
# ~/.bash_profile if it exists, otherwise ~/.profile (for login shells)
# and ~/.bashrc (for interactive shells) :
echo -e 'export PYENV_ROOT="$HOME/.pyenv"\n[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo -e 'eval "$(pyenv init -)"' >> ~/.bashrc
# Restart your shell for the changes to take effect.
# Load pyenv-virtualenv automatically by adding
# the following to ~/.bashrc:
echo -e 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc

Don’t forget to restart your shell for the changes to take effect.

Bash

source ~/.bashrc

Zsh

source ~/.zshrc

Install Python & Set Up Virtual Environment with Pyenv

Check available versions

pyenv install --list

Install specific version

pyenv install 3.10.14

Set global version with

pyenv global 3.10.14

or just assign to current shell with

pyenv shell 3.10.14

Install pyenv-virtualenv

git clone https://github.com/pyenv/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv

Update shell configuration:

eval "$(pyenv virtualenv-init -)"

Restart your shell:

exec $SHELL

Create a new virtual environment:

pyenv virtualenv 3.10.14 myenv

Activate the virtual environment:

pyenv activate myenv

Install Packages with pip

install packages with pip using mirrors

For one time use, install packages with mirror address

pip install <package> -i https://pypi.tuna.tsinghua.edu.cn/simple
# pip install <package> -i http://mirrors.aliyun.com/pypi/simple --truste-host mirrors.aliyun.com

For global set up, create file ~/.pip/pip.conf with the following content.

[global]
index-url=https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host=https://pypi.tuna.tsinghua.edu.cn

Create folder ~/.pip/ if it does not exist.

$ pip config list
global.index-url='https://pypi.tuna.tsinghua.edu.cn/simple'
install.trusted-host='https://pypi.tuna.tsinghua.edu.cn'

Tip

Some other significant mirrors

• 中国科学技术大学 : https://pypi.mirrors.ustc.edu.cn/simple
• 豆瓣：http://pypi.douban.com/simple/
• 阿里云：http://mirrors.aliyun.com/pypi/simple/

Python Package Management with Poetry

install poetry and add to PATH

curl -sSL https://install.python-poetry.org | python3 -
echo 'export PATH=$PATH:$HOME/.local/bin' >> ~/.bashrc
source ~/.bashrc

additional setup

I prefer to use virtual environment under the current project directory.

poetry config virtualenvs.in-project true

use poetry config --list to check current settings.

create new project

cd my-project
poetry init

poetry env use python
# poetry env use /Users/caijiaqi/.pyenv/versions/3.10.14/bin/python

Tip

Note that the python version is determined by python so choose the desired one.

install dependencies

poetry add <package>

Two files, pyproject.toml and poetry.lock, are created and updated accordingly by now. When you use poetry add command, Poetry will automatically do the following three things in this order:

1. update pyproject.toml
2. update poetry.lock according to pyproject.toml
3. update virtual environment according to poetry.lock

Caution

Do not manually edit poetry.lock and pyproject.toml. If you manually edited pyproject.toml, run poetry lock to update poetry.lock file.

read more

• Python 套件管理器——Poetry 完全入門指南

Git Clone Projects for Remote Development

clone private projects

git clone https://<github-username>:<PAT>@github.com/<your-repository>.git

Create GitHub Personal Access Token(PAT)

1. In GitHub, go to Settings —> Developer Settings —> Personal access tokens.
2. Click Generate new token and assign the token a name.
3. Grant the token privilege to the repo group.
4. Copy the token to a secure location. You won’t be able to view the token again.
5. At the bottom of the page, click Generate Token.