Anaconda (in short: conda) is a package and environment managing tool, mostly used for python. There is two major options how to use conda: Use the centralized conda installation or install conda by yourself:
centralized conda: module load anaconda
One option is to use the centralized conda module anaconda/2019.07
which is installed as a module on merlin. As soon as a user loads the module, conda will be available. There is no need to initialize it!
To try do:
module load anaconda
which conda
conda --version
This should show a path to the conda instance and the version that is currently installed (4.10.3). Remark: This conda instance is rather old and will be updated soon, might cause some problems for some difficult dependencies.
If you want to create a conda env with this conda instance, it is crucial to generate/edit your ~/.condarc
file, as the default will put both your environments and packages into your home
(only 4GB!)
Generate/Edit your file in ~/.condarc with you favorite editor as follows:
always_copy: true
envs_dirs:
- /data/user/$USER/conda/envs
pkgs_dirs:
- /data/user/$USER/conda/pkgs
- $ANACONDA_PREFIX/conda/pkgs
channels:
- conda-forge
- defaults
This will write the envs and packages to /data/user/$USER/conda/...
(max 1TB !).
Always verify this with the command conda info
displaying your current configuration! Double-check if something is not according to your expectations!
Remark: If you want to use this conda instance to activate one of your environments in a SLURM script (sbatch myscript.sh), you need to load the module anaconda inside the script. In the following simple example, the script only activates a personal environment and prints out the conda info
and which conda
command.
#!/bin/bash
#SBATCH --cluster=merlin6
#SBATCH --partition=hourly
#SBATCH --nodes=1
#SBATCH --job-name="test"
module load anaconda
conda activate /data/user/assman_g/conda/envs/test_env
conda info
which conda
personal conda: miniconda installation
If you do not want to use the central anaconda module, you can also install your own, personal conda instance. The most common solution for that is miniconda. Again, you want to install your miniconda and the associated environments + packages in /data/user/$USER
in order to prevent filling up your 4Gb home
directory.
miniconda installation
Further miniconda info: installation
Follow the steps below to install miniconda. Follow the process and check for error messages indicating that something went wrong with the installation!
cd /data/user/$USER
mkdir miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda3/miniconda.sh
bash miniconda/miniconda.sh -b -u -p miniconda3
# In order to add your conda installation to your ~/.bashrc (= initializing), do:
conda init bash
To verify your installation, check again with the following commands: conda info
, which conda
, and open your ~/.bashrc to check if a new miniconda section appears.
REMARK: It is important to undestand/keep in mind, that other conda instances (such as loading the anaconda module (see above) might cause conflicts ! Always verify which conda instance is currently active with which conda
and conda info
conda important commands
This quick steps will only be a very basic usage of conda, always have a look into the official documentation
generate a conda env
To generate a simple conda environment with a specific python version, do:
conda create -n myenv python=3.9
remove a conda env
To remove a conda env do:
conda remove --name myenv --all
general commands
conda env list : list all environments
conda info : show the current config
conda activate myenv : activate the env myenv
conda deactivate : deactivate the active env
Some Remarks:
- the currently active env will be shown in front of your username in the prompt
- to install packages, always activate the target env
- if you want to provide your env to other users, best is to generate an environment.yml file, which can be used for conda env creation.
- do not install packages into your "base" env, this is created by default upon a miniconda installation and should stay clean of packages, as this can cause problems.