Shell Environment on the Cluster

The Discovery Shell Environment and .bashrc

Discovery uses a Linux-based operating system (CentOS), where the shell program interfaces with the user. Bash (Bourne Again Shell) is one of the most popular shell implementations and the default shell on Discovery.

The shell script .bashrc is used by bash to initialize your shell environment. For example, it is typically used to define aliases, functions, and load modules. Note that environment variables settings (such as PATH) generally go in the .bash_profile or .profile files. Your .bashrc, .bash_profile, and .profile files live in your $HOME directory. You can change your .bashrc with a text editor like nano.

Caution

Making edits to your .bashrc file can result in many issues. Some changes may prevent you from launching apps or executing commands. Modifying your PATH variable may result in the inability to use basic shell commands (such as cd or ls) if not done correctly.

Before making changes to your .bashrc file, make a backup of the default .bashrc file, so you can restore it if necessary. If you need help with editing .bashrc, reach out to rchelp@northeastern.edu or schedule a consultation with a staff member who can help suggest edits and troubleshoot any issues you might be having.

About your .bashrc file

When your account is created, you have a default .bashrc file in your home directory. See the figure below for an example of a default .bashrc file.

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# Uncomment the following line if you don't like SystemCTL's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions

Important

We recommend keeping .bashrc unchanged when using Discovery. You can source environment shell scripts or load modules directly inside your job instead. This approach can prevent some runtime errors from loading incompatible modules, setting environment variables incorrectly, or mixing multiple software and Conda environments.

Editing your .bashrc file

The basic workflow for editing your .bashrc file is to sign in to Discovery, go to your $HOME directory, open the file in a text editor on the command line, make your edits, save the file, sign out of Discovery, then sign back in again. Your changes will take effect when you sign back in again.

Example procedure for editing your .bashrc file:

  1. Sign in to Discovery.

  2. (Optional) Type pwd to ensure you are in your /home directory.

  3. (Optional) Type ls -a to view the contents of your /home directory, including hidden files. Your .bashrc file is hidden (hidden files are preceded by a . ). Using the -a option with ls displays hidden files.

  4. (Recommended) Type cp .bashrc .bashrc-default to make a copy of your .bashrc file called .bashrc-default.

  5. Type nano .bashrc to open your .bashrc file in the nano text editor.

  6. Type the edits that you want to make to your file.

  7. Save the file and exit the editor.

  8. Sign out of Discovery and sign back in for the changes to take effect.

Sourcing a Shell script example

A safe alternative to .bashrc is to source a shell script inside your runtime job environment. Below is an example script to load an Anaconda module and source a Conda environment, which will be used inside the slurm script.

Create a shell script myenv.bash:

#!/bin/bash
module load anaconda3/2021.05
module load cuda/11.1
source activate pytorch_env_training

Then, source the shell script inside your sbatch slurm script (see Batch Jobs: sbatch Command):

#!/bin/bash
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --gres=gpu:1
#SBATCH --time=01:00:00
#SBATCH --job-name=gpu_run
#SBATCH --mem=4GB
#SBATCH --ntasks=1
#SBATCH --output=myjob.%j.out
#SBATCH --error=myjob.%j.err

source myenv.bash
python <myprogram>