How to use the Remeis cluster

How To use the Remeis Cluster

The Cluster of the Remeis observatory is designed to make all resources available for easy usage. Nevertheless, there are a few important things one has to know in order to use the cluster properly. In the following the most important information will be listed. Other questions or problems with the system or software sould be directed to admin@sternwarte.uni-erlangen.de.

Performing Computations

• You should never log in on servers directly and especially not run any computations on servers without using slurm.
• Computations and extractions on the Cluster should be submitted via slurm. Information on how to use it can be found here.
• Also interactive sessions can be started via slurm.

The Data-Storage System

• /userdata: Extracted data like images or spectra or other space consuming stuff or stuff which can re recreated/downloaded again should be put there. /userdata is a RAID6 system, i.e. two disks can fail simultaneously w/o data loss --> your data is safe on /userdata! However, there is NO backup of userdata, as /userdata is too big to be backed up simply. And there is no quota on /userdata.

• /home: This is the place for self-developed software, scripts, documents, TeX files, diploma/phd/... theses, papers, spreadsheets, etc. /home is also a RAID system, i.e. more than one disk may fail w/o data loss. Furthermore, /home is backed up every night to TWO different disks (in two machines) AND once a week on an external disk that is stored outside the observatory for the worst case (e.g., fire). It therefore does not make sense to create your own private backups on local scratch disks etc. /home, however, is quite small (5 TB for all the institute); we therefore have a quota of about 10GB per user. It is no problem to get more space, if you can't move stuff around anymore, just drop an Email to the admins. On the other hand, you don't have to fill your quota completely! To check your quota you can use the quota command on libra, like

 ssh libra quota -s -f /exports/disk1

• /scratch: For temporary or otherwise not very important data. You should not store anything important on /scratch!! The scratch disks are NOT backedup and the scratch disks are not fail safe, i.e. if a scratch disk is broken (which happens regularly), your data is GONE. scratch might be emptied at any time without warning.

• Satellite Data: We already have archives for most satellite raw data (applies mostly to the X-ray people). So please do not download any raw data yourself, but talk to Ingo.

Logging into Remeis

Remote Login

Using ssh

You can log-in via ssh simply by

 ssh -X <username>@<host>.sternwarte.uni-erlangen.de

Using x2go

For remote login with a graphical interface you can use X2Go Remote Desktop

Hosts

Host names are generally the constellation names, e.g., norma or mensa.

Connecting without password and short hostnames

Sick of always typing user@myfavouritehost.sternwarte.uni-erlangen.de and always having to enter your password when switching machines inside inside the Remeis Cluster? The follwing setup will make your life easier.

Config-File

In the file ~/.ssh/config you can set up the configuration for ssh. A few things are really useful.

  # activate compression, might lead to a faster connection
  Compression yes
  # do X11 forwarding by default, i.e., the option "-X" is 
  # always set when using "ssh"
  forwardX11 yes
  

In order to define a short-cut for a host, you have to add the following information to the ~/.ssh/config

  # the short-name you want to call the  host (most useful is the general name of the machine
  Host=norma
  # now define user and host such that <User>@<Hostname> is the correct adress
  Hostname=norma.sternwarte.uni-erlangen.de
  User=YourUsername
  

ssh-keys

If you setup the ssh-keys properly, you can login via ssh without entering a password (although you should definitely protect your ssh key with a password). A detailed description can be found here.

Setting the Default Options/Software

Each new account is equipped with a sample file at ~/.cshrc, which is loaded on every start of the terminal. Hence, this defines your options, i.e., the software available, short-cuts, your favourite editor, and so on. Commonly useful options are activated by default. In addition, there are a few additional features, which are commented out by default but still very useful for most users.

  # set your favourite text editor to be used by default; important, e.g., if you want to use GIT
  setenv EDITOR jed 
  
  # activate the X-ray software (HEADAS, satellite extraction scripts, ...)
  source /data/software/cshrc.in 

More information on the TC shell and how to configure it can be found here.