Working remotly

Working Remotely#

When using HPC resources, you rarely sit directly in front of the machine. Instead, you connect remotely to the cluster from your own computer. There are few important commands to know when working remotly.

SSH#

SSH (Secure Shell) is the most common way to log in securely to a remote machine.

ssh username@remote:/path/to/destination

username : your loging name on the remote machine
remote : the address to reach the remote machine, which can be a hostename like bioinfo-master1.ird.fr or an IP address like 192.168.1.1.

✅ Once connected, you will be in a Linux shell on the cluster. You can then navigate the filesystem, load modules, or submit jobs.

Tip

If you lose connection often, you can use:

ssh -Y -o ServerAliveInterval=60 username@remote

to keep the session alive.

Transfer data#

Think of data transfer as the bridge between your laptop and the remote machine: once you know how to cross it smoothly, you’ll always have your code, data, and results exactly where you need them. The remote machine become an extension of your local machine.

SCP#

SCP (Secure Copy) is used to copy files/folder between your local machine and the remote one. It is very simple, easy to use but ineficient for large dataset.

# Copy a file from local to remote
scp mydata.txt username@remote:/home/username/

# Copy a file from remote to local
scp username@remote:/home/username/results.txt ./results.txt

rsync#

Rsync is better than SCP for large datasets because it only transfers the differences between local and remote files/folder. It can even resume interrupted transfer. In counterpart it has a more complex options.

# Sync local project folder to the cluster
rsync -av myproject/ username@remote:/home/username/myproject/

# Sync results from the cluster back to local
rsync -av username@remote:/home/username/results/ ./results/

Common option	explanation
-a	archive mode (keeps permissions, timestamps, etc.)
-v	verbose (shows what’s happening)
-z	Compresses data during transfer → faster for large text files over slow networks.
-L	rsync follows the symlink and copies the file it points to instead.
--partial	Allows interrupted transfers to resume instead of restarting.
--progress	Shows progress for each file. Useful for large transfers.
--delete	Deletes files on the destination if they no longer exist in the source, keeps exact mirror. ⚠️ Be careful with this one!