SSH Mastery — Keys, Tunnels, Config and Security

Still typing passwords every time you SSH into a server? Still memorizing IP addresses? Still copying files with scp one server at a time? SSH is the most used tool in a DevOps engineer's arsenal, yet most people only use 10% of its power. Let's change that.

Key-Based Authentication — No More Passwords

Password authentication is slow, insecure (brute-force attacks), and doesn't scale. SSH keys solve all three problems.

# Generate a modern SSH key pair
ssh-keygen -t ed25519 -C "vivek@goelacademy.com"
# Generates:
#   ~/.ssh/id_ed25519       (private key — NEVER share this)
#   ~/.ssh/id_ed25519.pub   (public key — copy to servers)

# For older systems that don't support Ed25519
ssh-keygen -t rsa -b 4096 -C "vivek@goelacademy.com"

# Copy your public key to a remote server
ssh-copy-id user@server.example.com
# This adds your public key to ~/.ssh/authorized_keys on the server

# Now SSH without a password
ssh user@server.example.com

ssh-agent — Don't Re-enter Your Passphrase

If your key has a passphrase (it should!), ssh-agent remembers it so you only type it once per session.

# Start the agent
eval "$(ssh-agent -s)"
# Agent pid 12345

# Add your key (prompts for passphrase once)
ssh-add ~/.ssh/id_ed25519

# List loaded keys
ssh-add -l

# Add to your ~/.bashrc to auto-start
cat >> ~/.bashrc << 'EOF'
# Auto-start ssh-agent
if [ -z "$SSH_AUTH_SOCK" ]; then
  eval "$(ssh-agent -s)" > /dev/null
  ssh-add ~/.ssh/id_ed25519 2>/dev/null
fi
EOF

The SSH Config File — Your Productivity Secret Weapon

Instead of typing ssh -i ~/.ssh/deploy_key -p 2222 deploy@10.0.1.50 every time, define it once in ~/.ssh/config:

# Create or edit ~/.ssh/config
cat > ~/.ssh/config << 'EOF'
# Default settings for all connections
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes

# Production web server
Host prod-web
    HostName 10.0.1.50
    User deploy
    Port 2222
    IdentityFile ~/.ssh/deploy_key

# Staging server
Host staging
    HostName staging.example.com
    User deploy
    IdentityFile ~/.ssh/deploy_key

# Database server (only accessible via jump host)
Host prod-db
    HostName 10.0.2.100
    User dbadmin
    ProxyJump prod-web

# All AWS instances
Host aws-*
    User ec2-user
    IdentityFile ~/.ssh/aws-key.pem

# Specific AWS instance
Host aws-app1
    HostName 54.123.45.67

Host aws-app2
    HostName 54.123.45.68

# GitHub (use SSH key for git operations)
Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_key
EOF

# Set correct permissions
chmod 600 ~/.ssh/config

Now instead of long commands, just type:

ssh prod-web         # Connects to 10.0.1.50:2222 as deploy
ssh staging          # Connects to staging.example.com as deploy
ssh prod-db          # Jumps through prod-web automatically
ssh aws-app1         # Connects with EC2 key

Port Forwarding — SSH Tunnels

SSH tunnels let you securely access remote services as if they were local. This is how you connect to databases, admin panels, and internal tools behind firewalls.

Local Port Forwarding (Most Common)

"Make a remote service available on my local machine."

# Access remote PostgreSQL (port 5432) on localhost:5432
ssh -L 5432:localhost:5432 prod-db

# Now connect your database client to localhost:5432
psql -h localhost -U myuser -d production

# Access a remote web UI on localhost:8080
ssh -L 8080:localhost:3000 prod-web
# Open http://localhost:8080 in your browser

# Access a service on a different host through the SSH server
# Your machine → SSH server → database server
ssh -L 5432:db-server.internal:5432 prod-web

# Run in background (no interactive shell)
ssh -fNL 5432:localhost:5432 prod-db
# -f = background, -N = no command, -L = local forward

Remote Port Forwarding

"Make my local service available on the remote server."

# Expose your local dev server (port 3000) on the remote server's port 8080
ssh -R 8080:localhost:3000 prod-web
# Someone on the remote network can now access your dev server

Dynamic Port Forwarding (SOCKS Proxy)

"Route all traffic through the SSH server."

# Create a SOCKS proxy on port 1080
ssh -D 1080 prod-web

# Configure your browser to use SOCKS proxy: localhost:1080
# All browser traffic now goes through the SSH tunnel

Jump Hosts — Bastion Servers

In production environments, servers aren't directly accessible from the internet. You go through a jump host (bastion).

# The old way — SSH twice
ssh bastion.example.com
ssh 10.0.2.100

# The better way — ProxyJump
ssh -J bastion.example.com 10.0.2.100

# Multiple jumps
ssh -J bastion1,bastion2 final-server

# Best way — define in ~/.ssh/config (shown earlier)
# Host prod-db
#     ProxyJump prod-web

# Then just:
ssh prod-db

File Transfer — scp and rsync

scp — Simple Copies

# Copy file to remote
scp deploy.tar.gz prod-web:/opt/releases/

# Copy file from remote
scp prod-web:/var/log/app.log ./

# Copy directory recursively
scp -r ./config/ prod-web:/opt/myapp/config/

# Copy through jump host (uses your SSH config)
scp database.dump prod-db:/backup/

rsync — Smart Sync (Always Prefer This)

rsync is better than scp in every way: it only transfers changed bytes, supports compression, can resume interrupted transfers, and preserves permissions.

# Sync a directory to remote (trailing slash matters!)
rsync -avz ./dist/ prod-web:/var/www/html/

# Flags explained:
# -a = archive (preserves permissions, timestamps, symlinks)
# -v = verbose
# -z = compress during transfer

# Dry run first — see what would change
rsync -avzn ./dist/ prod-web:/var/www/html/

# Sync with deletion (remote matches local exactly)
rsync -avz --delete ./dist/ prod-web:/var/www/html/

# Exclude files
rsync -avz --exclude='node_modules' --exclude='.git' \
  ./project/ prod-web:/opt/myapp/

# Bandwidth limit (useful for production)
rsync -avz --bwlimit=5000 ./backup.tar.gz prod-web:/backup/
# 5000 KB/s limit

# Resume an interrupted transfer
rsync -avz --partial --progress large-file.tar.gz prod-web:/backup/

Hardening sshd_config — Securing Your SSH Server

A default SSH installation is a target for bots. Here's how to lock it down.

# Edit the SSH server configuration
sudo vim /etc/ssh/sshd_config

Apply these changes:

# /etc/ssh/sshd_config — Hardened configuration

# Change the default port (security through obscurity, but reduces bot noise)
Port 2222

# Disable root login — always
PermitRootLogin no

# Disable password authentication — keys only
PasswordAuthentication no
PubkeyAuthentication yes

# Disable empty passwords
PermitEmptyPasswords no

# Limit to specific users or groups
AllowUsers deploy vivek
# or
AllowGroups ssh-users

# Disable X11 forwarding (not needed on servers)
X11Forwarding no

# Set idle timeout (disconnect inactive sessions)
ClientAliveInterval 300
ClientAliveCountMax 2

# Limit max authentication attempts
MaxAuthTries 3

# Disable SSH protocol 1 (ancient, insecure)
Protocol 2

# Use strong key exchange algorithms only
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Test config before restarting (don't lock yourself out!)
sudo sshd -t
# If no errors:
sudo systemctl restart sshd

# CRITICAL: Keep your current session open while testing!
# Open a NEW terminal and verify you can still connect
ssh prod-web

Additional Security with fail2ban

# Install fail2ban
sudo apt install fail2ban

# Create a local config (survives updates)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Configure SSH protection
cat << 'EOF' | sudo tee /etc/fail2ban/jail.d/sshd.local
[sshd]
enabled = true
port = 2222
filter = sshd
maxretry = 3
bantime = 3600
findtime = 600
EOF

# Start fail2ban
sudo systemctl enable --now fail2ban

# Check banned IPs
sudo fail2ban-client status sshd

# Unban an IP
sudo fail2ban-client set sshd unbanip 203.0.113.50

SSH Troubleshooting

When SSH doesn't work, debug it systematically:

# Verbose connection (shows what's happening step by step)
ssh -v prod-web
# Add more v's for more detail
ssh -vvv prod-web

# Common issues and fixes:

# Permission denied (publickey)
# Check 1: Key file permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/config

# Check 2: Key is on the server
ssh-copy-id user@server

# Check 3: Correct key is being offered
ssh -v prod-web 2>&1 | grep "Offering"

# Connection refused
# Check: Is sshd running on the server?
sudo systemctl status sshd

# Connection timed out
# Check: Firewall, security groups, network connectivity
nc -zv server.example.com 2222

Quick Reference

Task	Command
Generate key	ssh-keygen -t ed25519
Copy key to server	ssh-copy-id user@host
Local port forward	ssh -L local:remote:port host
Remote port forward	ssh -R remote:local:port host
SOCKS proxy	ssh -D 1080 host
Jump host	ssh -J bastion target
Copy file to remote	scp file host:/path/
Sync directory	rsync -avz dir/ host:/path/
Background tunnel	ssh -fNL port:host:port server
Debug connection	ssh -vvv host

---

That wraps up our Linux networking and administration series! These SSH skills are the foundation for everything in DevOps — from deployments to Ansible to CI/CD pipelines. Practice them daily, and they'll become second nature.