How to Harden SSH Without Locking Yourself Out

Everyone who has hardened an SSH server has a story about the time they didn’t. Disable password authentication, reload sshd, close the terminal, and discover that the key you thought was installed is on a laptop at home.

The directives themselves are well documented and uncontroversial. The part that goes wrong is the order you apply them in.

The order that keeps you safe

There’s one rule that prevents almost every lockout: keep an existing session open until you’ve proved the new configuration works in a new one.

Around it, the sequence goes:

1. Install your public key and verify key login works — in a second terminal, while the first stays connected.
2. Set PermitRootLogin no.
3. Restrict who can log in with AllowUsers or AllowGroups.
4. Only then set PasswordAuthentication no.
5. Reload sshd — don’t restart it — and open a third session to confirm.

Password authentication goes last because it’s the fallback. Remove it before key login is proven and you’ve removed the thing that would have saved you.

Why people get stuck here

• Editing and reloading in one step. No verification between changes, so a mistake isn’t found until the session drops.
• Assuming the key is installed. ssh-copy-id failing quietly, or copying to the wrong user’s authorized_keys.
• Permissions on the home directory. sshd refuses keys if ~/.ssh or authorized_keys are group-writable, and the failure message on the client side is unhelpfully generic.
• Cloud console false confidence. A provider’s web console is a genuine backup — but only if you’ve tested that it works before you need it.

What’s worth doing and what isn’t

Worth doing

Key-only authentication is the single change that matters most; it eliminates brute-force password attacks entirely. Disabling root login means an attacker needs a username as well as a key. Restricting to specific users or a group limits the surface further. Keeping the server patched matters more than any config line.

Mostly theatre

Changing the SSH port reduces log noise from opportunistic scanners. It does not stop anyone targeting you specifically — a port scan finds it in seconds. Do it for tidiness, not protection.

Rate-limiting tools like fail2ban primarily reduce log volume once password authentication is off. The attacks they block can’t succeed anyway.

Common mistakes to avoid

• Closing your only session before testing the new configuration.
• Using systemctl restart sshd when reload would apply the config without dropping connections.
• Setting AllowUsers and forgetting the deployment or backup account that also needs access.
• Leaving ~/.ssh group-writable, so key authentication silently refuses.
• Disabling password authentication on a machine whose console access you’ve never actually tested.

How to do it with SSH Hardening Checklist

The SSH Hardening Checklist orders the steps so the risky one comes last.

1. Choose the server type — a public host needs more than an internal one.
2. Confirm key login works before changing anything else.
3. Apply the directives in the listed order, reloading after each group.
4. Verify from a new connection each time, keeping an existing session open throughout.
5. Test your out-of-band access — cloud console or serial — before you need it.

The OpenSSH sshd_config manual is the authority on every directive. Other server security tools are in the tools directory.

Frequently asked questions

What’s the safest order to apply these?

Prove key login works, keep an existing session open, and disable password authentication last. Almost every lockout comes from reversing those steps.

Should I change the SSH port?

It reduces automated scan noise in your logs and provides no real protection against anyone targeting you. Reasonable for tidiness; not a security measure.

Is fail2ban still worth running?

With password authentication disabled, it mostly saves log volume — the attempts it blocks couldn’t succeed. It’s more valuable on services that still accept passwords.

Final thought

The config is easy; the sequence is what matters. Never close the session that still works until a new one has proved the change was safe.

Try the free SSH Hardening Checklist