How to Build an SSH Config File for Multiple Hosts

You’re managing a handful of servers — a staging box, a production box, maybe a couple of personal Raspberry Pis — and every connection starts with typing out ssh -i ~/.ssh/id_prod_rsa -p 2222 deploy@203.0.113.42 from memory, or worse, digging through a notes file to find the right combination. It works, but it’s the kind of friction that adds up fast once you’re past two or three hosts.

The fix is a ~/.ssh/config file, and most people know it exists but have never actually sat down to write one from scratch. The syntax isn’t hard, but it’s fiddly — indentation matters, keyword casing has quirks, and a typo in IdentityFile just silently falls back to your default key instead of erroring loudly.

What an SSH config file actually does

~/.ssh/config lets you define a short alias (Host prod) that maps to the real connection details — HostName, User, Port, IdentityFile — so ssh prod does exactly what that long command did, every time, without you retyping or remembering it. It also gets picked up automatically by tools that shell out to SSH, like scp, rsync, and Git over SSH, so the same alias works everywhere.

Why people get stuck here

• Indentation and syntax are unforgiving. SSH config uses whitespace-indented keyword blocks, and a misplaced line can silently attach to the wrong Host block instead of throwing an error.
• Multiple hosts multiply the chance of a typo. One host is easy to get right by hand; six hosts with different ports, users, and key files is where mistakes creep in.
• It’s easy to forget what’s already there. Config files grow one ssh -i ... user@ip copy-paste at a time and never get cleaned up, so old, unused entries pile up alongside the ones you actually use.
• Wildcard and pattern hosts add complexity. Host *.internal blocks and per-host overrides interact in ways that aren’t obvious until you’ve been bitten by one silently overriding another.

What a good SSH config setup looks like

One alias per host, named for what it does

Host prod, Host staging, Host pi-office — short, memorable, and named for the role rather than the IP address, which changes more often than the purpose does.

Explicit identity files per host

If you use different SSH keys for different servers (a common and reasonable security practice), each host block should point at its own IdentityFile rather than relying on SSH trying every key in ~/.ssh/ until one works.

Ports and users spelled out, not assumed

Non-standard SSH ports (anything other than 22) and non-default usernames are exactly the details people forget under pressure. Having them written down in the config removes that guesswork permanently.

Common mistakes to avoid

• Editing ~/.ssh/config directly with copy-pasted blocks and introducing an indentation mismatch that silently breaks a host entry.
• Reusing one SSH key across every server without a reason, which turns a single leaked key into access to everything.
• Forgetting to set correct file permissions (chmod 600 ~/.ssh/config) — some SSH clients will refuse to use an overly permissive config file.
• Leaving old, decommissioned server entries in the file, which makes it harder to trust the config as a source of truth.
• Hardcoding an IP address for a host that’s actually on dynamic DNS or changes periodically, instead of using the hostname that stays stable.

How to do it with SSH Config Generator

Online Tool Store’s SSH Config Generator builds the file in your browser — nothing about your hosts or keys is sent anywhere.

1. Open the SSH Config Generator tool.
2. Add a host entry: alias, hostname or IP, username, port, and identity file path.
3. Repeat for each server you connect to regularly.
4. Copy the generated config block and paste it into ~/.ssh/config (create the file if it doesn’t exist).
5. Set the file’s permissions with chmod 600 ~/.ssh/config if you haven’t already, then connect with ssh <alias>.

Because it runs locally, it’s a fast way to build or extend a config without typing indentation-sensitive syntax by hand.

Frequently asked questions

Do I need to restart anything after editing the config file?

No. SSH reads ~/.ssh/config fresh on every connection, so changes apply the next time you run ssh <alias> — no daemon restart or terminal restart needed.

Can one config file cover multiple SSH keys?

Yes, and it’s a recommended practice. Each Host block can specify its own IdentityFile, so different servers can use different keys without you having to pass -i manually on every connection.

What happens if two Host blocks match the same alias?

SSH uses the first matching value it finds for each setting, reading top to bottom. If you have overlapping Host patterns (like a specific host and a Host * wildcard), put the more specific entry first so its settings take priority.

Final thought

A well-built SSH config file turns a paragraph of flags you have to remember into a single word you type. It’s a five-minute setup that pays for itself the second time you connect to any server you didn’t just create yesterday.

Try the free SSH Config Generator