22 Dec Common SSH Key Management Mistakes and How to Avoid Them
SSH keys make remote access quick; in parallel, they last a long time. That’s great until no one remembers who created a key, where the private half lives, or why a server still trusts it. In real environments – laptops, jump boxes, CI runners, and vendor accounts – small shortcuts pile up. The result is “ghost access”: credentials that still work even though the person, system, or project is long gone.
No ownership
Public keys show up as authorized keys, but there’s no note about the owner or purpose. When you can’t answer “who uses this?” you can’t clean it up safely. Fix it by adding a short comment for each entry (name, team, ticket, expiration) and keeping a simple inventory in whatever tool your team already uses.
One private key used everywhere
Reusing the same private key across multiple machines feels easy. It also means one compromised laptop can open every door. Use separate identities by context: workstation, jump host, CI, and each external integration. If one context is exposed, you can revoke it without breaking everything else.
Private keys stored like regular files
Private keys often land in Downloads, cloud-sync folders, or old backups. That’s how they leak. Store them in a protected directory with strict file permissions, keep them out of repo history, and avoid syncing them to consumer cloud drives. For automation, store secrets in a managed vault and audit access.
Skipping passphrases because they’re annoying
An unencrypted private key is convenient for attackers, too. Use a passphrase where you can, then pair it with an SSH agent, so people aren’t typing it all day. For CI, avoid passphrases on disk; use short-lived credentials, sealed runners, or a vault-backed fetch on demand.
Stale access after role changes
People switch teams, contractors leave, and systems get decommissioned, but access lingers. Put an access review on the calendar. Every 30–90 days, compare active users and services to what’s in authorized_keys. Remove anything that no longer matches a need, and log the change.
Shared accounts and shared SSH keys
“Everyone logs in as deploy” hides accountability. When something breaks, you’re left guessing. Prefer individual accounts with distinct SSH keys, then grant elevated actions through sudo, restricted commands, or controlled automation. If you must use a service account, scope it tightly and monitor it.
Old algorithms and inconsistent standards
Teams inherit a mix of RSA, DSA, and random settings from years of “just make it work.” Standardize what you generate today and plan to replace what no longer fits. Many teams now use Ed25519 for user keys and keep host keys current. Whatever you pick, write it down and make it the default.
No rotation plan
Credentials shouldn’t live forever. Rotation is easier when you already have ownership and separation by context. Rotate on events (lost device, employee exit, incident) and on a schedule for automation. Test rotation steps before you need them at 2 a.m. A short checklist keeps you honest:
- Every authorized key has an owner, purpose, and end date.
- Private keys are protected with a passphrase or stored in a vault.
- Access is reviewed on a schedule and removed when roles change.
- Rotation is planned for both humans and automation.
Clean up the unknown keys first, stop reusing across systems, and your SSH access picture gets calmer fast.
Ignoring host key warnings
When an SSH client warns that a host key has changed, many people click through. That trains the team to ignore the one moment SSH is saying, “This might be the wrong server.” Publish fingerprints for important systems in an internal doc, and treat unexpected changes as suspicious until you confirm a rebuild or planned rotation.
If you manage access at scale, automate what you can: scan servers for authorized_keys, record last-use, and alert on new entries. Small cleanup beats the big “we’ll fix it later” project.
In its entirety
SSH keys are powerful but easy to mismanage, leading to “ghost access” where old credentials still work long after people, systems, or projects change. The main risks come from poor ownership, key reuse, weak storage and protection, and a lack of review or rotation.
With a simple checklist, periodic reviews, and some automation to scan and track keys, teams can keep SSH access understandable, auditable, and far less stressful to manage at scale.