Efficient Management of Multiple SSH Private Keys on a Single Client: Configuration and Practice

Problem Background and Requirements Analysis

In modern IT environments, system administrators and developers often need to connect to multiple servers or services, each potentially requiring a different SSH private key. For instance, a single user might need to: use one key for server system administration, another for Git repository management, and a third for daily Git commits. Early approaches involved specifying keys via command-line parameters like ssh -i <key location>, but this method becomes cumbersome and error-prone with frequent use.

Core Solution: SSH Configuration Files

OpenSSH provides the ~/.ssh/config configuration file, allowing users to predefine connection parameters for different hosts or host patterns, including the private key file to use. This approach not only simplifies connection commands but also enhances security and maintainability.

Below is a typical configuration example showing how to assign dedicated keys for two different servers:

Host myshortname realname.example.com
    HostName realname.example.com
    IdentityFile ~/.ssh/realname_rsa
    User remoteusername

Host myother realname2.example.org
    HostName realname2.example.org
    IdentityFile ~/.ssh/realname2_rsa
    User remoteusername

After configuration, users can simply execute ssh myshortname or ssh myother to automatically use the corresponding private key for authentication, eliminating the need to manually specify the key path each time.

Key Generation and Configuration Steps

To implement a multi-key setup, first generate and configure the key pairs:

1. Generate Key Pairs: Use the ssh-keygen command to generate keys of different types. For example, generate RSA keys for personal and work accounts:

   ssh-keygen -t rsa -C "name@personal_email.com" -f ~/.ssh/id_rsa_personal
   ssh-keygen -t rsa -C "name@work_email.com" -f ~/.ssh/id_rsa_work

   Each command generates a private key file (e.g., id_rsa_personal) and a public key file (e.g., id_rsa_personal.pub).
2. Configure Public Keys: Add the public key content to the target server's authorized keys file (e.g., ~/.ssh/authorized_keys), or for Git services like GitHub and Bitbucket, add the SSH public key via their web interfaces.
3. Set File Permissions: Ensure private key files have strict permissions to prevent access by other users. Recommended command:

   chmod 600 ~/.ssh/id_rsa_personal
   chmod 600 ~/.ssh/id_rsa_work

   Incorrect permissions may cause SSH to ignore the key and issue a "Permissions are too open" error.

Configuration File Details

The SSH configuration file syntax is indentation-based, with each host block starting with a Host directive and supporting multiple aliases. Key directives include:

• Host: Defines host aliases or patterns, supporting wildcards. For example, Host *.example.com matches all subdomains of example.com.
• HostName: Specifies the actual hostname or IP address.
• IdentityFile: Specifies the absolute path to the private key file.
• User: Sets the login username.
• IdentitiesOnly: When set to yes, SSH uses only the keys specified in the configuration file, avoiding attempts with other default keys.

The following example configures different keys for GitHub and Bitbucket:

# Work account (Bitbucket)
Host bitbucket.org
    HostName bitbucket.org
    IdentityFile ~/.ssh/id_rsa_work
    User git
    IdentitiesOnly yes

# Personal account (GitHub)
Host github.com
    HostName github.com
    IdentityFile ~/.ssh/id_rsa_personal
    User git
    IdentitiesOnly yes

After configuration, Git clone operations automatically select the correct key: git clone git@github.com:username/repo.git uses the personal key, while git clone git@bitbucket.org:team/repo.git uses the work key.

Alternative Approaches and Comparison

Beyond host-specific configurations, SSH supports trying a sequence of keys via multiple IdentityFile directives:

IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/id_rsa_old
IdentityFile ~/.ssh/id_ed25519

During connection, SSH attempts each key in order until one is accepted by the server. This method suits scenarios with many servers where configuration maintenance is costly, but it lacks precision and may introduce security or management issues.

Compared to precise configurations, the key sequence method has these pros and cons:

• Pros: No need to configure each server individually, reducing initial setup effort.
• Cons: Key attempt order may impact connection performance; cannot ensure specific keys for specific servers, potentially violating security policies; more complex to debug connection issues.

Thus, host-based precise configurations are generally recommended for production environments.

Practical Cases and Troubleshooting

In practice, multi-key configurations apply to various scenarios:

• System Administration: Configure separate keys for different servers (e.g., production, testing) to enforce access isolation.
• Git Operations: Use distinct keys for personal and work Git accounts to prevent permission conflicts.
• Tool Integration: Some tools (e.g., Tabby terminal) can read ~/.ssh/config to automatically apply key configurations, improving user experience.

Common issues and solutions:

• Connection Failures: Check ssh -v output to verify key paths and permissions.
• Permission Errors: Use chmod 600 to fix private key file permissions.
• Configuration Not Applied: Ensure correct file syntax without indentation errors, and restart the SSH session.

Conclusion

Managing multiple private keys via SSH configuration files is an efficient and secure method that significantly simplifies daily operations. The steps and examples provided in this article cover the entire process from key generation to integration, applicable across various technical contexts. Users should choose between precise configurations and key sequences based on actual needs, and regularly audit key usage to maintain system security.