Configuring Multi-Repository Access in GitLab CI: A Comprehensive Guide to Deploy Keys

Challenges and Solution Overview for Multi-Repository Builds

In modern software development, projects are typically composed of multiple independent code repositories containing different modules, libraries, or services. When building the entire project in a continuous integration (CI) environment, the build system must access code from all relevant repositories. However, when these repositories are set as private, automated access faces authentication challenges. Traditional username-password approaches are unsuitable for automation scenarios, while GitLab offers two main solutions: the deploy keys method and the CI_JOB_TOKEN method.

Core Principles of the Deploy Keys Method

Deploy keys are SSH key pairs specifically designed for automation scenarios, allowing specific machines or services to access Git repositories without using personal account credentials. In the GitLab CI environment, this method works as follows: first, generate an SSH key pair (typically using RSA or Ed25519 algorithms); then, add the public key to each GitLab project that requires access as a deploy key; finally, securely configure the private key on the GitLab Runner executing CI jobs.

The configuration process can be divided into several key steps:

1. Generate an SSH key pair on the GitLab Runner server using the command ssh-keygen -t ed25519 -C "gitlab-runner@example.com"
2. Add the generated public key (typically located at ~/.ssh/id_ed25519.pub) to each GitLab project requiring access via the "Deploy Keys" page in project settings
3. Ensure the private key file (~/.ssh/id_ed25519) has permissions set to 600 (readable and writable only by the owner)
4. Configure the SSH agent in GitLab Runner configuration or CI scripts to ensure git commands automatically use the private key

Detailed Configuration Steps and Practical Examples

The following is a complete configuration example demonstrating how to implement multi-repository cloning in a .gitlab-ci.yml file:

before_script:
  # Set up SSH configuration
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  # Write private key content to file (in practice, read from secure variables)
  - echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_ed25519
  - chmod 600 ~/.ssh/id_ed25519
  # Configure SSH host verification
  - ssh-keyscan gitlab.example.com >> ~/.ssh/known_hosts

build:
  script:
    # Clone main repository (current project)
    - git clone git@gitlab.example.com:group/main-project.git
    # Clone dependency repositories
    - git clone git@gitlab.example.com:group/dependency1.git
    - git clone git@gitlab.example.com:group/dependency2.git
    # Execute build commands
    - cd main-project && make all

In actual deployment, $SSH_PRIVATE_KEY should be stored as a GitLab CI/CD variable, marked as protected and masked to ensure security. For multi-Runner environments, you can choose to generate independent key pairs for each Runner or configure all Runners with the same key pair—the former provides better security isolation, while the latter simplifies management.

Supplementary Explanation of the CI_JOB_TOKEN Method

As a complement to the deploy keys method, GitLab version 8.12 and above provides the CI_JOB_TOKEN environment variable. This method is suitable for scenarios requiring temporary access tokens, particularly when cross-project triggering is supported in GitLab Premium. Example usage:

clone_repos:
  script:
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/group/project.git

Compared to the deploy keys method, the CI_JOB_TOKEN method has the following characteristics: based on HTTP(S) protocol rather than SSH, tokens are automatically generated and valid only during the current job, eliminating long-term key management. However, it may be limited by GitLab version and license level, and may be less flexible than SSH in certain network configurations.

Security Considerations and Best Practices

Regardless of the chosen method, secure configuration is crucial. For the deploy keys method, it is recommended to:

• Generate key pairs specifically for the CI environment—do not reuse personal development keys
• Regularly rotate keys (recommended every 6-12 months)
• Restrict deploy key access permissions to "read-only" in GitLab projects unless necessary
• Use the Ed25519 algorithm for key generation as it is more secure and efficient than traditional RSA
• Manage private keys via GitLab CI/CD variables to avoid hardcoding in configuration files

For large projects, consider the following optimizations: use Git submodules to manage dependencies, pre-configure SSH keys in Docker images to reduce job startup time, or dynamically manage deploy keys via the GitLab API.

Method Comparison and Selection Recommendations

The main advantages of the deploy keys method are its universal applicability (supporting all GitLab versions), efficiency based on the SSH protocol, and good adaptability to complex network environments. It is particularly suitable for scenarios requiring long-term stable access to multiple repositories and teams already using SSH for code management.

The CI_JOB_TOKEN method is more suitable for temporary access needs or teams wishing to reduce long-term key management burdens. In GitLab Premium environments, it can also integrate better with pipeline triggering features.

In actual projects, both methods can be combined: use deploy keys to access core dependency repositories while using CI_JOB_TOKEN for temporary or auxiliary repositories. The key is to make appropriate choices based on the project's specific requirements, security policies, and infrastructure environment.