Complete Guide to Secure Secret Management in Docker Compose v3.1

Overview of Docker Secrets Feature

Docker Compose v3.1 specification introduced native support for secrets, a critical feature for managing sensitive data in Docker Swarm mode. Compared to traditional environment variables or configuration files, secrets provide a higher level of security assurance, ensuring that sensitive information like passwords, API keys, and TLS certificates are not exposed in plain text within images or code repositories.

Environment Initialization and Swarm Mode

To utilize Docker secrets functionality, you must first ensure that the current node is part of a Swarm cluster. If not already initialized, execute the following command:

docker swarm init

This command sets the current node as a Swarm manager node, enabling cluster functionality, which is a prerequisite for using secrets.

Creating External Secrets

External secrets are stored in Docker's encrypted database and created using the docker secret create command. The following example creates a secret named my_external_secret:

echo "This is external secret content" | docker secret create my_external_secret -

Note the trailing dash - at the end of the command, which indicates reading data from standard input—a crucial detail that's easy to miss.

Creating File Secrets

File secrets allow loading sensitive data directly from local files. First, create a file containing the secret content:

echo "This is file secret content" > my_file_secret.txt

This approach is suitable when you already have sensitive data files and don't need to pipe the data.

Configuring Docker Compose File

Configuring secrets in docker-compose.yml requires two key sections: a top-level secrets block that defines all available secrets, and service-level secrets blocks that specify which secrets each service needs to access.

version: '3.1'

services:
  web:
    image: nginxdemos/hello
    secrets:
     - my_external_secret
     - my_file_secret

secrets:
  my_external_secret:
    external: true
  my_file_secret:
    file: my_file_secret.txt

In this configuration, the web service is granted access to two secrets: one external secret from the Docker secret database, and another from a local file.

Deployment and Verification

Deploy services containing secrets using Docker Stack:

docker stack deploy --compose-file=docker-compose.yml secret_test

After successful deployment, verify that secrets are correctly mounted into the container through the following steps:

docker exec -ti secret_test_web.1.m2jgacogzsiaqhgq1z0yrwekd /bin/sh
cd /run/secrets/
ls
cat my_external_secret
cat my_file_secret

Inside the container, secrets exist as read-only files in the /run/secrets/ directory, with filenames matching the secret names. This design ensures the security of sensitive data within the runtime environment.

Security Characteristics and Best Practices

Docker secrets possess several important security characteristics: First, secrets are only available to authorized containers during service runtime and are not persisted in the final image; Second, secrets are stored encrypted within the Swarm cluster and protected by TLS during transmission; Finally, secrets exist as read-only files within containers, preventing accidental modifications.

Regarding how applications access these secrets, there are two main approaches: hardcoding paths or creating symbolic links. As mentioned in the reference article, hardcoding the path /run/secrets/ is straightforward but may raise security concerns since anyone with container access can read these files. A more secure method involves having applications dynamically discover secret paths or configuring paths through environment variables.

In practical applications, it's recommended to separate secret management from application logic, avoiding direct hardcoding of file paths in code. Security can be further enhanced by dynamically obtaining secret locations through configuration management or service discovery mechanisms.