A Comprehensive Guide to Dynamically Setting UID and GID in Docker Compose

Problem Context and Core Challenge

When working with Docker containers, controlling the execution privileges of containerized processes is often necessary, particularly for filesystem operations. With direct docker run commands, users can easily specify user and group identities using the --user parameter:

docker run --rm --user $(id -u):$(id -g) -e MYDATA=/some/path/to/data -e USER=$USER -p 8883-8887:8883-8887 ...

Here, $(id -u) and $(id -g) are executed in the shell environment, retrieving the current user's UID and GID respectively before passing them to the Docker engine. This dynamic approach works effectively for single command executions.

Configuration Limitations in Docker Compose

However, when migrating to Docker Compose configurations, developers might attempt similar syntax:

version: '3.7'
services:
  container_name: some-server
  image: some:img
  user: $(id -u):$(id -g)
  ...

This configuration will fail because Docker Compose does not execute shell command substitutions when parsing YAML files. Compose configuration files are static YAML documents where values are fixed during parsing and not dynamically computed at runtime. This is a design limitation of Compose and a common pitfall encountered by many developers.

Standard Solution: Environment Variable Passing

The correct solution involves passing UID and GID values through environment variables. First, use variable placeholders in the docker-compose.yml file:

version: '3.7'
services:
  container_name: some-server
  image: some:img
  user: "${UID}:${GID}"
  environment:
    - MYDATA=/some/path/to/data
    - USER=${USER}
  ports:
    - "8883-8887:8883-8887"

Note that the user field value uses quoted variable reference syntax "${UID}:${GID}". The quotes are essential to ensure the YAML parser correctly identifies the string value.

Environment Variable Setup Methods

There are multiple ways to provide these environment variables to Compose:

Method 1: Direct Command-Line Passing

Pass variables via environment variable prefix when starting Compose:

UID=$(id -u) GID=$(id -g) docker-compose up

This approach is most direct but requires specification with each command execution. In Bash or Zsh, $(id -u) and $(id -g) are replaced by the shell with actual numerical values before command execution.

Method 2: Pre-setting Environment Variables

Export variables in the shell session beforehand:

export UID=$(id -u)
export GID=$(id -g)
docker-compose up

Alternatively, add these export statements to shell configuration files (such as ~/.bashrc or ~/.zshrc) to set them automatically with each shell startup.

Method 3: Using .env Files

Create a .env file in the project root directory:

UID=1000
GID=1000
USER=yourusername

Docker Compose automatically loads .env files in the same directory. This method suits fixed development environments but lacks dynamism.

Technical Principle Deep Analysis

The core of this solution lies in understanding Docker Compose's variable resolution mechanism. Compose supports two types of variable substitution:

1. Environment Variable Substitution: Using ${VARIABLE} syntax to retrieve values from shell environment or .env files
2. Configuration File Internal Variables: Defining and using variables within the Compose file itself

When Compose parses "${UID}:${GID}", it searches for environment variables named UID and GID, replacing the placeholders with their values. This process occurs during Compose parsing, before container creation.

Security and Best Practices

Dynamically setting UID/GID not only solves technical problems but also provides security advantages:

• Principle of Least Privilege: Container processes run as non-root users, reducing potential security risks
• File Ownership Consistency: Files created within containers have correct ownership on the host, avoiding permission issues
• Multi-User Environment Adaptation: Different developers use their respective UID/GID values without interference

Recommended best practices include:

1. Always specify non-root users using the user field in Compose files
2. Pass UID/GID through environment variables rather than hardcoded values
3. Clearly document environment variable requirements in team projects
4. Consider using Docker's user namespace feature for enhanced isolation

Practical Application Scenario Example

Consider a web application development scenario where developers need to run a Node.js application with mounted local source code directories:

version: '3.8'
services:
  app:
    image: node:14-alpine
    user: "${UID}:${GID}"
    working_dir: /app
    volumes:
      - ./src:/app
    command: npm run dev

By setting correct UID/GID values, the Node process within the container can properly read and write mounted source files, while developers can also edit these files normally on the host machine.

Extended Discussion and Alternative Approaches

While the environment variable method is the most standard approach, other alternatives are worth understanding:

• Dynamic User Creation in Dockerfile: Creating users matching host users during image building
• Entrypoint Script Handling: Dynamically modifying user identity in container startup scripts
• Docker Compose Extension Fields: Using Compose extension features to define more complex logic

However, these approaches are typically more complex and may introduce maintenance overhead. For most use cases, the environment variable method described in this article provides the best balance.

Conclusion

Dynamically setting UID and GID in Docker Compose requires implementation through environment variable mechanisms rather than direct shell command substitution in YAML files. By configuring the user field as "${UID}:${GID}" and providing corresponding environment variables when running Compose, developers can securely and flexibly control container process permissions. This approach not only addresses technical limitations but also promotes better security practices and team collaboration.