Resolving kubectl Unauthorized Errors When Accessing Amazon EKS Clusters

Problem Background and Error Analysis

During the initial configuration of Amazon EKS clusters, many users encounter the error: You must be logged in to the server (Unauthorized) when using kubectl. The core cause of this error lies in EKS's RBAC authorization mechanism restrictions—only the IAM entity (user or role) that created the cluster is automatically granted administrator permissions to the Kubernetes API server.

From a technical implementation perspective, when an EKS cluster is created, AWS automatically adds the creator IAM entity's mapping to Kubernetes' RBAC authorization table. This means that other IAM users, even with correct AWS credentials and kubeconfig configuration, cannot pass Kubernetes API server authentication because they are not included in the cluster's authorization list.

Core Solution: Configuring aws-auth ConfigMap

To resolve the unauthorized error, access permissions for other IAM users must be added by editing the aws-auth ConfigMap. This ConfigMap resides in the kube-system namespace and manages the mapping between IAM entities and Kubernetes users.

First, use the cluster creator's identity to execute the following command to edit the ConfigMap:

kubectl edit -n kube-system configmap/aws-auth

In the editor, configure three key mapping sections:

apiVersion: v1
data:
  mapRoles: |
    - rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-NodeInstanceRole-74RF4UBDUKL6
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
  mapUsers: |
    - userarn: arn:aws:iam::111122223333:user/ops-user
      username: ops-user
      groups:
        - system:masters
  mapAccounts: |
    - "111122223333"

The mapUsers section is used to add specific IAM users, where userarn specifies the user's ARN, username defines the username used in Kubernetes, and groups specifies the Kubernetes groups the user belongs to. mapAccounts is used to map entire AWS accounts.

RBAC Permission Binding

Simply adding mappings in the ConfigMap is insufficient; specific permissions must be granted through RBAC RoleBinding or ClusterRoleBinding. For example, to grant cluster administrator permissions to the ops-user user:

kubectl create clusterrolebinding ops-user-cluster-admin-binding --clusterrole=cluster-admin --user=ops-user

This command creates a ClusterRoleBinding that binds the cluster-admin ClusterRole to the ops-user user, giving them administrator permissions across the entire cluster.

Access Configuration for Different Creation Scenarios

Access configuration varies significantly based on the type of IAM entity used to create the cluster:

Scenario: Cluster Created by IAM User

When the cluster is created by an IAM user, configuration is relatively straightforward. First, verify the identity currently used by AWS CLI:

aws sts get-caller-identity
{
"Account": "xxxxxxxxxxxx",
"UserId": "xxxxxxxxxxxxxxxxxxxxx",
"Arn": "arn:aws:iam::xxxxxxxxxxx:user/eks-user"
}

Then use the AWS EKS command to update kubeconfig:

aws eks --region region-code update-kubeconfig --name cluster_name

Scenario: Cluster Created by IAM Role

When the cluster is created by an IAM role, configuration is more complex, with four main methods:

Method 1: Direct Role Configuration in kubeconfig

First ensure the target user has permission to assume the role, then specify the role ARN when updating kubeconfig:

aws eks --region region-code update-kubeconfig --name cluster_name --role-arn arn:aws:iam::xxxxxxxxxxx:role/eks-role

Method 2: Using AWS Configuration Profiles

Configure AWS CLI configuration profiles, then reference the profile in kubeconfig:

aws eks update-kubeconfig --name devel --profile eks

This adds environment variables to the exec configuration in kubeconfig:

env:
- name: AWS_PROFILE
  value: eks

Method 3: Through EC2 Instance Profiles

When an IAM role is directly attached to an EC2 instance, the instance automatically obtains the role's credentials, allowing direct use of standard kubeconfig update commands.

Method 4: Manual Role Assumption

Manually assume the role via aws sts assume-role command, then set environment variables:

export AWS_ACCESS_KEY_ID=xxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxx
export AWS_SESSION_TOKEN=xxxxxxxxxx

Troubleshooting and Verification

After configuration completion, verify access permissions through the following steps:

First verify AWS identity:

aws sts get-caller-identity

Then test Kubernetes access:

kubectl get services

If unauthorized errors persist, check:

• ARN format correctness in ConfigMap
• Successful creation of RBAC bindings
• Validity of AWS credentials and EKS cluster access permissions
• Correctness of cluster endpoints in kubeconfig file

Technical Principle Deep Dive

EKS's authentication mechanism is based on aws-iam-authenticator, which converts AWS IAM credentials into Kubernetes-recognizable identity information. When kubectl initiates API requests, aws-iam-authenticator:

1. Retrieves current IAM entity identity information from AWS environment
2. Generates JWT tokens containing identity claims
3. Attaches tokens to Kubernetes API requests
4. API server validates tokens and checks RBAC authorization

Configuration errors at any stage of this process can cause unauthorized errors. Particularly important is that when using IAM roles, the ARN format changes after role assumption, which may affect mapping matches in ConfigMap.

Best Practice Recommendations

Based on practical deployment experience, the following best practices are recommended:

• Use dedicated IAM roles rather than personal users to create EKS clusters
• Manage aws-auth ConfigMap through infrastructure-as-code tools
• Follow principle of least privilege, granting administrator permissions only to necessary users
• Regularly audit cluster RBAC configurations and IAM mappings
• Use AWS Organizations and SCP for centralized EKS access policy management

By systematically understanding and applying these configuration methods, unauthorized issues in EKS cluster access can be effectively resolved, establishing a secure and reliable Kubernetes access management system.