AKS Permission Denied
Getting "Forbidden" or "permission denied" errors when working with AKS Arc clusters in the ACX Lab? This guide covers the most common causes and fixes.
Error: "Error from server (Forbidden): pods is forbidden"
Full error example:
Error from server (Forbidden): pods is forbidden: User "user@microsoft.com"
cannot list resource "pods" in API group "" in the namespace "default"
Cause
You are trying to access a namespace you don't have permission for. Each lab user is assigned a specific namespace — you cannot access default, kube-system, or other users' namespaces.
Fix
-
Use your assigned namespace. Your namespace was provided in your registration confirmation email. It typically follows the format
user-<youralias>.```bash
Wrong — accessing default namespace
kubectl get pods
Correct — specify your namespace
kubectl -n user-youralias get pods
Or set your namespace as default for the context
kubectl config set-context --current --namespace=user-youralias ```
-
Verify your namespace exists:
bash kubectl get namespaces | grep youralias -
If your namespace doesn't appear, your RBAC may not have propagated yet. Wait 15 minutes and try again, or check your registration status.
Error: "Unauthorized" When Running kubectl
Full error example:
error: You must be logged in to the server (Unauthorized)
Cause
Your Azure AD token has expired or you're not connected to the cluster proxy.
Fix
-
Re-authenticate:
bash az login --tenant <ACX-Lab-Tenant-ID> -
Re-establish the cluster proxy:
bash az connectedk8s proxy -n <cluster-name> -g <resource-group> -
In a separate terminal, verify:
bash kubectl get nodes
Error: "Cannot create resource" in Your Namespace
Full error example:
Error from server (Forbidden): error when creating "deployment.yaml":
deployments.apps is forbidden: User "user@microsoft.com" cannot create
resource "deployments" in API group "apps" in the namespace "user-youralias"
Cause
Your RBAC role doesn't include permission for the specific resource type you're trying to create. By default, lab users get the edit ClusterRole in their namespace, which covers most common resources. However, some operations are restricted:
| Allowed | Restricted |
|---|---|
| Pods, Deployments, Services | ClusterRoles, ClusterRoleBindings |
| ConfigMaps, Secrets | Custom Resource Definitions (CRDs) |
| Jobs, CronJobs | PersistentVolumes (cluster-scoped) |
| Ingresses, NetworkPolicies | Namespaces (create/delete) |
| PersistentVolumeClaims | Node operations |
Fix
- If you need a restricted resource type, contact the ACX Lab team with your use case
- For PersistentVolumeClaims, ensure you're using an allowed StorageClass:
bash kubectl get storageclass
Error: "ResourceQuota exceeded"
Full error example:
Error from server (Forbidden): exceeded quota: user-quota,
requested: cpu=4, used: cpu=4, limited: cpu=4
Cause
Your namespace has resource quotas. Default quotas per namespace:
| Resource | Default Limit |
|---|---|
| CPU | 4 cores |
| Memory | 8 Gi |
| Pods | 20 |
| Services | 10 |
| PVCs | 5 |
Fix
-
Check your current usage:
bash kubectl -n user-youralias describe resourcequota -
Free up resources by deleting unused pods/deployments:
bash kubectl -n user-youralias delete deployment old-deployment -
If you genuinely need more quota, request an increase through the ACX Lab Power App or contact the ACX Lab team.
Arc Run Command Timeout
When using the Run command feature in the Azure portal for AKS Arc clusters:
Cause
Arc Run Command has a default timeout of 120 seconds. Long-running kubectl operations (e.g., pulling large images, waiting for pods) will time out.
Fix
- Use the
kubectl proxymethod instead for long-running operations - Break commands into smaller operations
- For image pulls, pre-pull images using a DaemonSet or init container
RBAC Propagation Delay
After registration or access changes, RBAC updates can take up to 15 minutes to propagate to the AKS Arc cluster.
Symptoms:
- Namespace exists but you get "Forbidden"
- New role bindings don't take effect immediately
Fix:
- Wait 15 minutes and retry
- Sign out of
az loginand sign back in to get a fresh token - Verify your role binding exists:
bash kubectl -n user-youralias get rolebinding