How to solve kubernetes dashboard – Unauthorized (401): Invalid credentials provided

Problem

Yesterday, i was installing kubernetes dashboard v7.9.0 on EKS v1.31 on AWS cloud. I followed all steps from here Deploy and Access the Kubernetes Dashboard. The steps are:

• Installing kubernetes dashboard using helm
• Accessing kubernetes dashboard UI by using kubectl proxy
• Create service account to get token for login

All of the above steps i have followed. I successfully created token and paste its token to Bearer token field (see above image), URL:

http://<your ip>:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard-kong-proxy:443/proxy/#/login

After clicking Sign in and you know what? I got this error message and get failed login to dashboard.

Unauthorized (401): Invalid credentials provided

Yes, i think i have incorrect token here, maybe i missed some token words? so i create another new token:

kubectl create token my-user-admin -n kubernetes-dashboard 

I then paste its new token to Bearer token field again, click Sign in and the result is exactly the same as before, Unauthorized (401): Invalid credentials provided. What’s wrong?

The cause

After surfing on the internet to find the cause, i got this important discussion on github: unable to login with “kubectl proxy” way #8767. Yes, after reading that, finally i know what the cause. The cause is related to kubectl proxy perfoms to strip authorization header, it causes the token will be cut and you will get failed login. Hmm?

Solutions

There are 2 solutions to solve this issue:

• Migrating from kubectl proxy to kubectl port-forward
• Downgrade kubernetes dashboard version and keep using kubectl proxy

Solution 1: Migrating from kubectl proxy to kubectl port-forward

The first solution is migrating from kubectl proxy to kubectl port-forward. Yes! you need migrating to port-forward, because solution for fixing kubectl proxy on kubernetes dashboard v.7.9.0 is none for now, As Far As I Know. CMIIW.

Here is how i migrate from kubectl proxy to port-porward:

This is my current kubectl proxy configuration:

kubectl proxy --address=0.0.0.0 --port=8001 --accept-hosts=^*$ --disable-filter=true

And we can convert to kubectl port-forward, like this:

kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8001:443 --address=0.0.0.0

open dashboard on browser (make sure you run with https)

https://<your ip>:8001

As you see, https is not valid. But don’t worry just ignore it.

Now, create new token for login:

kubectl create token my-user-admin -n kubernetes-dashboard 

Copy token to Bearer token field, and click Sign in

If login succeeded, you will meet the dashboard:

Solution 2: Downgrade kubernetes dashboard version and keep using kubectl proxy

The second solution is to downgrade the version and if you want to keep kubectl proxy running, try this solution.

As i mentioned in early, i installed kubernetes dashboard using helm, which is using the latest version (v7.9.0) at that time.

So first, we have to remove existing dashboard we previously installed using helm. I installed at kuberentes-dashboard namespace.

helm delete kubernetes-dashboard --namespace kubernetes-dashboard

Then, install version 2.7.0 using kubectl

kubectl create namespace kubernetes-dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml --namespace kubernetes-dashboard

Make sure all pods in kubernetes-dashboard namespace are already running.

kubectl get pods -n kubernetes-dashboard

To access dashboard on browser, you have to:

• enabling kubectl proxy
• enabling reverse proxy (nginx) and using trusted HTTPS

enabling kubectl proxy

This is my kubectl proxy configuration:

kubectl proxy --address=0.0.0.0 --port=8001 --accept-hosts=^*$ --disable-filter=true

enabling reverse proxy (nginx) and using trusted HTTPS

To access dashboard from kubectl proxy, you have to use reverse proxy and trusted HTTPS. I am using nginx as reverse proxy here and below is my nginx configuration on sites-available:

Explained:

• i use domain eks-dev.x.com to access kubernetes dashboard
• eks-dev.x.com using HTTPS/SSL certificate from Let’s Encrypt
• nginx proxy will be redirected to internal kubernetes dashboard URL, which is:

http://18.x.x.y.y:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:443/proxy/;

If i open https://eks-dev.x.com on browser, it will show login page:

Now, create serviceaccount to get token for login:

eks-admin-service-account.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: eks-admin
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: eks-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: eks-admin
  namespace: kubernetes-dashboard

Apply to create serviceaccount:

kubectl apply -f eks-admin-service-account.yaml

Create token login:

kubectl create token eks-admin -n kubernetes-dashboard

Place the token to login page, and click Sign in

If succeed, you will get the dashboard:

Conclusion

• You get Unauthorized (401): Invalid credentials provided because of using kubectl proxy that strips authentication header (token).
• There are 2 solutions:
  • Migrate from kubectl proxy to kubectl port-forward
  • Or downgrading kubernetes dashboard version to v.2.7.0 (keep using kubectl proxy)