I have a Kubernetes cluster running on AWS and I exposes it

Question

I have a Kubernetes cluster running on AWS and I exposes it using an ingress If I run `helm install cerbos cerbos cerbos version=0 26 0` the installation goes through and I m able to access the API documentation on the endpoint in the browser To be able to test policy requests I want to serve policies from a public S3 bucket and added a config file as described here <https docs cerbos dev cerbos latest installation helm html> but with the following configuration ```cerbos config server httpListenAddr 3592 grpcListenAddr 3593 adminAPI adminCredentials passwordHash lt password hash gt username cerbos enabled true log level info storage driver s3 blob bucket s3 lt bucket name gt region=eu west 1 updatePollInterval 10s downloadTimeout 30s``` I tried the following `helm upgrade cerbos cerbos cerbos version=0 26 0 values=config yaml` gt This does nothing It states the upgrade is successful but nothing changes and Cerbos does not seem to read the policy The API documentation is still available `helm uninstall cerbos` `helm install cerbos cerbos cerbos version=0 26 0 values=config yaml` gt This breaks the system and the ingress returns 503 when trying to access the API documentation Rerunning `helm uninstall cerbos` `helm install cerbos cerbos cerbos version=0 26 0` restores the API access A couple of things How can I upgrade an existing cerbos service using helm in the cluster and if so what is the process for this How can I read access the deployment logs if something does not goes well What could be the issue here Am I missing some required configurations

Charith (Cerbos) · Answer

Hi The way to upgrade an existing Cerbos installation is to use `helm upgrade` You can find the logs using `kubectl logs f svc cerbos n YOUR NAMESPACE` and inspect the deployment using `kubectl describe deployment cerbos n YOUR NAMESPACE`

Charith (Cerbos) · Answer

My hunch is that your pods don t have access to the S3 bucket When you upgraded nothing happened because the old deployment was still the primary because the new one was failing When you uninstalled and re installed there s no fallback so the whole installation is in failed state You can check that with `helm history cerbos n YOUR NAMESPACE`

Charith (Cerbos) · Answer

You either need to add IAM roles to your k8s nodes to give them access to the s3 bucket OR add a key token with access to the bucket as a environment variable to the Cerbos pod

David Nilsdotter · Answer

Ah okay So setting the environment variables `AWS ACCESS KEY ID` and `AWS SECRET ACCESS KEY` it probably will solve the issue

Charith (Cerbos) · Answer

Yes Since you re already on EKS it might be possible to use IAM roles as well Unfortunately I am not an expert on AWS so I can t help you much with that slightly smiling face

David Nilsdotter · Answer

You helped quite a lot It is working now +1

David Nilsdotter · Answer

But as a follow up with the defined setup the request towards ` admin policy` returns ``` code 12 message Admin service is disabled by the configuration ```

Charith (Cerbos) · Answer

Is it using the same configuration you pasted earlier

Charith (Cerbos) · Answer

What s the output of `kubectl get cm cerbos o yaml n YOUR NAMESPACE`

David Nilsdotter · Answer

Sorry Got tapped on the shoulder slightly smiling face Went in again and ran `uninstall` and `install` again and checked the logs Then I was able to identify the issues There were a couple actually `log` is not a valid property for server `s3` is not a valid driver `missing authentication` without AWS credentials

Charith (Cerbos) · Answer

Oh YAML spacing issues slightly smiling face I should have noticed that The correct driver is `blob`

David Nilsdotter · Answer

Yeah now it works +1

David Nilsdotter · Answer

Got a better response or the admin policy request aswell slightly smiling face

David Nilsdotter · Answer

Thanks for the help +1

Charith (Cerbos) · Answer

Glad to help