https://cerbos.dev logo
Join Slack
Powered by
hey all - thank you for your help so far - I was a...
# help
d
hey all - thank you for your help so far - I was able to follow the helm deployment instructions here - https://docs.cerbos.dev/cerbos/latest/installation/helm.html everything makes sense -2 followup question in the thread
I can grab the logs and see that my policies were not grabbed from git
Copy code
% kubectl logs svc/dev-cerbos -n dev-namespae 
{"log.level":"info","@timestamp":"2023-06-02T19:32:35.742Z","log.logger":"cerbos.server","message":"maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined"}
{"log.level":"info","@timestamp":"2023-06-02T19:32:35.743Z","log.logger":"cerbos.server","message":"Loading configuration from /config/config.yaml"}
{"log.level":"info","@timestamp":"2023-06-02T19:32:35.744Z","log.logger":"cerbos.disk.store","message":"Initializing disk store from /work"}
{"log.level":"info","@timestamp":"2023-06-02T19:32:35.744Z","log.logger":"cerbos.index","message":"Found 0 executable policies"}
{"log.level":"info","@timestamp":"2023-06-02T19:32:35.744Z","log.logger":"cerbos.telemetry","message":"Anonymous telemetry enabled. Disable via the config file or by setting the CERBOS_NO_TELEMETRY=1 environment variable"}
Im assuming this based on the 
"message":"Found 0 executable policies"
msg
normally at this point I would exec directly onto the container and start verifying that things work “manually” - in this case trying to verity that the github token is present as an env and I can use it to reach my git repo (where policy is stored)
but I don’t think this is an option in this case by design (at least not w/ the default cerbos image)
Copy code
% kubectl exec -it dev-cerbos-9f8fc69c9-crrnz -n dev-namespace /bin/sh  
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
error: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "3d84c7966505518cff7f35baa923dd52059bfff0715a417d809de60364fef1fd": OCI runtime exec failed: exec failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown
so the above is context for question #1 “how to debug k8 deployed cerbos?”
question #2 - why is my /schema/swagger.json missing 🤔 - I have an ingress in front of cerbo - hence the 
/cerbos
prefix that I strip w/ traefik middleware before the request hits the service
thank you in advance
really only question #1
o
Hey!
Copy code
{
  "log.level": "info",
  "@timestamp": "2023-06-02T19:32:35.744Z",
  "log.logger": "cerbos.disk.store",
  "message": "Initializing disk store from /work"
}
I see the 
log.logger
field set as 
cerbos.disk.store
in the above log which means either the configuration is not right, or cerbos is running with the default configuration (default configuration sets the store as 
disk
). If the configuration file you’ve created has 
git
as a store, you could ensure it is provided to cerbos container correctly.
d
ok here is my config
Copy code
cerbos:
  service:
    type: ClusterIP
    httpPort: 3592
    grpcPort: 3593
    httpNodePort: 13592
    grpcNodePort: 13593
  envFrom:
    - secretRef:
        name: cerbos-github-token
  config:
    driver: "git"
    git:
      protocol: https
      # git URL
      url: my-git-url.cerbos-ABAC.git
      # Replace with the branch name of your repo.
      branch: some_branch
      # Remove or leave empty if the policies are not stored in a subdirectory.
      #subDir: hr
      # Path to checkout. By default, /work is a Kubernetes emptyDir volume that is only available for the lifetime of the pod.
      # If you want the work directory to persist between pod restarts, specify the mount path of a persistent volume here.
      #checkoutDir: /work
      # How often the remote repo should be checked for updates.
      updatePollInterval: 60s
      # Credentials used to login to the remote GitHub repo. We are using an environment variable mounted from the secret we created earlier.
      https:
        username: ${GITHUB_TOKEN} 
        password: ""
c
The indentation in your values file is incorrect. The 
service
and 
envFrom
sections should be at the top level. The storage configuration should be under 
cerbos.config.storage
. Try with this fixed config:
Copy code
service:
  type: ClusterIP
  httpPort: 3592
  grpcPort: 3593
  httpNodePort: 13592
  grpcNodePort: 13593

envFrom:
  - secretRef:
      name: cerbos-github-token

cerbos:
  config:
    storage:
      driver: "git"
      git:
        protocol: https
        url: my-git-url.cerbos-ABAC.git
        branch: some_branch
        updatePollInterval: 60s
        https:
          username: ${GITHUB_TOKEN}
          password: ""
See https://github.com/cerbos/cerbos/blob/main/deploy/charts/cerbos/values-git-storage.yaml for an example.
To answer your question about debugging the Cerbos container, you can use 
kubectl debug
to launch an ephemeral container with a shell. https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#debug
d
oh nice - good tips, have been using k8 for a while but did not know about 
kubecl debug
hmm - I have all my services configured by a single values yaml . . how how can I maybe sure that envFrom and service only apply to cerbos - I have about 4 or 5 other services being configure in this file..
i suppose I would have to edit the helm chart
c
Yeah, I don't think it's possible to use a single values file to install multiple charts using just Helm itself. If you really want to have just one values file, I'd suggest using Helmfile.
d
ic what you mean - I think one can do it - but its basically a matter of luck that I haven’t run into a variable naming conflict so far
thank for your help - looking at the templates here - looks like there is no way to set the namespace? https://artifacthub.io/packages/helm/cerbos/cerbos ?
2 Views
Open in Slack
PreviousNext
Powered by