https://cerbos.dev logo
Join SlackCommunities
Powered by
I'm experimenting deploying cerbos via minikube us...
# help
j
I'm experimenting deploying cerbos via minikube using the following configmap:
Copy code
apiVersion: v1
kind: ConfigMap
metadata:
  name: cerbos-config
  labels:
    app: {{ .Values.name }}
data:
  "config.yaml": |-
    server:
      httpListenAddr: ":{{ .Values.port }}"
      grpcListenAddr: ":{{ .Values.grpcPort }}"
      metricsEnabled: true
      logRequestPayloads: true
      playgroundEnabled: false
      adminAPI:
        enabled: true
        adminCredentials:
          username: {{ .Values.adminCredentials.username }}
          passwordHash: {{ .Values.adminCredentials.passwordHash }}
    storage:
      driver: "disk"
      disk:
        directory: /work
I'm getting this error:
Copy code
{
  "log.level": "error",
  "@timestamp": "2021-10-28T17:26:15.813Z",
  "log.logger": "cerbos.server",
  "message": "Failed to load configuration",
  "error": "failed to create config provider: couldn't expand environment: default is empty for \"RFeXyZ\" (use \"\" for empty string)"
}
It seems to be from the uber config tool here:
Copy code
<https://github.com/uber-go/config/blob/7eebe84240ad56046741394bf03b34129c07be12/expand.go#L75>
o
Hi @Jack Archer, Does helm values resolved correctly when you check the output of the 
kubectl get cm cerbos-config -o yaml
?
j
yup
Copy code
kubectl get cm cerbos-config -o yaml
apiVersion: v1
data:
  config.yaml: |-
    server:
      httpListenAddr: ":3592"
      grpcListenAddr: ":3593"
      metricsEnabled: true
      logRequestPayloads: true
      playgroundEnabled: false
      adminAPI:
        enabled: true
        adminCredentials:
          username: "cerbos"
          passwordHash: "<pass>"
    storage:
      driver: "disk"
      disk:
        directory: "/work"
kind: ConfigMap
metadata:
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: cerbos
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: default
  creationTimestamp: "2021-10-27T21:06:58Z"
  labels:
    app: cerbos
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
  name: cerbos-config
  namespace: default
  resourceVersion: "6445"
  uid: 8d986610-c76f-48eb-9eec-fa68e66fd19b
o
Would you mind also sharing the k8s deployment spec?
j
This is the relevant part:
Copy code
imagePullPolicy: Always
        args:
          - "server"
          - "--config=/config/config.yaml"
          - "--log-level={{ .Values.logLevel }}"
Why are you including the 
config.Expand
option? I didn't see anything in the docs about configuring via env vars
FYI (for anyone reading this thread in the future) -- the 
config.Expand
option is used to enable env var interpolation in the config file... see here: https://github.com/cerbos/cerbos/blob/main/internal/config/config_test.go#L58 The error I was experiencing was due to the "$" char being included in the password hash string... 
uber-go/config
was trying to interpolate it as an env var cthe
When I removed the 
$
the build succeeded
As food for thought... I'm wondering if you really need env var interpolation of config files... I would expect <something-else> should be handling the creation of dynamic config files... that feels like unnecessary complication for your service
^ @Alex Olivier (Cerbos)
o
Glad the issue has been resolved, also thanks for sharing the source of the problem with others 🙂.
j
And now it's working... so IDK what I did wrong before...
c
Environment variable interpolation is enabled to allow people to provide sensitive information (Eg. Database connection strings)
j
I know... but that should be handled by helm or another tool IMHO
c
I agree, but it's a common pattern (12-factor apps for example) and we want to support that.
j
well I might submit a PR later... the string interpolation is breaking with the bcrypted passwords
probably need to b64 encode the passwords
c
I am not at a computer right now but that seems odd. Environment variable interpolation happens only for strings like ${VAR}. Does your hash have something like that?
j
yes
Copy code
echo "cerbosAdmin123" | htpasswd -niBC 10 cerbos | cut -d ':' -f 2
$2y$10$n9lOaESI15cOkpeeO7ISJOjY1IV3SCFbfTrDXomt2Y.plRMRmlSlq
that password hash breaks the config
it's trying to interpolate the 
$
c
Hmm..I am pretty sure I have tested with bcrypt hashes containing $ signs. This must be something else.
j
when I don't include the $ it works
or it's only certain cases
Here's a specific example that you can recreate... I'll submit a PR this afternoon
Copy code
echo "cerbosAdmin123" | htpasswd -niBC 10 cerbos | cut -d ':' -f 2
$2y$10$zoVBUHm7YJu7Uj21WeeALOTObW1MQ0IxdjYBBwiv3i2j.BVPZAZ6C
Copy code
kubectl describe cm cerbos-config
Name:         cerbos-config
Namespace:    default
Labels:       app=cerbos
              <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
Annotations:  <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: cerbos
              <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: default

Data
====
config.yaml:
----
server:
  httpListenAddr: ":3592"
  grpcListenAddr: ":3593"
  metricsEnabled: true
  logRequestPayloads: true
  playgroundEnabled: false
  adminAPI:
    enabled: true
    adminCredentials:
      username: "cerbos"
      passwordHash: "$2y$10$zoVBUHm7YJu7Uj21WeeALOTObW1MQ0IxdjYBBwiv3i2j.BVPZAZ6C"
storage:
  driver: "disk"
  disk:
    directory: "/work"
Events:  <none>
c
In your manifest, you get the hash from a template variable. Can you try surrounding the template expression in quotes?
j
Copy code
kubectl logs cerbos-77d8d74875-rntmk 
{"log.level":"info","@timestamp":"2021-10-28T19:08:29.699Z","log.logger":"cerbos.server","message":"maxprocs: Updating GOMAXPROCS=1: using minimum allowed GOMAXPROCS"}
{"log.level":"error","@timestamp":"2021-10-28T19:08:29.700Z","log.logger":"cerbos.server","message":"Failed to load configuration","error":"failed to create config provider: couldn't expand environment: default is empty for \"zoVBUHm7YJu7Uj21WeeALOTObW1MQ0IxdjYBBwiv3i2j\" (use \"\" for empty string)"}
ERROR: failed to create config provider: couldn't expand environment: default is empty for "zoVBUHm7YJu7Uj21WeeALOTObW1MQ0IxdjYBBwiv3i2j" (use "" for empty string)
{"log.level":"info","@timestamp":"2021-10-28T19:08:29.701Z","log.logger":"cerbos.server","message":"maxprocs: Resetting GOMAXPROCS to 6"}
c
Ah ok. Thanks. I'll see what happens there.
j
Copy code
$zoVBUHm7YJu7Uj21WeeALOTObW1MQ0I => default is empty for \"zoVBUHm7YJu7Uj21WeeALOTObW1MQ0Ixdj
the bcrypt hashes need to be b64 encoded to be compatible with your string interpolation functionality
o
https://github.com/uber-go/config/blob/1ab480fe347f1174778e7fba6f83aee0a2e41347/option.go#L49
Copy code
...
// Expand allows variable references to take two forms: $VAR or
// ${VAR:default}.
...
Documentation for the 
config.Expand
function states 
$VAR
also expands the variable.
j
Do these docs solve the problem tho?
^ FYI - I don't mean that sarcastically
c
Temporary workaround is to use 
$$
to escape the 
$
sign.
Copy code
adminAPI:
    enabled: true
    adminCredentials:
      username: "cerbos"
      passwordHash: "$2y$10$$zoVBUHm7YJu7Uj21WeeALOTObW1MQ0IxdjYBBwiv3i2j.BVPZAZ6C"
j
That's not a good solution
can we just agree that the password hashes should be base64 encoded?
c
Yes, that's why I said it's temporary
j
Cool well I'll submit the PR shortly
Appreciate your help - I think overall this solution is elegant and I'm impressed with the codebase
c
Thanks. Let us know if there's anything else we can help you with.
j
@Charith (Cerbos) - can you please approve me running the GH actions? https://github.com/cerbos/cerbos/pull/406
and check out this PR please
c
Sure
We have released 0.9.1 with the fix
j
Thank you!
9 Views
Open in Slack
PreviousNext
Powered by