https://cerbos.dev logo
Join Slack
Powered by
hello everyone :slightly_smiling_face: does cerbos...
# help
a
hello everyone 🙂 does cerbos works with fly.io ? I tried to configure fly.toml but I see an error in the logs in my nodejs server
Copy code
{"type":"Error","message":"gRPC error 14 (UNAVAILABLE): read ECONNRESET","stack":"NotOK: gRPC error 14 (UNAVAILABLE): read ECONNRESET
my fly.toml configuration
Copy code
app = 'cerbos-service'
primary_region = 'waw'

[build]

[http_service]
  internal_port = 3592
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [http_service.http_options]
    h2_backend = true

[[services]]
  internal_port = 3592
  protocol = "tcp"
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    handlers = ["http"]
    start_port = 3592
    end_port = 3592

[[services]]
  internal_port = 3593
  protocol = "tcp"
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    handlers = ["tls", "http"]
    start_port = 3593
    end_port = 3593

  [services.ports.tls_options]
    alpn = ["h2"]

[[vm]]
  size = "shared-cpu-1x"
  memory = '256mb'
  cpu_kind = 'shared'
  cpus = 1
when I changed host and PORT from my nodejs server I get other error
Copy code
message":"gRPC error 1 (CANCELLED): Call cancelled","stack":"NotOK:
you can try check https://cerbos-service.fly.dev:3592/
o
Hi @Alex Dolid, I am not really familiar with configuring 
<http://fly.io|fly.io>
but bear with me. I think the first 
[[services]]
is not required (?). Since Cerbos uses port 
3952
for HTTP and 
3593
for gRPC, perhaps the configuration might look like this?
Copy code
app = 'cerbos-service'
primary_region = 'waw'

[build]

[http_service]
  internal_port = 3592
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [http_service.http_options]
    h2_backend = true

[[services]]
  internal_port = 3593
  protocol = "tcp"
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

  [[services.ports]]
    handlers = ["tls"]
    start_port = 3593
    end_port = 3593

  [services.ports.tls_options]
    alpn = ["h2"]

[[vm]]
  size = "shared-cpu-1x"
  memory = '256mb'
  cpu_kind = 'shared'
  cpus = 1
✅ 1
a
thanks for your help but with this variant of the configuration, we do not see the swagger statics at all and get another error gRPC error 1 (CANCELLED): Call cancelled it happend when I put like 
cerbos-service.fly.dev:3593
like CERBOS_URL
but when I put 
cerbos-service.fly.dev:3592
like CERBOS_URL in my config env vars I got the gRPC error 14 (UNAVAILABLE) again
I tried different configuration parameters of the 
fly.toml
file, but nothing worked and I still haven’t found a reason why fly doesn’t want to work correctly with cerbos even though it has first-class support for docker containers, because almost everything that is mounted with fly is mounted with docker. I have a suspicion that it’s a problem with the network, somehow fly can’t understand that two servers are up at once in one container for sure, but it’s really strange.
o
I will try to run cerbos on fly.io and get back to you as soon as possible
a
thank you very much🤗
by the way, I also tried on Heroku, but it didn’t work either. First, Heroku does not have gRPC support, and secondly, it dynamically determines the port for you, you cannot specify a port (and only one port is allocated), which makes it impossible to work correctly with Cerbos PDP Service, so I decided to try fly.io, because it is more flexible and, in theory, should be able to work correctly with Cerbos
c
Hey, Just in case it wasn't clear, Cerbos doesn't need both ports in order to work. All of our SDKs and the CLI use gRPC so as long as you can make that port accessible to your application, everything should work (provided that the cloud service supports HTTP/2). Pretty much all the SDKs allow you to specify which port to connect to as well so you don't need to worry about exposing 3593 specifically.
👍 1
o
Hi @Alex Dolid, I think this configuration will resolve the issue and it also has healthchecks and metrics:
Copy code
app = 'cerbos'
primary_region = 'lhr'

[build]
  image = 'ghcr.io/cerbos/cerbos:0.34.0'

[[mounts]]
  source = 'cerbos_policies'
  destination = '/policies'
  initial_size = '1GB'

[[services]]
  protocol = ''
  internal_port = 3592
  auto_stop_machines = true

[[services.ports]]
    port = 3592
    handlers = ['tls', 'http']

[[services.http_checks]]
    interval = '5s'
    timeout = '2s'
    grace_period = '5s'
    method = 'get'
    path = '/_cerbos/health'
    protocol = 'http'

[[services]]
  protocol = ''
  internal_port = 3593
  auto_stop_machines = true

[[services.ports]]
    port = 3593
    handlers = ['tls']

    [services.ports.tls_options]
      alpn = ['h2']

[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

[metrics]
  port = 3592
  path = "/_cerbos/metrics"
a
thanks, let me check 🙂
🙌🏻 1
@oguzhan still it doesn’t work either, we get static for OpenAPI documentation, but if we call a request, it doesn’t go anywhere and in my nodejs server I get the following error:
Copy code
{"type":"Error","message":"gRPC error 1 (CANCELLED): Call cancelled","stack":"NotOK: gRPC error 1 (CANCELLED): Call cancelled
c
Hey, so Cerbos seems to be deployed correctly. Since you posted your Cerbos URL, I took the liberty of sending it a dummy request and both REST and gRPC endpoints are working correctly
Copy code
curl -k <https://YOUR_SERVICE.fly.dev:3592/api/check/resources> -d @req01.json
{"requestId":"test", "results":[{"resource":{"id":"XX125", "kind":"leave_request", "policyVersion":"20210210"}, "actions":{"approve":"EFFECT_DENY", "create":"EFFECT_DENY", "defer":"EFFECT_DENY", "view:public":"EFFECT_DENY"}}, {"resource":{"id":"XX150", "kind":"leave_request", "policyVersion":"20210210"}, "actions":{"approve":"EFFECT_DENY", "create":"EFFECT_DENY", "view:public":"EFFECT_DENY"}}], "cerbosCallId":"01HRCSE8XYEC8QDCZKRVYX4GZN"}
Copy code
buf curl -k -d @req01.json --protocol=grpc <https://YOUR_SERVICE.fly.dev:3593/cerbos.svc.v1.CerbosService/CheckResources>
{
  "requestId": "test",
  "results": [
    {
      "resource": {
        "id": "XX125",
        "kind": "leave_request",
        "policyVersion": "20210210"
      },
      "actions": {
        "approve": "EFFECT_DENY",
        "create": "EFFECT_DENY",
        "defer": "EFFECT_DENY",
        "view:public": "EFFECT_DENY"
      }
    },
    {
      "resource": {
        "id": "XX150",
        "kind": "leave_request",
        "policyVersion": "20210210"
      },
      "actions": {
        "approve": "EFFECT_DENY",
        "create": "EFFECT_DENY",
        "view:public": "EFFECT_DENY"
      }
    }
  ],
  "cerbosCallId": "01HRCS7E5C1SAXX511W199FRR9"
}
So, whatever is happening is because of something else. Is your Node app deployed to fly.io as well?
o
I have checked the OpenAPI UI and the problem seems to be SSL termination:
Copy code
api-request.js:1143 Mixed Content: The page at 'https://<SERVICE_URL>.fly.dev:3592/#post-/api/check/resources' was loaded over HTTPS, but requested an insecure resource 'http://<SERVICE_URL>.fly.dev/api/check/resources'. This request has been blocked; the content must be served over HTTPS.
a
hmm it is strange, yes my Node app deployed to fly.io as well I used grpc client like
Copy code
import { GRPC as Cerbos } from '@cerbos/grpc';
this.#cerbos = new Cerbos(configService.get('CERBOS_URL'), { tls: false });
const isAllowed = await this.#cerbos.isAllowed({
      principal: {
        id: user.id,
        roles: user.roles,
      },
      action: 'read',
      resource: {
        kind: 'app',
        id: 'app',
      },
    });
in 
CERBOS_URL
I put 
cerbos-service.fly.dev:3593
c
Try without 
tls: false
. I believe the endpoint has TLS.
a
yes, looks like works, I appreciate your help, thanks of course another problem appeared, 
isAllowed
method locally return 
true
, but on fly.io return 
false
for the same request, but I think I’ll figure it out myself
@oguzhan @Charith (Cerbos) Thanks again to everyone for helping to figure out how to make Cerbos friends with fly.io ❤️
🎉 1
c
Glad it works! 🎉 Regarding the different responses, it could potentially be because you're missing a policy in the fly.io deployment.
a
it is strange because I have policy, maybe I should define custom container run command or image build command on fly.io
c
If you have the same policy both locally and remotely, the response should be the same. Only way it could differ is if the policy doesn't exist, the policy is different or if the request has different attributes.
o
If Admin API is enabled in your configuration you could list the policies in the PDP to double-check if the policies are available in the fly.io instance. (See ListPolicies RPC)
a
I fixed it, thanks 🙂
👍 1
2 Views
Open in Slack
PreviousNext
Powered by