hi folks, we have a setup where our main app using...
# help
l
hi folks, we have a setup where our main app using cerbos and the cerbos service are on the same container, so it's all local over gRPC, but we're getting errors
NotOK: gRPC error 14 (UNAVAILABLE): Stream refused by server
CPU usage looks fine overall so it doesn't seem to need vertical scaling, but could it be possibly due to the
requestLimits
? We're using the default (50), maybe we need to bump that? How would I go about figuring out the potential culprit? Thanks for your help 🙏
we're on cerbos 0.46 (I'm gonna upgrade to 0.47) and
@cerbos/grpc
0.23.0 (which I'll upgrade to 0.23.2)
a
Hey there. Could you provide a bit more detail of how you are running your service and the PDP in the same container?
l
hi Alex, what kind of details are you looking for? we run both as a service, policies are on disk
a
Thanks helpful how is the container running both services? Is there a process manager in there or are you using ‘cerbos run’? Is the service using one of the SDKs to talk to the PDP? Where is the container running? K8s? Have you tested different request limits? What is the expected request rate?
Happy to move to DM or have a quick call if easier
l
we use systemd, here's how we start the service:
Copy code
ExecStart=/usr/local/bin/cerbos server --config=/apps/cerbos/config.yaml --log-level="warn"
we user the
@cerbos/grpc
SDK (the service is a node app) to make the calls, checking one instance now it looks like we can avg. 30-50 rps but can peak at 100+ rps. we're not running on k8s, we have our own container orchestration and runtime. I haven't tested different request limits, that's one thing I was curious about if I should try to bump up the limit since the default is 50
I can do either, whatever is best for you, thanks for your help 😄
a
Great - I’ll DM you