Hello! We’re having issues with caching. The error is:

panic: gcache: Cache size <= 0

While in the docs it is:

You can turn off caching completely by setting compile.cacheSize configuration to 0. This would have a small performance hit but depending on your use case it could be negligible.

This happens when we set the property to

0

in helm

Hi @Roman Levytskyi, I will check this and get back to you as soon as possible.

version is latest:

0.44.0

The piece of information you've shared from the documentation is a mistake. I am going to fix it ASAP. Unfortunately, it is not possible to disable the compile cache. We suggest setting the

compile.cacheDuration

to really low value instead.

but can the cache duration guarantee the item eviction? what we need is to disable the caching completely because we have multiple replicas which need to be kept in sync and that’s why we want to disable cache.

Compiled policies in the cache gets evicted after the specified time in the

compile.cacheDuration

. After a compiled policy gets evicted from the cache, the next time it is needed it's retrieved from the database and cached again for the duration of

compile.cacheDuration

. In summary, setting the

compile.cacheDuration

to

60s

does what you have described.

in our case what we noticed is that policies althoug they have 60s cache storage, sometimes cache is out of date if the request hits another instance. if we drop all instances and they get recreated, the issue is solved. so does that mean that cache entries might exist for longer that specified by

cacheDuration

?

in our case what we noticed is that policies althoug they have 60s cache storage, sometimes cache is out of date if the request hits another instance.

Since each cerbos instance has its own in-memory cache, a specific policy could be cached at

t=0s

for one instance and

t=10s

for another instance. Let's say you've updated that policy in the database at

t=20s

, this means the first instance will fetch the new policy at

t=60s

, while the second instance will fetch it at

t=70s

. This means if another request comes to second instance at

t=65s

as an example, the old policy will be used for the result.

yeah, I got you, what i’m trying to say is that cache somehow lasts longer than it’s configured TTL

Would it be possible for you to share the configuration parameters passed to the cerbos helm chart?

sure

hi guys I'm facing the same issue here using cerbos

v0.44.0

and postgres db any progress here worth sharing? additionally, about the Admin API reload store call: is this bit of info correct? is postgres a "reloadable storage driver"?

Hi @Paulo Henrique Cardoso, We found a bug causing this issue and working on a solution currently. And yes, database stores (including

postgres

) are reloadable.

ok, great! looking forward for the next release

In the meantime, if you do not use role policies you could use the

cerbos v0.40.0

which doesn't have this issue. If you'd like to keep using the latest version, you could call reload store RPC of each Cerbos instance to clear the cache. This also works. GitHub issue to track: cerbos/cerbos#2601

one question: let's say I have 2 PDP's running,

A

and

B

, both with policy

X

cached. PDP

A

received the admin API call to create policy

Y

PDP

B

never had used policy

Y

, therefore it isn't cached yet if I call the reload store on

B

, would it fetch policy

X

and

Y

?

In this case, since the the policy

Y

is newly introduced, PDP

B

will fetch it as usual when a

checkResources

or

planResources

call arrives. No reload store call is necessary for this case. If you'd update the policy

X

via Admin API of the PDP

A

, then you'd need to call reload store in PDP

B

. But to answer your question regarding reload store, the reload store purges everything in the cache and this means the PDP must hit the underlying storage after a call to the

checkResources

or

planResources

.

ok, got it so, if I wanted to ensure that PBP

B

always has it's cache up to date, I'd have to call

plan

or

check

to every policy stored on db only after that a reload store would be relevant

Sorry, I might've confused you a bit with the

check

and

plan

bit. Every time a policy is

added/deleted/updated

, calling reload store RPC on all instances is enough to remedy this issue.

I'm using cerbos as a part of my credit card authorization flow ideally, I would like to avoid going to the db to fetch the policy during a purchase by my user, because our system must reply to the credit card processor within 2s with that in mind, I would like to have all policies cached before they need to be used if I understood correctly, cache for a policy is only built when a request for

check

or

planResources

arrive

Hey @Paulo Henrique Cardoso and @Roman Levytskyi, We released a new version fixing this problem. See https://github.com/cerbos/cerbos/releases/tag/v0.45.0

hey @oguzhan thanks i’ll update it immediately