Hi, I have a use case and hope that anyone here has experience or can help with. I'm building a multi-tenant SaaS platform where we have system-defined roles in each of the tenant. Users joining to a tenant must have a role. Each role will have an entire permission set to access the system. I want to allow users to customize the permissions of the roles, and even create their own roles. I'm planning to use Cerbos for this. My thinking policy model would be defining the default roles in the root scope. Then each of the tenant that has customization, I'll populate policies to the child scope, identified by scope ID. So the layout would be like

resource_policies
|-- tenant_1
|   |-- resource_1.yaml
|-- tenant_2
|   |-- resource_1.yaml
|-- resource_1.yaml

My question is, is this a scalable solution? I'm thinking like how would it end up if I have a lot of tenants with customized roles and permissions? Would it be a problem for Cerbos since the policy will be huge. Also, whenever there is a tenant customizing their roles, I have to reload Cerbos PDP. Would it only reload what is changed or the whole structure? Thanks

Hey, Cerbos is not really bounded by the number of policies. Both us and some of our customers have independently tested Cerbos with tens of thousands of policies successfully. Because Cerbos is stateless, if the number of policies gets too large, you have the option of sharding them over several Cerbos instances as well. If you only have a single Cerbos instance, you don't have to reload it. The reload call just makes Cerbos clear its cache. The size of your policy repo doesn't have an effect on that because the cache only contains the most frequently used policies and reloading them into the cache has very little overhead.

Hi Charith, I'm so excited to hear that. I've been looking around for a solution to my problem and Cerbos is really a niche. To elaborate more on my intention, I'll be deploying multiple Cerbos instances. I'm yet to decide whether it will be a ReplicaSet or sidecars in my k8s cluster. I intend to use disk storage. I'll mount a volume to all the instance. I'll have another service with UI for the users to customize their Roles and Permissions. That service will write new policy files with scopes to the volume; and the Cerbos instances will pickup the changes. I don't really know the mechanism under the hood, but I assume it'd be similar to calling the reload admin API in case I use DB storage. Can you help me evaluate if this is the right thing to do? Also, I'm very interested in sharding. Does Cerbos have any support for it? Or I'll have to do it my own?

Cerbos can detect changed policies on disk so, you don't need to reload. However, I think it'd be tricky to use disk storage for this because you'll need a volume type that can be mounted to multiple pods at once. A database store would be the easiest. If that's not an option, I'd recommend using at least the git or blob stores. Cerbos can automatically detect changes on git or blob stores so no need to reload either. Sharding is not natively supported but you can easily set it up with a proxy server.

Thank you, it's super clear. I'm deploying on AWS EKS so I think I can use EFS in this case to mount in to multiple pods. The reason why I was reluctant on using DB is because it'd be tricky to call reload API to all the instances. For blob storage, I'm afraid that the frequent polling on S3 with a lot of policy files would be not efficient and would increase AWS cost. Rate limit on S3 would also be an issue if there are so many instances with so many files...