Turns out it's possible. You need to replace the Zap logger in your test. Here's what I did:

logger := zaptest.NewLogger(t)
zap.ReplaceGlobals(logger)

Full code listing is on this gist: https://gist.github.com/charithe/f060269f676fde8f8a54374cf8a36eea

So it seems that my in-memory SQLite somehow got mixed up with another in-memory sqlite database, even though that database should be isolated:

failed to get check for [john.default]: failed to get compilation units: SQL logic error: no such table: policy_ancestor (1)

When I changed

:memory:

to

file:CERBOS?mode=memory&cache=shared&_fk=1

it started working consistently. I'm not entirely sure what the difference is, but the latter is what I usually use for in-memory sqlites (as I didn't know about the first option).