Skip to content

Add CacheListener for observing cache events - #269

Open
Grryum wants to merge 2 commits into
zio:series/2.xfrom
Grryum:cache-listener
Open

Add CacheListener for observing cache events#269
Grryum wants to merge 2 commits into
zio:series/2.xfrom
Grryum:cache-listener

Conversation

@Grryum

@Grryum Grryum commented Jul 27, 2026

Copy link
Copy Markdown

Motivation

Currently the only way to observe cache activity is polling cacheStats, which exposes just cumulative hit/miss counters. There is no way to react to individual cache events, so common operational needs are impossible to express:

Changes

This PR adds a CacheListener that is notified of cache events:

trait CacheListener[-Key, -Error, -Value] {
  def onHit(key: Key)(implicit unsafe: Unsafe): Unit
  def onMiss(key: Key)(implicit unsafe: Unsafe): Unit
  def onLoad(key: Key, exit: Exit[Error, Value], loadTime: Duration)(implicit unsafe: Unsafe): Unit
  def onEviction(key: Key, cause: CacheListener.EvictionCause)(implicit unsafe: Unsafe): Unit
}
  • onLoad fires for every completed lookup (both get misses and refresh) with the resulting Exit and the time the lookup took.
  • onEviction carries the cause: Capacity, Expired, or Invalidated.
  • A ready-made CacheListener.metrics(cacheName) reports all events with ZIO metrics (zio_cache_hits, zio_cache_misses, zio_cache_load_successes, zio_cache_load_failures, zio_cache_load_duration, zio_cache_evictions), tagged with the cache name.
  • A listener is attached via new overloads of Cache.make, makeWith, and makeWithKey; with makeWithKey the listener observes the keys produced by the keyBy function.

Design notes

  • Listeners are synchronous side-effecting callbacks, not ZIO effects. Hits and evictions are tracked on the non-effectful hot path of the cache (getUnsafe, the eviction loop in trackAccess), where running effects would require either blocking the eviction loop on user code or giving up the zero-allocation hit path. This mirrors Caffeine's StatsCounter. An effectful adapter (e.g. forking a handler on a Runtime) can be layered on top later without breaking anything.
  • All methods have no-op defaults, so listeners override only the events they care about, and future events can be added without breaking existing implementations.
  • A listener that throws cannot corrupt the cache: exceptions are caught around every notification, so promises are always completed and the internal state stays consistent.
  • Existing constructors are untouched (binary and source compatible); the only MiMa filter added is for the private CacheImplementation class (its constructor gained a parameter).
  • invalidateAll does not emit per-key events since the underlying map is cleared wholesale; this is documented.

@Grryum
Grryum requested a review from kyri-petrou as a code owner July 27, 2026 11:41
Comment thread zio-cache/shared/src/main/scala/zio/cache/CacheListener.scala Outdated
Comment thread zio-cache/shared/src/main/scala/zio/cache/Cache.scala
Comment thread zio-cache/shared/src/main/scala/zio/cache/CacheListener.scala
Comment thread zio-cache/shared/src/main/scala/zio/cache/Cache.scala Outdated
Comment on lines +315 to +324
var invalidated: List[Key] = Nil
val iterator = map.keySet().iterator()
while (iterator.hasNext) {
val key = iterator.next()
if (map.remove(key) ne null) {
invalidated = key :: invalidated
}
}
if (invalidated eq Nil) Exit.unit
else ZIO.foreachDiscard(invalidated)(key => trackEviction(key, CacheListener.EvictionCause.Invalidated))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be simplified / optimized to:

val invalidated = map.keySet()
map.clear()
if (invalidated.isEmpty) Exit.unit
else ZIO.foreachDiscard(invalidated)(key => trackEviction(key, CacheListener.EvictionCause.Invalidated))

Note that we can also have a conditional if (isNoopListener) and simply do a map.clear() in that case to avoid the penalty of all the extra operations

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConcurrentHashMap#keySet() returns a live view backed by the map rather than a snapshot, so after map.clear() the invalidated set would be empty and no eviction events would be emitted. I kept the remove-and-check approach for a real listener, since it also ensures we only report entries actually removed by this invocation. I did add the suggested no-op fast path, which uses the original map.clear() directly.

Comment thread zio-cache/shared/src/main/scala/zio/cache/Cache.scala
Comment thread zio-cache/shared/src/main/scala/zio/cache/CacheListener.scala
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants