API Reference

Core Cache

class cache.async_cache.AsyncCache(maxsize=128, default_ttl=None, batch_window_ms=5, max_batch_size=100, backup=None, remote_cache=None)

Bases: object

async get(key, loader=None, batch_loader=None, ttl=None)

Two-tier cache get: L1 (local) -> L2 (remote) -> loader.

Read pattern:

1. Check L1 (in-memory) — immediate

2. If miss and remote_cache configured, check L2 (remote)

3. If miss, call loader (with thundering herd protection)

4. Write result to L1 synchronously, L2 asynchronously

Thundering herd protection and batch loading work with remote_cache. Redis errors are caught and logged — cache degrades to L1 only.

set(key, value, ttl=<object object>, write_remote=True)

Set a value in the cache. Writes to L1 synchronously, L2 async in background.

delete(key)

Delete a key from L1 and L2 cache.

clear()

Clear L1 cache and reset metrics. Also clears L2 async.

get_metrics()

Get cache metrics (hits, misses, size, hit_rate).

async warmup(keys_with_loaders)

Warmup: serial gets (each locks internally for hit/miss).

save_to_backup()

Persist current cache contents to the configured backup.

Call this on application shutdown to survive restarts. Entries are stored with their remaining TTL (relative seconds). No-op if no backup is configured.

load_from_backup()

Reload cache contents from the configured backup.

Useful for manual reload without re-creating the cache object. No-op if no backup is configured.

class cache.async_lru.AsyncLRUInvalidator(cached_fn, skip_args=0, clear_all=False, key_fn=None)

Bases: object

Decorator that invalidates a specific AsyncLRU-cached function when the decorated mutation/write function is called.

Usage:

@AsyncLRU(maxsize=128)
async def get_user(user_id):
    return await db.get_user(user_id)

@AsyncLRUInvalidator(get_user)
async def update_user(user_id, data):
    await db.update_user(user_id, data)

await get_user(1)         # miss -> loads
await get_user(1)         # hit
await update_user(1, {})  # invalidates get_user(1), then runs mutation;
                          # cache key is evicted before the mutation so
                          # no reader ever sees stale data, even if the
                          # mutation raises
await get_user(1)         # miss -> reloads

Invalidation happens before the mutation executes so that no reader ever sees stale data, even if the mutation raises.

Parameters:

• cached_fn – The AsyncLRU-wrapped function to invalidate.

• skip_args – Removed. Passing a non-zero value raises TypeError. Use key_fn instead to map mutation args to cache-key args.

• clear_all – If True, clear the entire cache instead of a specific key.

• key_fn – Optional callable (mutation_args, mutation_kwargs) -> (args, kwargs) that extracts the cached function’s key args from the mutation function’s args. If None (default), the mutation function’s full positional args and kwargs are passed to the cached function’s invalidate_cache(), which applies its own skip_args to derive the correct key.

class cache.async_lru.AsyncLRU(maxsize=128, skip_args=0, backup=None, remote_cache=None)

Bases: object

Parameters:

skip_args (int)

clear_cache()

Clears the LRU cache.

This method empties the cache, removing all stored entries and effectively resetting the cache.

Returns:

None

class cache.async_ttl.AsyncTTLInvalidator(cached_fn, skip_args=0, clear_all=False, key_fn=None)

Bases: object

Decorator that invalidates a specific AsyncTTL-cached function when the decorated mutation/write function is called.

Usage:

@AsyncTTL(time_to_live=60)
async def get_session(sid):
    return await db.get_session(sid)

@AsyncTTLInvalidator(get_session)
async def end_session(sid):
    await db.delete_session(sid)

await get_session("s1")    # miss -> loads
await get_session("s1")    # hit
await end_session("s1")    # invalidates get_session("s1"), then runs
                           # mutation; cache key is evicted before the
                           # mutation so no reader ever sees stale data,
                           # even if the mutation raises
await get_session("s1")    # miss -> reloads

Invalidation happens before the mutation executes so that no reader ever sees stale data, even if the mutation raises.

Parameters:

• cached_fn – The AsyncTTL-wrapped function to invalidate.

• skip_args – Removed. Passing a non-zero value raises TypeError. Use key_fn instead to map mutation args to cache-key args.

• clear_all – If True, clear the entire cache instead of a specific key.

• key_fn – Optional callable (mutation_args, mutation_kwargs) -> (args, kwargs) that extracts the cached function’s key args from the mutation function’s args. If None (default), the mutation function’s full positional args and kwargs are passed to the cached function’s invalidate_cache(), which applies its own skip_args to derive the correct key.

class cache.async_ttl.AsyncTTL(time_to_live=60, maxsize=1024, skip_args=0, backup=None, remote_cache=None)

Bases: object

Parameters:

skip_args (int)

clear_cache()

Clears the TTL cache.

This method empties the cache, removing all stored entries and effectively resetting the cache.

Returns:

None

Cache Backends

Pluggable cache backend abstraction.

Provides: - CacheBackend: abstract interface for persistent cache storage - DiskBackend: file-based persistence using pickle (stdlib only)

Designed for extensibility: Redis, Memcached, or custom backends can be added by subclassing CacheBackend without changing core cache logic.

class cache.backend.CacheBackend

Bases: ABC

Abstract interface for persistent cache storage.

Subclass this to implement Redis, Memcached, or any other backend. All methods are synchronous; async wrappers live in the cache layer.

abstractmethod load()

Load persisted data and return a dict of {key: (value, expiration)}.

expiration is a float (monotonic deadline) or None (no TTL). Expired entries MAY be included; the caller filters them. Returns an empty dict if no persisted data exists.

abstractmethod save(data)

Persist cache data.

Parameters:

data – dict of {key: (value, ttl_remaining_seconds_or_None)} ttl_remaining is seconds remaining (float) or None for no-TTL entries.

abstractmethod clear()

Remove all persisted data.

class cache.backend.DiskBackend(path)

Bases: CacheBackend

File-based cache persistence using pickle.

Data is written atomically (write-to-temp then rename) to prevent corruption on crash. No external dependencies required.

Parameters:

path – filesystem path for the cache file (e.g. ‘/tmp/my_cache.pkl’)

load()

Load cache entries from disk.

Returns {key: (value, monotonic_expiration_or_None)}. Translates stored relative TTLs back to absolute monotonic deadlines, subtracting time elapsed since save.

save(data)

Persist cache entries to disk atomically.

Parameters:

data – {key: (value, ttl_remaining_seconds_or_None)}

clear()

Remove the persisted cache file.

Remote Cache Backends

Remote cache backend abstraction for distributed (L2) caching.

Provides: - RemoteCacheBackend: async abstract interface for remote cache stores - RedisBackend: lightweight pure-Python Redis client (no external deps)

The remote cache is used as L2 in the two-tier caching architecture:

L1 = local in-memory cache (AsyncCache) L2 = remote cache (Redis, Memcached, etc.)

All methods are async since remote operations involve network I/O.

class cache.remote.RemoteCacheBackend

Bases: ABC

Async abstract interface for remote (L2) cache storage.

Subclass this to implement Redis, Memcached, or any other remote store. All methods are async to support non-blocking network I/O.

abstractmethod async get(key)

Get a value from the remote cache.

Parameters:

key (str) – string cache key

Returns:

deserialized value, or None on miss

abstractmethod async set(key, value, ttl=None)

Set a value in the remote cache.

Parameters:

• key (str) – string cache key

• value – value to store (will be serialized)

• ttl – optional TTL in seconds (float or int)

abstractmethod async delete(key)

Delete a key from the remote cache.

Parameters:

key (str)

abstractmethod async clear()

Remove all entries managed by this cache instance.

abstractmethod async close()

Close the connection to the remote store.

class cache.remote.RedisBackend(host='localhost', port=6379, db=0, password=None, prefix='ac:', connect_timeout=5, socket_timeout=5)

Bases: RemoteCacheBackend

Pure-Python async Redis client for L2 caching.

Uses raw TCP sockets speaking the RESP protocol — no external dependencies required. Suitable for GET/SET/DEL/FLUSHDB operations.

Parameters:

• host – Redis server hostname (default “localhost”)

• port – Redis server port (default 6379)

• db – Redis database number (default 0)

• password – Optional Redis password

• prefix – Key prefix to namespace entries (default “ac:”)

• connect_timeout – Connection timeout in seconds (default 5)

• socket_timeout – Read/write timeout in seconds (default 5)

async get(key)

Get a value from Redis. Returns None on miss or error.

Parameters:

key (str)

async set(key, value, ttl=None)

Set a value in Redis with optional TTL (in seconds).

Parameters:

key (str)

async delete(key)

Delete a key from Redis.

Parameters:

key (str)

async clear()

Remove all keys with the configured prefix using SCAN + DEL.

async close()

Close the Redis connection.

Key Generation

class cache.key.KEY(args, kwargs)

Bases: object

Immutable, hash/eq-stable key for cache (args + kwargs). Fixes prior bugs: - __eq__ was hash-only (violated contract: a==b but hashes differ possible; collisions). - Hash unstable (dict.items() order pre-3.7, str(vars) arbitrary, kwargs.pop mutated caller!). - Now: frozen tuples, recursive _to_hashable (sorted dicts, stable obj repr), no mutation. Guarantees hash(a)==hash(b) iff a==b; stable across runs/Python versions. Used via make_key in decorators/AsyncCache.

cache.key.make_key(func, args, kwargs, skip_args=0)

Reusable key: func name + sliced args + cleaned kwargs. Handles skip_args (e.g., ‘self’); stable for complex types.

Agent Cache

Agent-aware caching layer for AI agent workflows.

Provides: - AgentCache: decorator for caching async read-tool results - AgentCacheInvalidator: decorator for write-tools that invalidate cached reads - AgentCacheSession: async context manager with per-session caching and loop detection

exception agent_cache.core.AgentLoopDetectedError

Bases: Exception

Raised when an agent execution loop is detected (on_loop=’raise’).

agent_cache.core.get_metrics()

Return global aggregate metrics across all sessions.

Return type:

dict

class agent_cache.core.AgentCacheSession(session_id=None, loop_detection=True, max_tool_repeats=5, max_execution_depth=50, on_loop='raise')

Bases: object

Async context manager providing session-scoped caching and loop detection.

Usage:

async with AgentCacheSession(loop_detection=True, max_tool_repeats=5) as session:
    await some_cached_tool(...)

get_execution_trace()

Return a copy of the execution trace.

get_metrics()

Return type:

dict

class agent_cache.core.AgentCache(resource, scope='global', ttl=None, maxsize=128, skip_args=0, backup=None, remote_cache=None)

Bases: object

Decorator that caches async tool results with resource tagging and scope.

Example:

@AgentCache(resource="cart", scope="global", ttl=60)
async def get_cart(user_id):
    ...

Parameters:

• resource (str)

• scope (str)

• maxsize (int)

• skip_args (int)

class agent_cache.core.AgentCacheInvalidator(resource, scope='global', skip_args=0, clear_all=True, key_fn=None)

Bases: object

Decorator for write/mutation tools that invalidate related cached reads.

Invalidation happens before the mutation executes so that no reader ever sees stale data, even if the mutation raises.

Example:

@AgentCacheInvalidator(resource="cart", scope="global")
async def add_to_cart(user_id, item):
    ...

Parameters:

• resource (str) – The resource tag to invalidate (must match an @AgentCache).

• scope (str) – "global" or "session".

• clear_all (bool) – If True (default), clear all entries for the resource. If False, requires key_fn to target a specific key.

• key_fn – Optional (args, kwargs) -> cache_key for selective invalidation when clear_all=False.

• skip_args (int) – Removed. Passing a non-zero value raises TypeError. Use key_fn instead to map mutation args to cache-key args.