MemoryCache & configuration¶

MemoryCache¶

memory_reuse.core.MemoryCache ¶

MemoryCache(config: CacheConfig | None = None, **kwargs: Any)

High-level cache client for AI agent workloads.

MemoryCache is the primary public interface. It wires together a storage backend, the exact-match LLM cache, the TTL-backed tool cache, and statistics tracking.

Parameters:

Name	Type	Description	Default
`config`	`CacheConfig \| None`	Cache configuration. When omitted a default :class:`~memory_reuse.config.CacheConfig` is used (in-memory backend, 1-hour TTL, global scope).	`None`
`**kwargs`	`Any`	Keyword arguments forwarded to :class:`CacheConfig` when `config` is `None`. Allows quick construction:: `cache = MemoryCache(backend="redis", redis_url=os.environ["REDIS_URL"])`	`{}`

Example::

from memory_reuse import MemoryCache, CacheConfig

cache = MemoryCache(CacheConfig(backend="memory", default_ttl=600))
cache.set_context(user_id="alice")

result = await cache.exact.get(["my-prompt"], scope="user",
                                scope_id="alice")

stats `property` ¶

stats: CacheStats

Current cache statistics snapshot.

Returns:

Name	Type	Description
`A`	`CacheStats`	class:`~memory_reuse.stats.CacheStats` dataclass.

Example::

print(cache.stats.hit_rate)

from_env `classmethod` ¶

from_env() -> MemoryCache

Create a :class:MemoryCache from MEMORY_REUSE_* environment variables.

See :meth:~memory_reuse.config.CacheConfig.from_env for the full list of recognised variables.

Returns:

Type	Description
`MemoryCache`	A configured :class:`MemoryCache` instance.

Example::

import os
os.environ["MEMORY_REUSE_BACKEND"] = "redis"
os.environ["MEMORY_REUSE_REDIS_URL"] = "redis://localhost:6379/0"
cache = MemoryCache.from_env()

set_context ¶

set_context(*, user_id: str | None = None, session_id: str | None = None, tenant_id: str | None = None) -> None

Set the user/session context for scoped cache keys.

Context values are used by the LangGraph decorators when no explicit scope ID is passed. They do not affect calls that provide their own scope_id argument.

Parameters:

Name	Type	Description	Default
`user_id`	`str \| None`	Identifier for the current user.	`None`
`session_id`	`str \| None`	Identifier for the current session.	`None`
`tenant_id`	`str \| None`	Identifier for the current tenant (future use).	`None`

Example::

cache.set_context(user_id="alice", session_id="sess-123")

clear_context ¶

clear_context() -> None

Reset all context values to None.

Call this between requests in a shared server context to avoid leaking one user's context into another request.

reset_stats ¶

reset_stats() -> None

Reset all hit/miss/error counters to zero.

lookup `async` ¶

lookup(key_parts: list, query_text: str, *, scope: str, scope_id: str | None, exact_only: bool = False, threshold: float | None = None) -> Any | None

Look up a cached value, trying the exact cache before the semantic cache.

The combined flow tries the fastest, cheapest path first: an exact hash-match is attempted before any embedding is computed. Only when the exact cache misses — and semantic caching is enabled and not disabled for this call — is the query embedded and matched by similarity.

Parameters:

Name	Type	Description	Default
`key_parts`	`list`	Ordered list of values identifying the exact-cache entry.	required
`query_text`	`str`	The natural-language query used for semantic matching.	required
`scope`	`str`	Cache scope — `"global"`, `"user"`, or `"session"`.	required
`scope_id`	`str \| None`	User or session identifier for non-global scopes.	required
`exact_only`	`bool`	When `True`, the semantic cache is never consulted, forcing Phase 1 exact-only behaviour for this call site (for example a tool with side effects).	`False`
`threshold`	`float \| None`	Optional per-call similarity threshold overriding :attr:`~memory_reuse.config.CacheConfig.similarity_threshold`.	`None`

Returns:

Type	Description
`Any \| None`	The cached value on an exact or semantic hit, or `None` on a miss.

Raises:

Type	Description
`ScopeViolationError`	If `scope` requires a `scope_id` but none is provided.

store `async` ¶

store(key_parts: list, query_text: str, value: Any, *, scope: str, scope_id: str | None, ttl: int | None = None, exact_only: bool = False) -> None

Store a value in the exact cache and, when enabled, the semantic cache.

The exact-match entry is always written so a subsequent identical request hits the faster exact path. When semantic caching is enabled and not disabled for this call, the query's embedding is also stored so reworded but equivalent requests can match later.

Parameters:

Name	Type	Description	Default
`key_parts`	`list`	Ordered list of values identifying the exact-cache entry.	required
`query_text`	`str`	The natural-language query whose embedding is stored.	required
`value`	`Any`	The value to cache. Must be JSON-serialisable.	required
`scope`	`str`	Cache scope — `"global"`, `"user"`, or `"session"`.	required
`scope_id`	`str \| None`	User or session identifier for non-global scopes.	required
`ttl`	`int \| None`	Time-to-live in seconds. Falls back to :attr:`~memory_reuse.config.CacheConfig.default_ttl` when `None`.	`None`
`exact_only`	`bool`	When `True`, only the exact-match entry is written and the semantic cache is left untouched.	`False`

Raises:

Type	Description
`ScopeViolationError`	If `scope` requires a `scope_id` but none is provided.

ping `async` ¶

ping() -> bool

Check whether the backend is reachable.

Returns:

Type	Description
`bool`	`True` if the backend responds successfully.

close `async` ¶

close() -> None

Close backend connections gracefully.

Should be called on application shutdown to prevent resource leaks, especially when using the Redis backend.

Example::

async with asyncio.timeout(5):
    await cache.close()

flush `async` ¶

flush() -> None

Flush all cached entries from the backend.

Warning

This irreversibly removes every cached entry. Use only in development or test environments.

CacheConfig¶

memory_reuse.config.CacheConfig `dataclass` ¶

CacheConfig(backend: Literal['memory', 'redis'] = 'memory', redis_url: str | None = None, default_ttl: int | None = 3600, default_scope: Literal['global', 'user', 'session'] = 'global', key_prefix: str = 'memreuse', max_key_size: int = 512, enable_stats: bool = True, semantic_enabled: bool = False, similarity_threshold: float = 0.95, embedding_provider: Literal['openai', 'local', 'litellm'] | None = None, embedding_model: str | None = None, max_vectors_per_namespace: int = 10000, store_exact_on_semantic_hit: bool = True, extract_answer: bool = False, extract_min_similarity: float = 0.5)

Configuration for the MemoryCache.

All values can be overridden via environment variables when using :meth:MemoryCache.from_env.

Attributes:

Name	Type	Description
`backend`	`Literal['memory', 'redis']`	Storage backend to use. `"memory"` requires no extra dependencies; `"redis"` requires `pip install memory-reuse[redis]`.
`redis_url`	`str \| None`	Connection URL for the Redis backend. Should be supplied via the `MEMORY_REUSE_REDIS_URL` environment variable rather than hardcoded.
`default_ttl`	`int \| None`	Default time-to-live in seconds for cached entries. `None` means entries never expire.
`default_scope`	`Literal['global', 'user', 'session']`	Default cache scope applied when no explicit scope is passed to cache operations.
`key_prefix`	`str`	String prepended to every cache key for namespacing.
`max_key_size`	`int`	Maximum allowed cache-key length in bytes. Requests that would produce a longer key raise `ValueError`.
`enable_stats`	`bool`	When `True`, hit/miss/error counts are tracked and accessible via :attr:`MemoryCache.stats`.
`semantic_enabled`	`bool`	When `True`, the semantic (similarity-based) cache is enabled in addition to the exact cache. Off by default so existing behaviour is unchanged; requires `embedding_provider`.
`similarity_threshold`	`float`	Minimum cosine similarity (normalised to `[0.0, 1.0]`) at which two requests are treated as the same and a cached result is reused. Higher values favour precision over recall.
`embedding_provider`	`Literal['openai', 'local', 'litellm'] \| None`	Which embedding provider to use for semantic lookups. `None` disables provider selection; required when `semantic_enabled` is `True`.
`embedding_model`	`str \| None`	Optional model name passed to the embedding provider. `None` lets the provider pick its default model.
`max_vectors_per_namespace`	`int`	Maximum number of stored vectors per scope namespace before LRU eviction applies.
`store_exact_on_semantic_hit`	`bool`	When `True`, a semantic hit also writes an exact-match entry so the next identical request hits the faster exact path.
`extract_answer`	`bool`	When `True` (and the cached value is a string), a semantic hit returns only the sentence(s) of the stored answer that best match the query, rather than the whole answer. This is a purely extractive, embedding-based narrowing — no LLM call and no generation — so it can only return text already present in the stored answer. Off by default; the core cache returns values verbatim.
`extract_min_similarity`	`float`	Minimum normalised cosine similarity (`[0.0, 1.0]`) a sentence must reach against the query for :attr:`extract_answer` to return just that sentence. When no sentence clears this bar the full stored answer is returned, so extraction never yields an empty result.

Example::

config = CacheConfig(
    backend="redis",
    redis_url=os.environ["REDIS_URL"],
    default_ttl=600,
    default_scope="user",
)

__post_init__ ¶

__post_init__() -> None

Validate the configuration after initialisation.

from_env `classmethod` ¶

from_env() -> CacheConfig

Create a :class:CacheConfig from MEMORY_REUSE_* environment variables.

Recognised variables:

MEMORY_REUSE_BACKEND — "memory" or "redis"
MEMORY_REUSE_REDIS_URL — Redis connection URL
MEMORY_REUSE_DEFAULT_TTL — integer seconds or "none"
MEMORY_REUSE_DEFAULT_SCOPE — "global", "user", or "session"
MEMORY_REUSE_KEY_PREFIX — string prefix for all keys
MEMORY_REUSE_ENABLE_STATS — "true" / "false"
MEMORY_REUSE_SEMANTIC_ENABLED — "true" / "false"
MEMORY_REUSE_SIMILARITY_THRESHOLD — float in [0.0, 1.0]
MEMORY_REUSE_EMBEDDING_PROVIDER — "openai", "local", or "litellm"
MEMORY_REUSE_EMBEDDING_MODEL — embedding model name

Returns:

Name	Type	Description
`A`	`CacheConfig`	class:`CacheConfig` populated from the environment.

Statistics¶

memory_reuse.stats.CacheStats `dataclass` ¶

CacheStats(hits: int = 0, exact_hits: int = 0, semantic_hits: int = 0, misses: int = 0, errors: int = 0, total_requests: int = 0)

Snapshot of cache performance counters.

Attributes:

Name	Type	Description
`hits`	`int`	Number of successful cache lookups. Always equals `exact_hits + semantic_hits`.
`exact_hits`	`int`	Number of hits served by the exact (hash-based) cache.
`semantic_hits`	`int`	Number of hits served by the semantic (similarity) cache.
`misses`	`int`	Number of cache lookups that found no entry.
`errors`	`int`	Number of backend errors encountered during cache operations.
`total_requests`	`int`	Total number of cache lookup attempts (hits + misses).

Example::

stats = cache.stats
print(f"Hit rate: {stats.hit_rate:.1%}")

hit_rate `property` ¶

hit_rate: float

Fraction of requests that were cache hits, in the range [0.0, 1.0].

Returns:

Type	Description
`float`	`0.0` when no requests have been made yet.

to_dict ¶

to_dict() -> dict

Return a plain-dict representation of the stats snapshot.

Returns:

Type	Description
`dict`	Dictionary with keys `hits`, `exact_hits`, `semantic_hits`,
`dict`	`misses`, `errors`, `total_requests`, and `hit_rate`.

memory_reuse.stats.StatsTracker ¶

StatsTracker()

Thread-safe, asyncio-compatible statistics tracker.

Uses an :class:asyncio.Lock to serialise counter updates so that concurrent coroutines always see a consistent view of the counters.

Hits are tracked by type: :meth:record_exact_hit and :meth:record_semantic_hit each also bump the aggregate hits counter, so hits always equals exact_hits + semantic_hits.

Statistics recording is best-effort and never fatal: every record_* method swallows any internal error so a cache operation can always return its result even if recording fails.

Example::

tracker = StatsTracker()
tracker.record_exact_hit()
tracker.record_miss()
print(tracker.get_stats().hit_rate)  # 0.5

record_exact_hit ¶

record_exact_hit() -> None

Record a hit served by the exact cache.

Increments the exact-hit counter, the aggregate hit counter, and the total-request counter. Any internal error is swallowed so recording is never fatal to the caller.

record_semantic_hit ¶

record_semantic_hit() -> None

Record a hit served by the semantic cache.

Increments the semantic-hit counter, the aggregate hit counter, and the total-request counter. Any internal error is swallowed so recording is never fatal to the caller.

record_hit ¶

record_hit() -> None

Record an exact hit. Alias for :meth:record_exact_hit.

Retained for backward compatibility with Phase 1 callers.

record_miss ¶

record_miss() -> None

Increment the miss counter and total-request counter.

Any internal error is swallowed so recording is never fatal.

record_error ¶

record_error() -> None

Increment the error counter (does not affect total_requests).

Any internal error is swallowed so recording is never fatal.

get_stats ¶

get_stats() -> CacheStats

Return an immutable snapshot of the current counters.

Returns:

Name	Type	Description
`A`	`CacheStats`	class:`CacheStats` dataclass with the current values.

reset ¶

reset() -> None

Reset all counters to zero.

Exceptions¶

memory_reuse.exceptions ¶

Custom exceptions for the memory-reuse package.

AgentMemoryError ¶

Bases: Exception

Base exception for all memory-reuse errors.

All exceptions raised by this library inherit from this class, making it easy to catch any library error with a single handler.

BackendConnectionError ¶

Bases: AgentMemoryError

Raised when the cache backend cannot be reached.

Typically wraps connection failures from Redis or other network backends. Check your redis_url and network connectivity.

ScopeViolationError ¶

Bases: AgentMemoryError

Raised when user-scoped data would be cached under the global scope.

This is a safety guard: if you call a cache operation with scope='user' but no user_id is available in the current context, this exception is raised rather than silently caching data that could be shared across users.

InvalidTTLError ¶

Bases: AgentMemoryError

Raised when a TTL value is not valid.

TTL must be a positive integer (seconds) or None for no expiry.

BackendNotAvailableError ¶

Bases: AgentMemoryError

Raised when the requested backend is not importable or configured.

For example, requesting the redis backend without the redis package installed will raise this exception.

EmbeddingProviderError ¶

Bases: AgentMemoryError

Raised when an embedding provider cannot be used.

Typically raised when the provider's optional dependency is not installed (naming the extra to install, e.g. pip install "memory-reuse[semantic]") or when the provider's backing model or API call fails.

ProviderMismatchError ¶

Bases: AgentMemoryError

Raised when embeddings from different providers or models are mixed.

Vectors are namespaced by provider:model. Comparing vectors from incompatible providers or models is refused rather than silently producing meaningless similarity scores.

ConfigurationError ¶

Bases: AgentMemoryError

Raised when a :class:CacheConfig value is invalid.

For example, a similarity_threshold outside [0.0, 1.0] or enabling semantic_enabled without selecting an embedding_provider.

MemoryCache & configuration¶

MemoryCache¶

memory_reuse.core.MemoryCache ¶

stats property ¶

from_env classmethod ¶

set_context ¶

clear_context ¶

reset_stats ¶

lookup async ¶

store async ¶

ping async ¶

close async ¶

flush async ¶

CacheConfig¶

memory_reuse.config.CacheConfig dataclass ¶

__post_init__ ¶

from_env classmethod ¶

Statistics¶

memory_reuse.stats.CacheStats dataclass ¶

hit_rate property ¶

to_dict ¶

memory_reuse.stats.StatsTracker ¶

record_exact_hit ¶

record_semantic_hit ¶

record_hit ¶

record_miss ¶

record_error ¶

get_stats ¶

reset ¶

Exceptions¶

memory_reuse.exceptions ¶

AgentMemoryError ¶

BackendConnectionError ¶

ScopeViolationError ¶

InvalidTTLError ¶

BackendNotAvailableError ¶

EmbeddingProviderError ¶

ProviderMismatchError ¶

ConfigurationError ¶

stats `property` ¶

from_env `classmethod` ¶

lookup `async` ¶

store `async` ¶

ping `async` ¶

close `async` ¶

flush `async` ¶

memory_reuse.config.CacheConfig `dataclass` ¶

from_env `classmethod` ¶

memory_reuse.stats.CacheStats `dataclass` ¶

hit_rate `property` ¶