Vector index¶

VectorIndex (interface) & data models¶

memory_reuse.vector.base ¶

Abstract vector-index interface and data models for the semantic cache.

The semantic cache stores query embeddings and searches them by cosine similarity. This lives in a dedicated :class:VectorIndex abstraction rather than on :class:~memory_reuse.backends.base.AbstractBackend because vector search has a fundamentally different shape (search-by-similarity rather than get-by-key), and keeping it separate leaves the Phase 1 backend interface untouched.

Namespaces and record ids¶

Every stored vector belongs to a namespace that encodes its cache scope so that a search never crosses scope boundaries:

"global" — the global scope, shared by all callers.
"user:<user_id>" — a per-user scope.
"session:<session_id>" — a per-session scope.

A search only ever compares against records in the single namespace it is given, which enforces the same global/user/session isolation as the exact cache.

Within a namespace each record is keyed by a record id derived from the embedding provider identity and the query text::

record_id = hash_value([provider_model, query_text])

Because the id is deterministic, re-storing the same query_text under the same provider_model overwrites the existing record rather than creating a duplicate.

Provider consistency and expiry¶

The provider_model carried by every record namespaces vectors by their origin. Implementations validate it on both :meth:VectorIndex.add and :meth:VectorIndex.search: an operation whose provider_model differs from the records already present in a namespace must raise :class:~memory_reuse.exceptions.ProviderMismatchError rather than compare vectors of incompatible dimensionality or origin.

Implementations must also filter out records whose expires_at has passed on read, so an expired entry is never returned as a match even if a backend's native expiry mechanism fails.

VectorRecord `dataclass` ¶

VectorRecord(vector: list[float], value: bytes, provider_model: str, expires_at: float | None)

A stored embedding together with its cached result and metadata.

Attributes:

Name	Type	Description
`vector`	`list[float]`	The embedding vector produced by the embedding provider.
`value`	`bytes`	The cached result, gzip-compressed JSON bytes as produced by :func:`memory_reuse._utils.serialize_value`.
`provider_model`	`str`	The stable `"provider:model"` identity of the embedding provider that produced `vector` (for example `"openai:text-embedding-3-small"`). Used to namespace vectors by origin and reject mismatched comparisons.
`expires_at`	`float \| None`	The monotonic timestamp (as returned by :func:`time.monotonic`) after which this record is considered expired and must not be returned as a match, or `None` for a record that never expires.

VectorMatch `dataclass` ¶

VectorMatch(score: float, value: bytes)

A single similarity-search result.

Attributes:

Name	Type	Description
`score`	`float`	Cosine similarity normalised to `[0.0, 1.0]` (see :func:`memory_reuse._utils.cosine_similarity`), where `1.0` means identical and `0.0` means opposite.
`value`	`bytes`	The cached result of the matched record, gzip-compressed JSON bytes as produced by :func:`memory_reuse._utils.serialize_value`.

VectorIndex ¶

Bases: ABC

Interface that all vector indexes must implement.

A vector index stores :class:VectorRecord entries partitioned by namespace and supports nearest-neighbour search by cosine similarity within a single namespace.

Every method is a coroutine so that network-bound implementations (for example a Redis-backed index) can be awaited without blocking the event loop, while an in-process index simply returns immediately.

Implementors must:

never let a :meth:search return records from a namespace other than the one requested (scope isolation);
validate provider_model consistency on :meth:add and :meth:search, raising :class:~memory_reuse.exceptions.ProviderMismatchError on mismatch;
filter out records whose expires_at has passed on read.

add `abstractmethod` `async` ¶

add(namespace: str, record_id: str, record: VectorRecord) -> None

Store or overwrite a record in a namespace.

If a record already exists for record_id in namespace it is overwritten, so re-adding an identical query updates rather than duplicates the entry.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace, one of `"global"`, `"user:<user_id>"`, or `"session:<session_id>"`.	required
`record_id`	`str`	The deterministic record id, typically `hash_value([provider_model, query_text])`.	required
`record`	`VectorRecord`	The :class:`VectorRecord` to store.	required

Raises:

Type	Description
`ProviderMismatchError`	If `record.provider_model` differs from the `provider_model` of records already present in `namespace`.

search `abstractmethod` `async` ¶

search(namespace: str, query: list[float], provider_model: str, top_k: int = 1) -> list[VectorMatch]

Search a namespace for the closest records by cosine similarity.

Only records within namespace are considered, so a search never crosses scope boundaries. Expired records are filtered out before scoring.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to search within.	required
`query`	`list[float]`	The query embedding vector.	required
`provider_model`	`str`	The `"provider:model"` identity of the embedding that produced `query`. Must match the records stored in `namespace`.	required
`top_k`	`int`	The maximum number of matches to return, highest score first.	`1`

Returns:

Type	Description
`list[VectorMatch]`	Up to `top_k` :class:`VectorMatch` results ordered by descending
`list[VectorMatch]`	similarity score. An empty list when the namespace holds no
`list[VectorMatch]`	(non-expired) records.

Raises:

Type	Description
`ProviderMismatchError`	If `provider_model` differs from the `provider_model` of records already present in `namespace`.

delete_namespace `abstractmethod` `async` ¶

delete_namespace(namespace: str) -> None

Remove every record stored under a namespace.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to clear. A no-op if the namespace holds no records.	required

flush `abstractmethod` `async` ¶

flush() -> None

Delete all records managed by this index across every namespace.

Use with caution in production — this removes every stored vector.

expiry_for_ttl ¶

expiry_for_ttl(ttl: int | None) -> float | None

Compute the expires_at value for a record with a given TTL.

Different index implementations interpret :attr:VectorRecord.expires_at against different clocks — the in-memory index uses :func:time.monotonic while the Redis index uses wall-clock :func:time.time. Callers that only know a TTL (such as the semantic cache) delegate to this method so the produced expires_at matches whatever clock the concrete index reads on expiry.

The default implementation uses :func:time.monotonic, matching the in-memory index; implementations that expire against a different clock must override it.

Parameters:

Name	Type	Description	Default
`ttl`	`int \| None`	Time-to-live in seconds, or `None` for a record that never expires.	required

Returns:

Type	Description
`float \| None`	The `expires_at` timestamp on this index's clock, or `None` when
`float \| None`	`ttl` is `None`.

InMemoryVectorIndex¶

memory_reuse.vector.memory.InMemoryVectorIndex ¶

InMemoryVectorIndex(max_vectors_per_namespace: int = _DEFAULT_MAX_VECTORS)

Bases: VectorIndex

Fully in-memory vector index — no external dependencies required.

Features:

Brute-force cosine search — every non-expired record in a namespace is scored with :func:memory_reuse._utils.cosine_similarity and the top top_k are returned, highest score first.
Namespace isolation — records are partitioned by namespace; a search never crosses namespace boundaries.
Provider consistency — provider_model is validated on add and search; a mismatch raises :class:~memory_reuse.exceptions.ProviderMismatchError.
Expiry safety — expired records are filtered (and removed) on read.
LRU eviction — when a namespace reaches max_vectors_per_namespace the least-recently-used record is dropped to make room.
Async-safe — an :class:asyncio.Lock serialises all mutations.

Parameters:

Name	Type	Description	Default
`max_vectors_per_namespace`	`int`	Maximum number of records to hold per namespace before LRU eviction applies. Defaults to 10 000.	`_DEFAULT_MAX_VECTORS`

Example::

index = InMemoryVectorIndex(max_vectors_per_namespace=500)
await index.add("user:alice", record_id, record)
matches = await index.search("user:alice", query_vec, "fake:m", top_k=1)

add `async` ¶

add(namespace: str, record_id: str, record: VectorRecord) -> None

Store or overwrite a record in a namespace.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace, one of `"global"`, `"user:<user_id>"`, or `"session:<session_id>"`.	required
`record_id`	`str`	The deterministic record id, typically `hash_value([provider_model, query_text])`.	required
`record`	`VectorRecord`	The :class:`VectorRecord` to store.	required

Raises:

Type	Description
`ProviderMismatchError`	If `record.provider_model` differs from the `provider_model` of the live (non-expired) records already present in `namespace`.

search `async` ¶

search(namespace: str, query: list[float], provider_model: str, top_k: int = 1) -> list[VectorMatch]

Search a namespace for the closest records by cosine similarity.

Only records in namespace are considered, so a search never crosses scope boundaries. Expired records are filtered out (and removed) before scoring.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to search within.	required
`query`	`list[float]`	The query embedding vector.	required
`provider_model`	`str`	The `"provider:model"` identity of the embedding that produced `query`.	required
`top_k`	`int`	Maximum number of matches to return, highest score first.	`1`

Returns:

Type	Description
`list[VectorMatch]`	Up to `top_k` :class:`VectorMatch` results ordered by descending
`list[VectorMatch]`	similarity score, or an empty list when the namespace holds no
`list[VectorMatch]`	live records.

Raises:

Type	Description
`ProviderMismatchError`	If `provider_model` differs from the `provider_model` of the live records present in `namespace`.

delete_namespace `async` ¶

delete_namespace(namespace: str) -> None

Remove every record stored under a namespace.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to clear. A no-op if the namespace holds no records.	required

flush `async` ¶

flush() -> None

Delete all records across every namespace.

namespace_size ¶

namespace_size(namespace: str) -> int

Return the number of records currently held in namespace.

Includes records that are expired but not yet purged. Intended for tests and introspection.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The namespace to measure.	required

Returns:

Type	Description
`int`	The record count, or `0` if the namespace is unknown.

RedisVectorIndex¶

memory_reuse.vector.redis.RedisVectorIndex ¶

RedisVectorIndex(url: str, *, index_name: str = 'memreuse_vec_idx', max_scan_candidates: int = _DEFAULT_MAX_SCAN_CANDIDATES, max_connections: int = _MAX_CONNECTIONS)

Bases: VectorIndex

Persistent vector index backed by Redis.

Requires the optional redis extra::

pip install memory-reuse[redis]

The connection is established lazily on the first operation and, on Redis Stack, the search index is created on demand. Connection errors are converted to :class:~memory_reuse.exceptions.BackendConnectionError so callers need not handle redis-specific exceptions.

Parameters:

Name	Type	Description	Default
`url`	`str`	Redis connection URL (e.g. `redis://localhost:6379/0`). Prefer reading this from the `MEMORY_REUSE_REDIS_URL` environment variable rather than hardcoding it.	required
`index_name`	`str`	Name of the Redis Stack search index created over the record hashes. Defaults to `"memreuse_vec_idx"`.	`'memreuse_vec_idx'`
`max_scan_candidates`	`int`	Maximum number of records a namespace may hold for the in-process fallback (used only when the search module is absent). Searching a larger namespace raises :class:`~memory_reuse.exceptions.BackendConnectionError`. Defaults to 10 000.	`_DEFAULT_MAX_SCAN_CANDIDATES`
`max_connections`	`int`	Maximum size of the underlying connection pool. Defaults to 20.	`_MAX_CONNECTIONS`

Example::

import os
index = RedisVectorIndex(url=os.environ["MEMORY_REUSE_REDIS_URL"])
await index.add("user:alice", record_id, record)
matches = await index.search("user:alice", query_vec, "openai:m", top_k=1)

add `async` ¶

add(namespace: str, record_id: str, record: VectorRecord) -> None

Store or overwrite a record in a namespace.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace, one of `"global"`, `"user:<user_id>"`, or `"session:<session_id>"`.	required
`record_id`	`str`	The deterministic record id, typically `hash_value([provider_model, query_text])`.	required
`record`	`VectorRecord`	The :class:`VectorRecord` to store.	required

Raises:

Type	Description
`ProviderMismatchError`	If `record.provider_model` differs from the `provider_model` of the live records already present in `namespace`.
`BackendConnectionError`	On Redis connectivity failure.

search `async` ¶

search(namespace: str, query: list[float], provider_model: str, top_k: int = 1) -> list[VectorMatch]

Search a namespace for the closest records by cosine similarity.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to search within.	required
`query`	`list[float]`	The query embedding vector.	required
`provider_model`	`str`	The `"provider:model"` identity of the embedding that produced `query`.	required
`top_k`	`int`	Maximum number of matches to return, highest score first.	`1`

Returns:

Type	Description
`list[VectorMatch]`	Up to `top_k` :class:`VectorMatch` results ordered by descending
`list[VectorMatch]`	similarity score, or an empty list when the namespace holds no live
`list[VectorMatch]`	records.

Raises:

Type	Description
`ProviderMismatchError`	If `provider_model` differs from the `provider_model` of the live records present in `namespace`.
`BackendConnectionError`	On Redis connectivity failure, or when the in-process fallback would exceed `max_scan_candidates`.

delete_namespace `async` ¶

delete_namespace(namespace: str) -> None

Remove every record stored under a namespace.

Parameters:

Name	Type	Description	Default
`namespace`	`str`	The scope namespace to clear. A no-op if the namespace holds no records.	required

Raises:

Type	Description
`BackendConnectionError`	On Redis connectivity failure.

flush `async` ¶

flush() -> None

Delete every record managed by this index across all namespaces.

Only keys under the memreuse:vec: prefix are removed; unrelated keys in the same database are left untouched.

Raises:

Type	Description
`BackendConnectionError`	On Redis connectivity failure.

close `async` ¶

close() -> None

Close the connection pool gracefully.

Call this during application shutdown to release Redis connections.

expiry_for_ttl ¶

expiry_for_ttl(ttl: int | None) -> float | None

Compute a wall-clock expires_at for a record with a given TTL.

The Redis index stores and reads expires_at as a wall-clock timestamp (:func:time.time), so this overrides the monotonic default on :class:~memory_reuse.vector.base.VectorIndex.

Parameters:

Name	Type	Description	Default
`ttl`	`int \| None`	Time-to-live in seconds, or `None` for no expiry.	required

Returns:

Type	Description
`float \| None`	The wall-clock `expires_at` timestamp, or `None` when `ttl` is
`float \| None`	`None`.

Vector index¶

VectorIndex (interface) & data models¶

memory_reuse.vector.base ¶

Namespaces and record ids¶

Provider consistency and expiry¶

VectorRecord dataclass ¶

VectorMatch dataclass ¶

VectorIndex ¶

add abstractmethod async ¶

search abstractmethod async ¶

delete_namespace abstractmethod async ¶

flush abstractmethod async ¶

expiry_for_ttl ¶

InMemoryVectorIndex¶

memory_reuse.vector.memory.InMemoryVectorIndex ¶

add async ¶

search async ¶

delete_namespace async ¶

flush async ¶

namespace_size ¶

RedisVectorIndex¶

memory_reuse.vector.redis.RedisVectorIndex ¶

add async ¶

search async ¶

delete_namespace async ¶

flush async ¶

close async ¶

expiry_for_ttl ¶

VectorRecord `dataclass` ¶

VectorMatch `dataclass` ¶

add `abstractmethod` `async` ¶

search `abstractmethod` `async` ¶

delete_namespace `abstractmethod` `async` ¶

flush `abstractmethod` `async` ¶

add `async` ¶

search `async` ¶

delete_namespace `async` ¶

flush `async` ¶

add `async` ¶

search `async` ¶

delete_namespace `async` ¶

flush `async` ¶

close `async` ¶