plexus.analysis.embedding_cache module

S3-backed embedding cache and embedding service.

Maps hash(normalized_text + model_id + preprocessing_version) to embedding vectors stored in S3. Re-indexing the vector store does not require re-embedding.

class plexus.analysis.embedding_cache.EmbeddingCache(bucket_name: str | None = None, s3_client=None)

Bases: object

S3-backed cache for embedding vectors. Cache key = SHA-256(normalize(text) + “|” + model_id + “|” + preprocessing_version). S3 path = embeddings/{model_id}/{key_prefix}/{key}.json

__init__(bucket_name: str | None = None, s3_client=None)

get(model_id: str, key: str) → ndarray | None

Retrieve embedding from S3. Returns None on cache miss or error.

put(model_id: str, key: str, embedding: ndarray) → None

Write embedding to S3. Non-fatal on failure (embedding was computed in memory).

class plexus.analysis.embedding_cache.EmbeddingService(cache: EmbeddingCache | None = None, model_id: str = 'all-MiniLM-L6-v2', preprocessing_version: str = '1')

Bases: object

Batch embedding with cache-first strategy. Checks cache for each text, computes misses via sentence-transformers, writes through.

__init__(cache: EmbeddingCache | None = None, model_id: str = 'all-MiniLM-L6-v2', preprocessing_version: str = '1')

batch_embed(texts: List[str], model_id: str | None = None, preprocessing_version: str | None = None) → List[ndarray]

Embed texts, using cache for hits and model for misses. Preserves original ordering.

plexus.analysis.embedding_cache.cache_key(text: str, model_id: str, preprocessing_version: str) → str

Deterministic cache key: SHA-256(normalize(text) + “|” + model_id + “|” + preprocessing_version).

plexus.analysis.embedding_cache.normalize_text(text: str) → str

Collapse whitespace and lowercase for deterministic cache keys.