Application Programming Interface Reference
Core
Corpus storage and ingestion for Biblicus.
- class biblicus.corpus.Corpus(root)[source]
Local corpus manager for Biblicus.
- Variables:
root (Path) – Corpus root directory.
meta_dir (Path) – Metadata directory under the corpus root.
raw_dir (Path) – Raw item directory under the corpus root.
config (CorpusConfig or None) – Parsed corpus config, if present.
- Parameters:
root (Path)
- property analysis_dir: Path
Location of analysis artifacts for the corpus.
- Returns:
Analysis artifacts directory.
- Return type:
Path
- analysis_run_dir(*, analysis_id, snapshot_id)[source]
Resolve an analysis snapshot directory.
- Parameters:
analysis_id (str) – Analysis backend identifier.
snapshot_id (str) – Analysis snapshot identifier.
- Returns:
Analysis snapshot directory.
- Return type:
Path
- property analysis_runs_dir: Path
Location of analysis snapshot artifacts.
- Returns:
Path to the analysis snapshots directory.
- Return type:
Path
- catalog_generated_at()[source]
Return the catalog generation timestamp.
- Returns:
International Organization for Standardization 8601 timestamp.
- Return type:
str
- property catalog_path: Path
Return the path to the corpus catalog file,
metadata/catalog.json.This is part of the corpus layout contract: callers locate the catalog through this property instead of assembling the path themselves.
- Returns:
Catalog file path.
- Return type:
Path
- create_crawl_id()[source]
Create a new crawl identifier.
- Returns:
Crawl identifier.
- Return type:
str
- delete_extraction_snapshot(*, extractor_id, snapshot_id)[source]
Delete an extraction snapshot directory and its derived artifacts.
- Parameters:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
- Returns:
None.
- Return type:
None
- Raises:
FileNotFoundError – If the extraction snapshot directory does not exist.
- delete_item(item_id)[source]
Delete an item payload and its sidecar metadata from the corpus.
- Parameters:
item_id (str) – Item identifier to delete.
- Returns:
None.
- Return type:
None
- Raises:
KeyError – If the item identifier is not present in the catalog.
- property extracted_dir: Path
Location of extraction artifacts for the corpus.
- Returns:
Extracted artifacts directory.
- Return type:
Path
- extracted_text_path(snapshot_id, item_id)[source]
Resolve the final extracted text artifact of an item in a pipeline extraction snapshot.
The layout is
extracted/pipeline/<snapshot_id>/text/<item_id>.txt. The path is computed from identifiers only; the artifact exists only when the snapshot extracted text for the item.- Parameters:
snapshot_id (str) – Pipeline extraction snapshot identifier.
item_id (str) – Item identifier.
- Returns:
Extracted text artifact path.
- Return type:
Path
- extraction_snapshot_dir(*, extractor_id, snapshot_id)[source]
Resolve an extraction snapshot directory.
- Parameters:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
- Returns:
Extraction snapshot directory.
- Return type:
Path
- property extraction_snapshots_dir: Path
Location of extraction snapshot artifacts.
- Returns:
Path to the extraction snapshots directory.
- Return type:
Path
- classmethod find(start)[source]
Locate a corpus by searching upward from a path.
- Parameters:
start (Path) – Starting path to search.
- Returns:
Located corpus instance.
- Return type:
- Raises:
FileNotFoundError – If no corpus config is found.
- get_item(item_id)[source]
Fetch a catalog item by identifier.
- Parameters:
item_id (str) – Item identifier.
- Returns:
Catalog item.
- Return type:
- Raises:
KeyError – If the item identifier is unknown.
- property graph_dir: Path
Location of graph artifacts for the corpus.
- Returns:
Graph artifacts directory.
- Return type:
Path
- graph_snapshot_dir(*, extractor_id, snapshot_id)[source]
Resolve a graph snapshot directory.
- Parameters:
extractor_id (str) – Graph extractor identifier.
snapshot_id (str) – Graph snapshot identifier.
- Returns:
Graph snapshot directory.
- Return type:
Path
- graph_snapshot_manifest_path(extractor_id, snapshot_id)[source]
Resolve the manifest file of a graph snapshot.
The layout is
graph/<extractor_id>/<snapshot_id>/manifest.json. The path is computed from identifiers only; the manifest exists only after the graph snapshot was built.- Parameters:
extractor_id (str) – Graph extractor identifier.
snapshot_id (str) – Graph snapshot identifier.
- Returns:
Graph snapshot manifest path.
- Return type:
Path
- property graph_snapshots_dir: Path
Location of graph snapshot artifacts.
- Returns:
Path to the graph snapshots directory.
- Return type:
Path
- has_items()[source]
Return whether the corpus catalog contains any items.
- Returns:
True when the catalog has at least one item.
- Return type:
bool
- import_tree(source_root, *, tags=())[source]
Import a folder tree into the corpus, preserving relative paths and provenance.
Imported content must already live under the corpus root. The import registers files in-place and writes sidecars when needed.
- Parameters:
source_root (Path) – Root directory of the folder tree to import.
tags (Sequence[str]) – Tags to associate with imported items.
- Returns:
Import statistics.
- Return type:
dict[str, int]
- Raises:
FileNotFoundError – If the source_root does not exist.
ValueError – If the source root is outside the corpus root.
- ingest_crawled_payload(*, crawl_id, relative_path, data, filename, media_type, source_uri, tags)[source]
Ingest a crawled payload under a crawl import namespace.
- Parameters:
crawl_id (str) – Crawl identifier used to group crawled artifacts.
relative_path (str) – Relative path within the crawl prefix.
data (bytes) – Raw payload bytes.
filename (str) – Suggested filename from the payload metadata.
media_type (str) – Internet Assigned Numbers Authority media type.
source_uri (str) – Source uniform resource identifier (typically an http or https uniform resource locator).
tags (Sequence[str]) – Tags to attach to the stored item.
- Returns:
None.
- Return type:
None
- ingest_item(data, *, filename=None, media_type='application/octet-stream', title=None, tags=(), metadata=None, source_uri='unknown', storage_subdir='imports')[source]
Ingest a single raw item into the corpus.
This is the modality-neutral primitive: callers provide bytes + a media type. Higher-level conveniences (ingest_note, ingest_source, and related methods) build on top.
- Parameters:
data (bytes) – Raw item bytes.
filename (str or None) – Optional filename for the stored item.
media_type (str) – Internet Assigned Numbers Authority media type for the item.
title (str or None) – Optional title metadata.
tags (Sequence[str]) – Tags to associate with the item.
metadata (dict[str, Any] or None) – Optional metadata mapping.
source_uri (str) – Source uniform resource identifier for provenance.
storage_subdir (str or None) – Optional subdirectory under the raw root.
- Returns:
Ingestion result summary.
- Return type:
- Raises:
ValueError – If markdown is not Unicode Transformation Format 8.
IngestCollisionError – If a source uniform resource identifier is already ingested.
- ingest_item_stream(stream, *, filename=None, media_type='application/octet-stream', tags=(), metadata=None, source_uri='unknown', storage_subdir='imports')[source]
Ingest a binary item from a readable stream.
This method is intended for large non-markdown items. It writes bytes to disk incrementally while computing a checksum.
- Parameters:
stream (object) – Readable binary stream.
filename (str or None) – Optional filename for the stored item.
media_type (str) – Internet Assigned Numbers Authority media type for the item.
tags (Sequence[str]) – Tags to associate with the item.
metadata (dict[str, Any] or None) – Optional metadata mapping.
source_uri (str) – Source uniform resource identifier for provenance.
storage_subdir (str or None) – Optional subdirectory under the raw root.
- Returns:
Ingestion result summary.
- Return type:
- Raises:
ValueError – If the media_type is text/markdown.
- ingest_note(text, *, title=None, tags=(), source_uri=None, metadata=None)[source]
Ingest a text note as Markdown.
- Parameters:
text (str) – Note content.
title (str or None) – Optional title metadata.
tags (Sequence[str]) – Tags to associate with the note.
source_uri (str or None) – Optional source uniform resource identifier for provenance.
metadata (dict[str, Any] or None) – Optional metadata mapping.
- Returns:
Ingestion result summary.
- Return type:
- ingest_source(source, *, tags=(), source_uri=None, metadata=None, allow_external=False)[source]
Ingest a file path or uniform resource locator source.
- Parameters:
source (str or Path) – File path or uniform resource locator.
tags (Sequence[str]) – Tags to associate with the item.
source_uri (str or None) – Optional override for the source uniform resource identifier.
metadata (dict[str, Any] or None) – Optional metadata mapping.
allow_external (bool) – Whether to ingest files outside the corpus root by copying them into imports.
- Returns:
Ingestion result summary.
- Return type:
- classmethod init(root, *, force=False)[source]
Initialize a new corpus on disk.
- Parameters:
root (Path) – Corpus root directory.
force (bool) – Whether to overwrite existing config.
- Returns:
Initialized corpus instance.
- Return type:
- Raises:
FileExistsError – If the corpus already exists and force is False.
- latest_extraction_snapshot_reference(*, extractor_id=None)[source]
Return the most recent extraction snapshot reference.
- Parameters:
extractor_id (str or None) – Optional extractor identifier filter.
- Returns:
Latest extraction snapshot reference or None when no snapshots exist.
- Return type:
- property latest_snapshot_id: str | None
Latest retrieval snapshot identifier recorded in the catalog.
- Returns:
Latest snapshot identifier or None.
- Return type:
str or None
- list_extraction_snapshots(*, extractor_id=None)[source]
List extraction snapshots stored under the corpus.
- Parameters:
extractor_id (str or None) – Optional extractor identifier filter.
- Returns:
Summary list entries for each snapshot.
- Return type:
- list_items(*, limit=50)[source]
List items from the catalog.
- Parameters:
limit (int) – Maximum number of items to return.
- Returns:
Catalog items ordered by recency.
- Return type:
list[CatalogItem]
- load_catalog()[source]
Load the current corpus catalog.
- Returns:
Parsed corpus catalog.
- Return type:
- Raises:
FileNotFoundError – If the catalog file does not exist.
ValueError – If the catalog schema is invalid.
- load_extraction_snapshot_manifest(*, extractor_id, snapshot_id)[source]
Load an extraction snapshot manifest from the corpus.
- Parameters:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
- Returns:
Parsed extraction snapshot manifest.
- Return type:
- Raises:
FileNotFoundError – If the manifest file does not exist.
ValueError – If the manifest data is invalid.
- load_snapshot(snapshot_id)[source]
Load a retrieval snapshot manifest by identifier.
- Parameters:
snapshot_id (str) – Snapshot identifier.
- Returns:
Parsed snapshot manifest.
- Return type:
- Raises:
FileNotFoundError – If the snapshot manifest does not exist.
- property name: str
Return the corpus name (directory basename).
- Returns:
Corpus name.
- Return type:
str
- pull_source(*, tag_resolver=None)[source]
Mirror a remote source into the corpus.
- Returns:
Pull summary.
- Return type:
- Raises:
ValueError – If the corpus has no configured remote source.
- Parameters:
tag_resolver (Callable[[str], List[str]] | None)
- purge(*, confirm)[source]
Delete all ingested items and derived files, preserving corpus identity/config.
- Parameters:
confirm (str) – Confirmation string matching the corpus name.
- Returns:
None.
- Return type:
None
- Raises:
ValueError – If the confirmation does not match.
- read_extracted_text(*, extractor_id, snapshot_id, item_id)[source]
Read extracted text for an item from an extraction snapshot, when present.
- Parameters:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
item_id (str) – Item identifier.
- Returns:
Extracted text or None if the artifact does not exist.
- Return type:
str or None
- Raises:
OSError – If the file exists but cannot be read.
- reindex()[source]
Rebuild/refresh the corpus catalog from the current on-disk corpus contents.
This is the core “mutable corpus with re-indexing” loop: edit raw files or sidecars, then reindex to refresh the derived catalog.
- Returns:
Reindex statistics.
- Return type:
dict[str, int]
- Raises:
ValueError – If a markdown file cannot be decoded as Unicode Transformation Format 8.
- property retrieval_dir: Path
Location of retrieval artifacts for the corpus.
- Returns:
Retrieval artifacts directory.
- Return type:
Path
- property snapshots_dir: Path
Location of retrieval snapshot manifests.
- Returns:
Path to the snapshots directory.
- Return type:
Path
- property uri: str
Return the canonical uniform resource identifier for the corpus root.
- Returns:
Corpus uniform resource identifier.
- Return type:
str
- write_snapshot(snapshot)[source]
Persist a retrieval snapshot manifest and update the catalog pointer.
- Parameters:
snapshot (RetrievalSnapshot) – Snapshot manifest to persist.
- Returns:
None.
- Return type:
None
High-level knowledge base workflow for turnkey usage.
- class biblicus.knowledge_base.KnowledgeBase(corpus, retriever_id, snapshot, defaults, _temp_dir)[source]
High-level knowledge base wrapper for turnkey workflows.
- Variables:
corpus (Corpus) – Corpus instance that stores the ingested items.
retriever_id (str) – Retriever identifier used for retrieval.
snapshot (RetrievalSnapshot) – Retrieval snapshot manifest associated with the knowledge base.
defaults (KnowledgeBaseDefaults) – Default configuration used for this knowledge base.
- Parameters:
corpus (Corpus)
retriever_id (str)
snapshot (RetrievalSnapshot)
defaults (KnowledgeBaseDefaults)
_temp_dir (TemporaryDirectory | None)
- context_pack(result, *, join_with='\n\n', max_tokens=None)[source]
Build a context pack from a retrieval result.
- Parameters:
result (RetrievalResult) – Retrieval result to convert into context.
join_with (str) – Join string for evidence blocks.
max_tokens (int or None) – Optional token budget for the context pack.
- Returns:
Context pack text and metadata.
- Return type:
- defaults: KnowledgeBaseDefaults
- classmethod from_folder(folder, *, retriever_id=None, configuration_name=None, query_budget=None, tags=None, corpus_root=None)[source]
Build a knowledge base from a folder of files.
- Parameters:
folder (str or Path) – Folder containing source files.
retriever_id (str or None) – Optional retriever identifier override.
configuration_name (str or None) – Optional configuration name override.
query_budget (QueryBudget or None) – Optional query budget override.
tags (Sequence[str] or None) – Optional tags to apply during import.
corpus_root (str or Path or None) – Optional corpus root override. Must contain the source folder.
- Returns:
Knowledge base instance.
- Return type:
- Raises:
FileNotFoundError – If the folder does not exist.
NotADirectoryError – If the folder is not a directory.
- query(query_text, *, budget=None)[source]
Query the knowledge base for evidence.
- Parameters:
query_text (str) – Query text to execute.
budget (QueryBudget or None) – Optional budget override.
- Returns:
Retrieval result containing evidence.
- Return type:
- retriever_id: str
- snapshot: RetrievalSnapshot
- class biblicus.knowledge_base.KnowledgeBaseDefaults(*, retriever_id='scan', configuration_name='Knowledge base', query_budget=<factory>, tags=<factory>)[source]
Default configuration for a knowledge base workflow.
- Variables:
retriever_id (str) – Retriever identifier to use for retrieval.
configuration_name (str) – Human-readable retrieval configuration name.
query_budget (QueryBudget) – Default query budget to apply to retrieval.
tags (list[str]) – Tags to apply when importing the folder.
- Parameters:
retriever_id (str)
configuration_name (str)
query_budget (QueryBudget)
tags (List[str])
- configuration_name: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- query_budget: QueryBudget
- retriever_id: str
- tags: List[str]
Pydantic models for Biblicus domain concepts.
- class biblicus.models.CatalogItem(*, id, relpath, sha256, bytes, media_type, title=None, tags=<factory>, metadata=<factory>, dates=None, created_at, source_uri=None)[source]
Catalog entry derived from a raw corpus item.
- Variables:
id (str) – Universally unique identifier of the item.
relpath (str) – Relative path to the raw item file.
sha256 (str) – Secure Hash Algorithm 256 digest of the stored bytes.
bytes (int) – Size of the raw item in bytes.
media_type (str) – Internet Assigned Numbers Authority media type for the item.
title (str or None) – Optional human title extracted from metadata.
tags (list[str]) – Tags extracted or supplied for the item.
metadata (dict[str, Any]) – Merged front matter or sidecar metadata.
dates (CatalogItemDates or None) – Canonical date metadata projected from metadata.dates.
created_at (str) – International Organization for Standardization 8601 timestamp when the item was first indexed.
source_uri (str or None) – Optional source uniform resource identifier used at ingestion time.
- Parameters:
id (str)
relpath (str)
sha256 (str)
bytes (Annotated[int, Ge(ge=0)])
media_type (str)
title (str | None)
tags (List[str])
metadata (Dict[str, Any])
dates (CatalogItemDates | None)
created_at (str)
source_uri (str | None)
- bytes: int
- created_at: str
- dates: CatalogItemDates | None
- id: str
- media_type: str
- metadata: Dict[str, Any]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- relpath: str
- sha256: str
- source_uri: str | None
- tags: List[str]
- title: str | None
- class biblicus.models.CatalogItemDates(*, published_at=None, updated_at=None, retrieved_at=None)[source]
Canonical date metadata projected from item metadata.
- Variables:
published_at (str or None) – Original content publication date or timestamp.
updated_at (str or None) – Original content update date or timestamp.
retrieved_at (str or None) – Retrieval timestamp for the stored source.
- Parameters:
published_at (str | None)
updated_at (str | None)
retrieved_at (str | None)
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- published_at: str | None
- retrieved_at: str | None
- updated_at: str | None
- class biblicus.models.CollectionMembership(*, collection_name, corpus_name)[source]
Collection membership metadata for a corpus.
- Variables:
collection_name (str) – Collection name.
corpus_name (str) – Corpus name within the collection.
- Parameters:
collection_name (Annotated[str, MinLen(min_length=1)])
corpus_name (Annotated[str, MinLen(min_length=1)])
- collection_name: str
- corpus_name: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.models.ConfigurationManifest(*, configuration_id, retriever_id, name, created_at, configuration=<factory>, description=None)[source]
Reproducible configuration for a retriever.
- Variables:
configuration_id (str) – Deterministic configuration identifier.
retriever_id (str) – Retriever identifier for the configuration.
name (str) – Human-readable name for the configuration.
created_at (str) – International Organization for Standardization 8601 timestamp for configuration creation.
configuration (dict[str, Any]) – Retriever-specific configuration values.
description (str or None) – Optional human description.
- Parameters:
configuration_id (str)
retriever_id (str)
name (str)
created_at (str)
configuration (Dict[str, Any])
description (str | None)
- configuration: Dict[str, Any]
- configuration_id: str
- created_at: str
- description: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- name: str
- retriever_id: str
- class biblicus.models.CorpusCatalog(*, schema_version, generated_at, corpus_uri, raw_dir='.', latest_run_id=None, latest_snapshot_id=None, items=<factory>, order=<factory>)[source]
Snapshot of the derived corpus catalog.
- Variables:
schema_version (int) – Version of the catalog schema.
generated_at (str) – International Organization for Standardization 8601 timestamp of catalog generation.
corpus_uri (str) – Canonical uniform resource identifier for the corpus root.
raw_dir (str) – Relative path to the raw items folder.
latest_run_id (str or None) – Latest extraction run identifier, if any.
latest_snapshot_id (str or None) – Latest retrieval snapshot identifier, if any.
items (dict[str, CatalogItem]) – Mapping of item IDs to catalog entries.
order (list[str]) – Display order of item IDs (most recent first).
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
generated_at (str)
corpus_uri (str)
raw_dir (str)
latest_run_id (str | None)
latest_snapshot_id (str | None)
items (Dict[str, CatalogItem])
order (List[str])
- corpus_uri: str
- generated_at: str
- items: Dict[str, CatalogItem]
- latest_run_id: str | None
- latest_snapshot_id: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- order: List[str]
- raw_dir: str
- schema_version: int
- class biblicus.models.CorpusConfig(**data)[source]
Canonical on-disk config for a local Biblicus corpus.
- Variables:
schema_version (int) – Version of the corpus config schema.
created_at (str) – International Organization for Standardization 8601 timestamp for corpus creation.
corpus_uri (str) – Canonical uniform resource identifier for the corpus root.
raw_dir (str) – Relative path to the raw items folder.
notes (dict[str, Any] or None) – Optional free-form notes for operators.
hooks (list[HookSpec] or None) – Optional hook specifications for corpus lifecycle events.
collection (CollectionMembership or None) – Optional collection membership metadata.
- Parameters:
data (Any)
- collection: 'CollectionMembership' | None
- corpus_uri: str
- created_at: str
- hooks: List[HookSpec] | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- notes: Dict[str, Any] | None
- raw_dir: str
- schema_version: int
- source: 'RemoteCorpusSourceConfig' | None
- class biblicus.models.Evidence(*, item_id, source_uri=None, media_type, score, rank, text=None, content_ref=None, span_start=None, span_end=None, stage, stage_scores=None, configuration_id, snapshot_id, metadata=<factory>, hash=None)[source]
Structured retrieval evidence returned from a retriever.
- Variables:
item_id (str) – Item identifier that produced the evidence.
source_uri (str or None) – Source uniform resource identifier from ingestion metadata.
media_type (str) – Media type for the evidence item.
score (float) – Retrieval score (higher is better).
rank (int) – Rank within the final evidence list (1-based).
text (str or None) – Optional text payload for the evidence.
content_ref (str or None) – Optional reference for non-text content.
span_start (int or None) – Optional start offset in the source text.
span_end (int or None) – Optional end offset in the source text.
stage (str) – Retrieval stage label (for example, scan, full-text search, rerank).
stage_scores (dict[str, float] or None) – Optional per-stage scores for multi-stage retrieval.
configuration_id (str) – Configuration identifier used to create the snapshot.
snapshot_id (str) – Retrieval snapshot identifier.
metadata (dict[str, Any]) – Optional metadata payload from the catalog item.
hash (str or None) – Optional content hash for provenance.
- Parameters:
item_id (str)
source_uri (str | None)
media_type (str)
score (float)
rank (Annotated[int, Ge(ge=1)])
text (str | None)
content_ref (str | None)
span_start (int | None)
span_end (int | None)
stage (str)
stage_scores (Dict[str, float] | None)
configuration_id (str)
snapshot_id (str)
metadata (Dict[str, Any])
hash (str | None)
- configuration_id: str
- content_ref: str | None
- hash: str | None
- item_id: str
- media_type: str
- metadata: Dict[str, Any]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- rank: int
- score: float
- snapshot_id: str
- source_uri: str | None
- span_end: int | None
- span_start: int | None
- stage: str
- stage_scores: Dict[str, float] | None
- text: str | None
- class biblicus.models.ExtractedText(*, text, producer_extractor_id, source_stage_index=None, confidence=None, metadata=<factory>)[source]
Text payload produced by an extractor plugin.
- Variables:
text (str) – Extracted text content.
producer_extractor_id (str) – Extractor identifier that produced this text.
source_stage_index (int or None) – Optional pipeline stage index where this text originated.
confidence (float or None) – Optional confidence score from 0.0 to 1.0.
metadata (dict[str, Any]) – Optional structured metadata for passing data between pipeline stages.
- Parameters:
text (str)
producer_extractor_id (Annotated[str, MinLen(min_length=1)])
source_stage_index (Annotated[int | None, Ge(ge=1)])
confidence (Annotated[float | None, Ge(ge=0.0), Le(le=1.0)])
metadata (Dict[str, Any])
- confidence: float | None
- metadata: Dict[str, Any]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- producer_extractor_id: str
- source_stage_index: int | None
- text: str
- class biblicus.models.ExtractionSnapshotListEntry(*, extractor_id, snapshot_id, configuration_id, configuration_name, catalog_generated_at, created_at, stats=<factory>)[source]
Summary entry for an extraction snapshot stored in a corpus.
- Variables:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
configuration_id (str) – Deterministic configuration identifier.
configuration_name (str) – Human-readable configuration name.
catalog_generated_at (str) – Catalog timestamp used for the snapshot.
created_at (str) – International Organization for Standardization 8601 timestamp for snapshot creation.
stats (dict[str, object]) – Snapshot statistics.
- Parameters:
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
configuration_id (Annotated[str, MinLen(min_length=1)])
configuration_name (Annotated[str, MinLen(min_length=1)])
catalog_generated_at (Annotated[str, MinLen(min_length=1)])
created_at (Annotated[str, MinLen(min_length=1)])
stats (Dict[str, object])
- catalog_generated_at: str
- configuration_id: str
- configuration_name: str
- created_at: str
- extractor_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- stats: Dict[str, object]
- class biblicus.models.ExtractionSnapshotReference(*, extractor_id, snapshot_id)[source]
Reference to an extraction snapshot.
- Variables:
extractor_id (str) – Extractor plugin identifier.
snapshot_id (str) – Extraction snapshot identifier.
- Parameters:
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
- as_string()[source]
Serialize the reference as a single string.
- Returns:
Reference in the form extractor_id:snapshot_id.
- Return type:
str
- extractor_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- class biblicus.models.ExtractionStageOutput(*, stage_index, extractor_id, status, text=None, text_characters=0, producer_extractor_id=None, source_stage_index=None, confidence=None, metadata=<factory>, error_type=None, error_message=None)[source]
In-memory representation of a pipeline stage output for a single item.
- Variables:
stage_index (int) – One-based pipeline stage index.
extractor_id (str) – Extractor identifier for the stage.
status (str) – Stage status, extracted, skipped, or errored.
text (str or None) – Extracted text content, when produced.
text_characters (int) – Character count of the extracted text.
producer_extractor_id (str or None) – Extractor identifier that produced the text content.
source_stage_index (int or None) – Optional stage index that supplied the text for selection-style extractors.
confidence (float or None) – Optional confidence score from 0.0 to 1.0.
metadata (dict[str, Any]) – Optional structured metadata for passing data between pipeline stages.
error_type (str or None) – Optional error type name for errored stages.
error_message (str or None) – Optional error message for errored stages.
- Parameters:
stage_index (Annotated[int, Ge(ge=1)])
extractor_id (str)
status (str)
text (str | None)
text_characters (Annotated[int, Ge(ge=0)])
producer_extractor_id (str | None)
source_stage_index (Annotated[int | None, Ge(ge=1)])
confidence (Annotated[float | None, Ge(ge=0.0), Le(le=1.0)])
metadata (Dict[str, Any])
error_type (str | None)
error_message (str | None)
- confidence: float | None
- error_message: str | None
- error_type: str | None
- extractor_id: str
- metadata: Dict[str, Any]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- producer_extractor_id: str | None
- source_stage_index: int | None
- stage_index: int
- status: str
- text: str | None
- text_characters: int
- class biblicus.models.IngestResult(*, item_id, relpath, sha256)[source]
Minimal summary for an ingestion event.
- Variables:
item_id (str) – Universally unique identifier assigned to the ingested item.
relpath (str) – Relative path to the raw item file.
sha256 (str) – Secure Hash Algorithm 256 digest of the stored bytes.
- Parameters:
item_id (str)
relpath (str)
sha256 (str)
- item_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- relpath: str
- sha256: str
- class biblicus.models.PipelineAnalysisConfig(*, kind, configuration)[source]
Analysis configuration for a pipeline recipe.
- Variables:
kind (str) – Analysis kind identifier.
configuration (str) – Path to analysis configuration file.
- Parameters:
kind (Annotated[str, MinLen(min_length=1)])
configuration (Annotated[str, MinLen(min_length=1)])
- configuration: str
- kind: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.models.PipelineCorpusSelector(*, path=None, collection=None, selector=None)[source]
Corpus selection for a pipeline recipe.
- Variables:
path (str or None) – Optional corpus path.
collection (str or None) – Optional collection name or path.
selector (str or None) – Optional selector pattern for collection corpora.
- Parameters:
path (str | None)
collection (str | None)
selector (str | None)
- collection: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- path: str | None
- selector: str | None
- class biblicus.models.PipelineExtractionConfig(*, recipe)[source]
Extraction configuration for a pipeline recipe.
- Variables:
recipe (str) – Path to extraction recipe YAML.
- Parameters:
recipe (Annotated[str, MinLen(min_length=1)])
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- recipe: str
- class biblicus.models.PipelineMirrorConfig(*, collection)[source]
Mirror configuration for a pipeline recipe.
- Variables:
collection (str) – Collection path or name to mirror before running.
- Parameters:
collection (Annotated[str, MinLen(min_length=1)])
- collection: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.models.PipelineRecipeConfig(*, corpus, mirror=None, extraction=None, retrieval=None, analysis=None)[source]
Pipeline recipe configuration.
- Variables:
corpus (PipelineCorpusSelector) – Corpus selection information.
mirror (PipelineMirrorConfig or None) – Optional mirror configuration.
extraction (PipelineExtractionConfig or None) – Optional extraction configuration.
retrieval (PipelineRetrievalConfig or None) – Optional retrieval configuration.
analysis (list[PipelineAnalysisConfig] or None) – Optional analysis configuration list.
- Parameters:
corpus (PipelineCorpusSelector)
mirror (PipelineMirrorConfig | None)
extraction (PipelineExtractionConfig | None)
retrieval (PipelineRetrievalConfig | None)
analysis (List[PipelineAnalysisConfig] | None)
- analysis: List[PipelineAnalysisConfig] | None
- corpus: PipelineCorpusSelector
- extraction: PipelineExtractionConfig | None
- mirror: PipelineMirrorConfig | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- retrieval: PipelineRetrievalConfig | None
- class biblicus.models.PipelineRetrievalConfig(*, retriever, configuration)[source]
Retrieval configuration for a pipeline recipe.
- Variables:
retriever (str) – Retriever identifier.
configuration (str) – Path to retriever configuration file.
- Parameters:
retriever (Annotated[str, MinLen(min_length=1)])
configuration (Annotated[str, MinLen(min_length=1)])
- configuration: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- retriever: str
- class biblicus.models.QueryBudget(*, max_total_items, offset=0, maximum_total_characters=None, max_items_per_source=None)[source]
Evidence selection budget for retrieval.
The budget constrains the returned evidence. It intentionally does not change how a backend scores candidates, only how many evidence items are selected and how much text is allowed through.
- Variables:
max_total_items (int) – Maximum number of evidence items to return.
offset (int) – Number of ranked candidates to skip before selecting evidence. This enables simple pagination by re-running the same query with a higher offset.
maximum_total_characters (int or None) – Optional maximum total characters across evidence text.
max_items_per_source (int or None) – Optional cap per source uniform resource identifier.
- Parameters:
max_total_items (Annotated[int, Ge(ge=1)])
offset (Annotated[int, Ge(ge=0)])
maximum_total_characters (Annotated[int | None, Ge(ge=1)])
max_items_per_source (Annotated[int | None, Ge(ge=1)])
- max_items_per_source: int | None
- max_total_items: int
- maximum_total_characters: int | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- offset: int
- class biblicus.models.RemoteCollectionPullResult(*, discovered=0, created=0, mirrored=0, archived=0, errored=0)[source]
Summary of a collection pull operation.
- Variables:
discovered (int) – Number of discovered subfolders or partitions.
created (int) – Number of corpora created.
mirrored (int) – Number of corpora mirrored.
archived (int) – Number of corpora archived.
errored (int) – Number of errors.
- Parameters:
discovered (Annotated[int, Ge(ge=0)])
created (Annotated[int, Ge(ge=0)])
mirrored (Annotated[int, Ge(ge=0)])
archived (Annotated[int, Ge(ge=0)])
errored (Annotated[int, Ge(ge=0)])
- archived: int
- created: int
- discovered: int
- errored: int
- mirrored: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.models.RemoteCorpusCollectionConfig(*, schema_version, created_at, collection_name, source, discovery, corpus_root, auto_create=True, deletion_policy='archive')[source]
Configuration for a remote corpus collection.
- Variables:
schema_version (int) – Version of the collection config schema.
created_at (str) – International Organization for Standardization 8601 timestamp.
collection_name (str) – Collection name.
source (RemoteCorpusSourceConfig) – Remote source configuration.
discovery (RemoteCorpusCollectionDiscovery) – Discovery configuration.
corpus_root (str) – Filesystem path to the corpus root directory.
auto_create (bool) – Whether to auto-create discovered corpora.
deletion_policy (str) – Policy for missing remote folders (archive or delete).
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
created_at (str)
collection_name (Annotated[str, MinLen(min_length=1)])
source (RemoteCorpusSourceConfig)
discovery (RemoteCorpusCollectionDiscovery)
corpus_root (Annotated[str, MinLen(min_length=1)])
auto_create (bool)
deletion_policy (Annotated[str, MinLen(min_length=1)])
- auto_create: bool
- collection_name: str
- corpus_root: str
- created_at: str
- deletion_policy: str
- discovery: RemoteCorpusCollectionDiscovery
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- schema_version: int
- source: RemoteCorpusSourceConfig
- class biblicus.models.RemoteCorpusCollectionDiscovery(*, mode, depth=1, include_root_files=False)[source]
Discovery configuration for a remote collection.
- Variables:
mode (str) – Discovery mode (subfolder or partition).
depth (int) – Subfolder depth to discover.
include_root_files (bool) – Whether to include root files under a reserved corpus.
- Parameters:
mode (Annotated[str, MinLen(min_length=1)])
depth (Annotated[int, Ge(ge=1)])
include_root_files (bool)
- depth: int
- include_root_files: bool
- mode: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.models.RemoteCorpusSourceConfig(*, kind, profile, name=None, bucket=None, container=None, prefix='', folder_url=None)[source]
Configuration for a remote corpus source.
- Variables:
kind (str) – Remote source kind (s3, azure-blob, or google-drive).
profile (str) – Source profile name in user configuration.
name (str or None) – Optional local namespace for storage.
bucket (str or None) – S3 bucket name.
container (str or None) – Azure Blob container name.
prefix (str) – Optional remote prefix to scope the mirror.
folder_url (str or None) – Google Drive folder URL for google-drive sources.
- Parameters:
kind (Annotated[str, MinLen(min_length=1)])
profile (Annotated[str, MinLen(min_length=1)])
name (str | None)
bucket (str | None)
container (str | None)
prefix (str)
folder_url (str | None)
- bucket: str | None
- container: str | None
- folder_url: str | None
- kind: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- name: str | None
- prefix: str
- profile: str
- class biblicus.models.RemoteSourceItem(*, key, source_uri, etag=None, last_modified=None, size, content_type=None)[source]
Remote source object metadata.
- Variables:
key (str) – Remote object key or blob name.
source_uri (str) – Source uniform resource identifier.
etag (str or None) – Optional entity tag for change detection.
last_modified (str or None) – Optional International Organization for Standardization 8601 timestamp.
size (int) – Size of the object in bytes.
content_type (str or None) – Optional media type.
- Parameters:
key (str)
source_uri (str)
etag (str | None)
last_modified (str | None)
size (Annotated[int, Ge(ge=0)])
content_type (str | None)
- content_type: str | None
- etag: str | None
- key: str
- last_modified: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- size: int
- source_uri: str
- class biblicus.models.RemoteSourcePullResult(*, listed=0, downloaded=0, updated=0, skipped=0, pruned=0, errored=0)[source]
Summary of a remote source pull operation.
- Variables:
listed (int) – Number of remote items listed.
downloaded (int) – Number of new items downloaded.
updated (int) – Number of existing items updated.
skipped (int) – Number of items skipped (no change).
pruned (int) – Number of local items pruned.
errored (int) – Number of items that failed to process.
- Parameters:
listed (Annotated[int, Ge(ge=0)])
downloaded (Annotated[int, Ge(ge=0)])
updated (Annotated[int, Ge(ge=0)])
skipped (Annotated[int, Ge(ge=0)])
pruned (Annotated[int, Ge(ge=0)])
errored (Annotated[int, Ge(ge=0)])
- downloaded: int
- errored: int
- listed: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- pruned: int
- skipped: int
- updated: int
- class biblicus.models.RetrievalResult(*, query_text, budget, snapshot_id, configuration_id, retriever_id, generated_at, evidence=<factory>, stats=<factory>)[source]
Retrieval result bundle returned from a retriever query.
- Variables:
query_text (str) – Query text issued against the backend.
budget (QueryBudget) – Evidence selection budget applied to results.
snapshot_id (str) – Retrieval snapshot identifier.
configuration_id (str) – Configuration identifier used for this query.
retriever_id (str) – Retriever identifier used for this query.
generated_at (str) – International Organization for Standardization 8601 timestamp for the query result.
evidence (list[Evidence]) – Evidence objects selected under the budget.
stats (dict[str, Any]) – Backend-specific query statistics.
- Parameters:
query_text (str)
budget (QueryBudget)
snapshot_id (str)
configuration_id (str)
retriever_id (str)
generated_at (str)
evidence (List[Evidence])
stats (Dict[str, Any])
- budget: QueryBudget
- configuration_id: str
- generated_at: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- query_text: str
- retriever_id: str
- snapshot_id: str
- stats: Dict[str, Any]
- class biblicus.models.RetrievalSnapshot(*, snapshot_id, configuration, corpus_uri, catalog_generated_at, created_at, snapshot_artifacts=<factory>, stats=<factory>)[source]
Immutable record of a retrieval snapshot.
- Variables:
snapshot_id (str) – Unique snapshot identifier.
configuration (ConfigurationManifest) – Configuration manifest for this snapshot.
corpus_uri (str) – Canonical uniform resource identifier for the corpus root.
catalog_generated_at (str) – Catalog timestamp used for the snapshot.
created_at (str) – International Organization for Standardization 8601 timestamp for snapshot creation.
snapshot_artifacts (list[str]) – Relative paths to materialized artifacts.
stats (dict[str, Any]) – Retriever-specific snapshot statistics.
- Parameters:
snapshot_id (str)
configuration (ConfigurationManifest)
corpus_uri (str)
catalog_generated_at (str)
created_at (str)
snapshot_artifacts (List[str])
stats (Dict[str, Any])
- catalog_generated_at: str
- configuration: ConfigurationManifest
- corpus_uri: str
- created_at: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_artifacts: List[str]
- snapshot_id: str
- stats: Dict[str, Any]
- biblicus.models.parse_extraction_snapshot_reference(value)[source]
Parse an extraction snapshot reference in the form extractor_id:snapshot_id.
- Parameters:
value (str) – Raw reference string.
- Returns:
Parsed extraction snapshot reference.
- Return type:
- Raises:
ValueError – If the reference is not well formed.
Shared retrieval helpers for Biblicus retrievers.
- biblicus.retrieval.apply_budget(evidence, budget)[source]
Apply a query budget to a ranked evidence list.
- Parameters:
evidence (Iterable[Evidence]) – Ranked evidence iterable (highest score first).
budget (QueryBudget) – Budget constraints to enforce.
- Returns:
Evidence list respecting the budget.
- Return type:
list[Evidence]
- biblicus.retrieval.create_configuration_manifest(*, retriever_id, name, configuration, description=None)[source]
Create a deterministic configuration manifest from a retriever configuration.
- Parameters:
retriever_id (str) – Retriever identifier for the configuration.
name (str) – Human-readable configuration name.
configuration (dict[str, Any]) – Retriever-specific configuration values.
description (str or None) – Optional configuration description.
- Returns:
Deterministic configuration manifest.
- Return type:
- biblicus.retrieval.create_snapshot_manifest(corpus, *, configuration, stats, snapshot_artifacts=None)[source]
Create a retrieval snapshot manifest tied to the current catalog snapshot.
- Parameters:
corpus (Corpus) – Corpus used to generate the snapshot.
configuration (ConfigurationManifest) – Configuration manifest for the snapshot.
stats (dict[str, Any]) – Retriever-specific snapshot statistics.
snapshot_artifacts (list[str] or None) – Optional relative paths to materialized artifacts.
- Returns:
Snapshot manifest.
- Return type:
- biblicus.retrieval.hash_text(text)[source]
Hash a text payload for provenance.
- Parameters:
text (str) – Text to hash.
- Returns:
Secure Hash Algorithm 256 hex digest.
- Return type:
str
Context pack building for Biblicus.
A context pack is the text that your application sends to a large language model. Biblicus produces a context pack from structured retrieval results so that evidence remains a stable contract while context formatting remains an explicit policy surface.
- class biblicus.context.CharacterBudget(*, max_characters)[source]
Character budget for a context pack.
- Variables:
max_characters (int) – Maximum characters permitted for the final context pack text.
- Parameters:
max_characters (Annotated[int, Ge(ge=1)])
- max_characters: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.context.ContextBlock(*, block_id, section, text, required=False, priority=0, metadata=None)[source]
Ordered context block used outside retrieval-specific evidence flows.
- Variables:
block_id (str) – Stable block identifier.
section (str) – Section key such as doctrine, taxonomy, desk_memory, or fresh_evidence.
text (str) – Text payload for the block.
required (bool) – Whether this block is mandatory even when budgets are tight.
priority (int) – Higher values indicate more important blocks within the same order.
metadata (dict[str, object] or None) – Optional source/debug metadata for the block.
- Parameters:
block_id (Annotated[str, MinLen(min_length=1)])
section (Annotated[str, MinLen(min_length=1)])
text (Annotated[str, MinLen(min_length=1)])
required (bool)
priority (int)
metadata (Dict[str, object] | None)
- block_id: str
- metadata: Dict[str, object] | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- priority: int
- required: bool
- section: str
- text: str
- class biblicus.context.ContextBlockBuildRequest(*, blocks=<factory>, join_with='\n\n', max_tokens=None, max_characters=None, section_budgets=<factory>, token_counter=<factory>)[source]
Request to build a budgeted context pack from ordered context blocks.
- Variables:
blocks (list[ContextBlock]) – Ordered context blocks.
join_with (str) – Separator inserted between blocks.
max_tokens (int or None) – Optional overall token budget.
max_characters (int or None) – Optional overall character budget.
section_budgets (list[ContextSectionBudget]) – Optional per-section token share budgets.
token_counter (TokenCounter) – Token counter configuration.
- Parameters:
blocks (List[ContextBlock])
join_with (str)
max_tokens (Annotated[int | None, Ge(ge=1)])
max_characters (Annotated[int | None, Ge(ge=1)])
section_budgets (List[ContextSectionBudget])
token_counter (TokenCounter)
- blocks: List[ContextBlock]
- join_with: str
- max_characters: int | None
- max_tokens: int | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- section_budgets: List[ContextSectionBudget]
- token_counter: TokenCounter
- class biblicus.context.ContextBlockBuildResult(*, text, included_blocks=<factory>, dropped_blocks=<factory>, section_token_counts=<factory>, total_tokens, total_characters, max_tokens=None, max_characters=None)[source]
Result of building a budgeted context pack from ordered context blocks.
- Variables:
text (str) – Final context-pack text.
included_blocks (list[ContextBlock]) – Blocks included in the final output.
dropped_blocks (list[ContextBlock]) – Blocks removed during budget fitting.
section_token_counts (dict[str, int]) – Token counts per included section.
total_tokens (int) – Total token count for the final text.
total_characters (int) – Total character count for the final text.
max_tokens (int or None) – Applied overall token budget.
max_characters (int or None) – Applied overall character budget.
- Parameters:
text (str)
included_blocks (List[ContextBlock])
dropped_blocks (List[ContextBlock])
section_token_counts (Dict[str, int])
total_tokens (Annotated[int, Ge(ge=0)])
total_characters (Annotated[int, Ge(ge=0)])
max_tokens (Annotated[int | None, Ge(ge=1)])
max_characters (Annotated[int | None, Ge(ge=1)])
- dropped_blocks: List[ContextBlock]
- included_blocks: List[ContextBlock]
- max_characters: int | None
- max_tokens: int | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- section_token_counts: Dict[str, int]
- text: str
- total_characters: int
- total_tokens: int
- class biblicus.context.ContextPack(**data)[source]
Context pack derived from retrieval evidence.
- Variables:
text (str) – Context pack text suitable for inclusion in a model call.
evidence_count (int) – Number of evidence blocks included in the context pack.
blocks (list[ContextPackBlock]) – Structured blocks that produced the context pack.
- Parameters:
data (Any)
- blocks: List['ContextPackBlock']
- evidence_count: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- text: str
- class biblicus.context.ContextPackBlock(*, evidence_item_id, text, metadata=None)[source]
A single context pack block derived from one evidence item.
- Variables:
evidence_item_id (str) – Item identifier that produced this block.
text (str) – Text included in this block.
metadata (dict[str, object] or None) – Optional metadata included with the block.
- Parameters:
evidence_item_id (Annotated[str, MinLen(min_length=1)])
text (Annotated[str, MinLen(min_length=1)])
metadata (Dict[str, object] | None)
- evidence_item_id: str
- metadata: Dict[str, object] | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- text: str
- class biblicus.context.ContextPackPolicy(*, join_with='\n\n', ordering='rank', include_metadata=False, metadata_fields=None)[source]
Policy that controls how evidence becomes context pack text.
- Variables:
join_with (str) – Separator inserted between evidence text blocks.
ordering (str) – Evidence ordering policy (rank, score, or source).
include_metadata (bool) – Whether to include evidence metadata lines in each block.
metadata_fields (list[str] or None) – Optional evidence metadata fields to include.
- Parameters:
join_with (str)
ordering (Annotated[str, MinLen(min_length=1)])
include_metadata (bool)
metadata_fields (List[str] | None)
- include_metadata: bool
- join_with: str
- metadata_fields: List[str] | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- ordering: str
- class biblicus.context.ContextSectionBudget(*, section, share)[source]
Token share budget for one logical section of an assembled context pack.
- Variables:
section (str) – Section key.
share (float) – Fraction of the total token budget reserved for the section.
- Parameters:
section (Annotated[str, MinLen(min_length=1)])
share (Annotated[float, Gt(gt=0), Le(le=1)])
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- section: str
- class biblicus.context.TokenBudget(*, max_tokens)[source]
Token budget for a context pack.
- Variables:
max_tokens (int) – Maximum tokens permitted for the final context pack text.
- Parameters:
max_tokens (Annotated[int, Ge(ge=1)])
- max_tokens: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.context.TokenCounter(*, tokenizer_id='naive-whitespace')[source]
Token counter configuration for token budget fitting.
This is a lightweight model wrapper so token fitting remains explicit and testable even when the underlying tokenizer is provided by an optional dependency.
- Variables:
tokenizer_id (str) – Tokenizer identifier (for example, naive-whitespace).
- Parameters:
tokenizer_id (Annotated[str, MinLen(min_length=1)])
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- tokenizer_id: str
- biblicus.context.build_context_pack(result, *, policy)[source]
Build a context pack from a retrieval result using an explicit policy.
- Parameters:
result (RetrievalResult) – Retrieval result containing ranked evidence.
policy (ContextPackPolicy) – Policy controlling how evidence text is joined.
- Returns:
Context pack containing concatenated evidence text.
- Return type:
- biblicus.context.build_context_pack_from_blocks(request)[source]
Build a deterministic context pack from ordered context blocks.
The algorithm preserves caller-provided order. It first enforces per-section token share budgets, then overall character and token budgets by dropping trailing non-required blocks.
- Parameters:
request (ContextBlockBuildRequest) – Context block build request.
- Returns:
Budgeted context-pack result.
- Return type:
- biblicus.context.count_tokens(text, *, tokenizer_id)[source]
Count tokens in a text using a tokenizer identifier.
The default tokenizer is naive-whitespace, which counts whitespace-separated tokens.
- Parameters:
text (str) – Text payload to count.
tokenizer_id (str) – Tokenizer identifier.
- Returns:
Token count.
- Return type:
int
- Raises:
KeyError – If the tokenizer identifier is unknown.
- biblicus.context.fit_context_pack_to_character_budget(context_pack, *, policy, character_budget)[source]
Fit a context pack to a character budget by dropping trailing blocks.
- Parameters:
context_pack (ContextPack) – Context pack to fit.
policy (ContextPackPolicy) – Policy controlling how blocks are joined into text.
character_budget (CharacterBudget) – Character budget to enforce.
- Returns:
Fitted context pack.
- Return type:
- biblicus.context.fit_context_pack_to_token_budget(context_pack, *, policy, token_budget, token_counter=None)[source]
Fit a context pack to a token budget by dropping trailing blocks.
This function is deterministic. It never rewrites block text. It only removes blocks from the end of the block list until the token budget is met.
- Parameters:
context_pack (ContextPack) – Context pack to fit.
policy (ContextPackPolicy) – Policy controlling how blocks are joined into text.
token_budget (TokenBudget) – Token budget to enforce.
token_counter (TokenCounter or None) – Optional token counter configuration.
- Returns:
Fitted context pack.
- Return type:
Evaluation and benchmarking tools for extraction pipelines.
This module provides tools for quantifying the performance of OCR and other extraction pipelines against ground truth data.
- class biblicus.evaluation.BenchmarkConfig(benchmark_name, categories, pipelines, aggregate_weights, output_dir=PosixPath('results'))[source]
Configuration for a complete benchmark run.
- Parameters:
benchmark_name (str)
categories (Dict[str, CategoryConfig])
pipelines (List[Path])
aggregate_weights (Dict[str, float])
output_dir (Path)
- aggregate_weights: Dict[str, float]
- benchmark_name: str
- categories: Dict[str, CategoryConfig]
- classmethod load(config_path)[source]
Load benchmark configuration from YAML file.
- Parameters:
config_path (Path) – Path to configuration file.
- Returns:
Loaded configuration.
- Return type:
- output_dir: Path = PosixPath('results')
- pipelines: List[Path]
- class biblicus.evaluation.BenchmarkReport(evaluation_timestamp, corpus_path, pipeline_configuration, total_documents, avg_precision, avg_recall, avg_f1, median_precision, median_recall, median_f1, min_f1, max_f1, avg_word_error_rate, avg_sequence_accuracy, avg_lcs_ratio, median_word_error_rate, median_sequence_accuracy, median_lcs_ratio, avg_bigram_overlap, avg_trigram_overlap, processing_time_seconds, per_document_results)[source]
Aggregate benchmark results across multiple documents.
- Parameters:
evaluation_timestamp (str)
corpus_path (str)
pipeline_configuration (Dict[str, Any])
total_documents (int)
avg_precision (float)
avg_recall (float)
avg_f1 (float)
median_precision (float)
median_recall (float)
median_f1 (float)
min_f1 (float)
max_f1 (float)
avg_word_error_rate (float)
avg_sequence_accuracy (float)
avg_lcs_ratio (float)
median_word_error_rate (float)
median_sequence_accuracy (float)
median_lcs_ratio (float)
avg_bigram_overlap (float)
avg_trigram_overlap (float)
processing_time_seconds (float)
per_document_results (List[Dict[str, Any]])
- avg_bigram_overlap: float
- avg_f1: float
- avg_lcs_ratio: float
- avg_precision: float
- avg_recall: float
- avg_sequence_accuracy: float
- avg_trigram_overlap: float
- avg_word_error_rate: float
- corpus_path: str
- evaluation_timestamp: str
- max_f1: float
- median_f1: float
- median_lcs_ratio: float
- median_precision: float
- median_recall: float
- median_sequence_accuracy: float
- median_word_error_rate: float
- min_f1: float
- per_document_results: List[Dict[str, Any]]
- pipeline_configuration: Dict[str, Any]
- processing_time_seconds: float
- total_documents: int
- class biblicus.evaluation.BenchmarkResult(benchmark_version='1.0.0', benchmark_name='', timestamp='', categories=<factory>, aggregate=<factory>, recommendations=<factory>, total_documents=0, total_processing_time_seconds=0.0)[source]
Complete benchmark results across all categories.
- Parameters:
benchmark_version (str)
benchmark_name (str)
timestamp (str)
categories (Dict[str, CategoryResult])
aggregate (Dict[str, float])
recommendations (Dict[str, str])
total_documents (int)
total_processing_time_seconds (float)
- aggregate: Dict[str, float]
- benchmark_name: str = ''
- benchmark_version: str = '1.0.0'
- categories: Dict[str, CategoryResult]
- recommendations: Dict[str, str]
- timestamp: str = ''
- to_markdown(path)[source]
Export results to Markdown file.
- Parameters:
path (Path)
- Return type:
None
- total_documents: int = 0
- total_processing_time_seconds: float = 0.0
- class biblicus.evaluation.BenchmarkRunner(config)[source]
Orchestrates multi-category benchmarking.
- Usage:
config = BenchmarkConfig.load(“configs/benchmark/standard.yaml”) runner = BenchmarkRunner(config) results = runner.run_all() results.to_json(Path(“results/benchmark.json”))
- Parameters:
config (BenchmarkConfig)
- run_all()[source]
Run benchmark across all configured categories.
- Returns:
Complete benchmark results.
- Return type:
- run_category(cat_config)[source]
Run benchmark for a single category.
- Parameters:
cat_config (CategoryConfig) – Category configuration.
- Returns:
Category results.
- Return type:
- class biblicus.evaluation.CategoryConfig(name, dataset, primary_metric, pipelines=<factory>, corpus_path=None, ground_truth_subdir=None, subset_size=None, tags=<factory>)[source]
Configuration for a single benchmark category.
- Parameters:
name (str)
dataset (str)
primary_metric (str)
pipelines (List[object])
corpus_path (Path | None)
ground_truth_subdir (str | None)
subset_size (int | None)
tags (List[str])
- corpus_path: Path | None = None
- dataset: str
- ground_truth_subdir: str | None = None
- name: str
- pipelines: List[object]
- primary_metric: str
- subset_size: int | None = None
- tags: List[str]
- class biblicus.evaluation.CategoryResult(category_name, dataset, documents_evaluated, pipelines, best_pipeline, best_score, primary_metric, primary_score, processing_time_seconds)[source]
Results for a single category.
- Parameters:
category_name (str)
dataset (str)
documents_evaluated (int)
pipelines (List[Dict[str, Any]])
best_pipeline (str)
best_score (float)
primary_metric (str)
primary_score (float)
processing_time_seconds (float)
- best_pipeline: str
- best_score: float
- category_name: str
- dataset: str
- documents_evaluated: int
- pipelines: List[Dict[str, Any]]
- primary_metric: str
- primary_score: float
- processing_time_seconds: float
- class biblicus.evaluation.OCRBenchmark(corpus)[source]
Runs OCR evaluation across multiple documents.
Evaluates extraction snapshots against ground truth data and generates comprehensive reports with per-document and aggregate metrics.
- Parameters:
corpus (Corpus)
- evaluate_extraction(snapshot_reference, ground_truth_dir=None, pipeline_config=None)[source]
Evaluate an extraction snapshot against ground truth.
- Args:
snapshot_reference: Snapshot ID or reference ground_truth_dir: Directory containing ground truth files
(defaults to corpus/metadata/funsd_ground_truth)
pipeline_config: Configuration used to create snapshot
- Returns:
BenchmarkReport with detailed results
- Parameters:
snapshot_reference (str)
ground_truth_dir (Path | None)
pipeline_config (Dict | None)
- Return type:
- class biblicus.evaluation.OCREvaluationResult(document_id, image_path, ground_truth_text, extracted_text, precision, recall, f1_score, character_accuracy, true_positives, false_positives, false_negatives, word_count_gt, word_count_ocr, word_error_rate, sequence_accuracy, lcs_ratio, normalized_edit_distance, bigram_overlap, trigram_overlap)[source]
Results for evaluating a single document.
- Parameters:
document_id (str)
image_path (str)
ground_truth_text (str)
extracted_text (str)
precision (float)
recall (float)
f1_score (float)
character_accuracy (float)
true_positives (int)
false_positives (int)
false_negatives (int)
word_count_gt (int)
word_count_ocr (int)
word_error_rate (float)
sequence_accuracy (float)
lcs_ratio (float)
normalized_edit_distance (float)
bigram_overlap (float)
trigram_overlap (float)
- bigram_overlap: float
- character_accuracy: float
- document_id: str
- extracted_text: str
- f1_score: float
- false_negatives: int
- false_positives: int
- ground_truth_text: str
- image_path: str
- lcs_ratio: float
- normalized_edit_distance: float
- precision: float
- recall: float
- sequence_accuracy: float
- trigram_overlap: float
- true_positives: int
- word_count_gt: int
- word_count_ocr: int
- word_error_rate: float
- biblicus.evaluation.calculate_character_accuracy(ground_truth, extracted)[source]
Calculate character-level accuracy using edit distance.
Uses Levenshtein distance to compute how similar the strings are. Returns 1.0 - (distance / max_length).
- Args:
ground_truth: Expected text extracted: Actual OCR output
- Returns:
Accuracy between 0.0 and 1.0
- Parameters:
ground_truth (str)
extracted (str)
- Return type:
float
- biblicus.evaluation.calculate_ngram_overlap(ground_truth, extracted, n=2)[source]
Calculate n-gram overlap to measure local word ordering.
N-grams capture short sequences of words. High n-gram overlap means the extracted text preserves local word ordering, even if global order differs.
- Args:
ground_truth: Expected text extracted: Actual OCR output n: N-gram size (default 2 for bigrams)
- Returns:
N-gram overlap ratio (0.0 to 1.0)
- Parameters:
ground_truth (str)
extracted (str)
n (int)
- Return type:
float
- biblicus.evaluation.calculate_word_metrics(ground_truth, extracted)[source]
Calculate word-level precision, recall, and F1 score.
Compares word sets after normalization (lowercase, remove punctuation).
- Args:
ground_truth: Expected text extracted: Actual OCR output
- Returns:
Dictionary with precision, recall, f1_score, and counts
- Parameters:
ground_truth (str)
extracted (str)
- Return type:
Dict[str, Any]
- biblicus.evaluation.calculate_word_order_metrics(ground_truth, extracted)[source]
Calculate order-aware metrics that measure reading sequence quality.
These metrics are critical for evaluating layout-aware OCR where the goal is to preserve correct reading order (e.g., left column before right column).
Metrics: - Word Error Rate (WER): Edit distance on word sequences (insertions, deletions, substitutions) - Sequence accuracy: What % of word sequences match exactly - Longest Common Subsequence (LCS): Longest sequence of words in correct order - Normalized edit distance: Word-level Levenshtein distance normalized by length
- Args:
ground_truth: Expected text in correct reading order extracted: Actual OCR output
- Returns:
Dictionary with order-aware metrics
- Parameters:
ground_truth (str)
extracted (str)
- Return type:
Dict[str, Any]
- biblicus.evaluation.evaluate_snapshot(*, corpus, snapshot, dataset, budget)[source]
Evaluate a retrieval snapshot against a dataset.
- Parameters:
corpus (Corpus) – Corpus associated with the snapshot.
snapshot (RetrievalSnapshot) – Retrieval snapshot manifest.
dataset (EvaluationDataset) – Evaluation dataset.
budget (QueryBudget) – Evidence selection budget.
- Returns:
Evaluation result bundle.
- Return type:
EvaluationResult
Typed commands
Typed Python application programming interface for the commands that Papyrus calls.
Each command has one Pydantic request model, one Pydantic response model and one plain function
that takes the request and returns the response. The command-line interface is a thin wrapper over
these functions and prints the response model as JSON. The API_COMMANDS registry lists
every command so JSON Schemas can be generated and published.
- class biblicus.api.ApiCommand(name, group, request_model, response_model, function)[source]
Registry entry tying a command to its request model, response model and function.
- Variables:
name (str) – Stable schema file stem, for example
taxonomy-discover.group (str) – Command-line group, for example
taxonomy.request_model (type[pydantic.BaseModel]) – Pydantic request model class.
response_model (type[pydantic.BaseModel]) – Pydantic response model class.
function (collections.abc.Callable) – Plain function taking the request and returning the response.
- Parameters:
name (str)
group (str)
request_model (Type[BaseModel])
response_model (Type[BaseModel])
function (Callable[[...], Any])
- function: Callable[[...], Any]
Alias for field number 4
- group: str
Alias for field number 1
- name: str
Alias for field number 0
- request_model: Type[BaseModel]
Alias for field number 2
- response_model: Type[BaseModel]
Alias for field number 3
- class biblicus.api.ApiRequestModel[source]
Base class for typed application programming interface requests.
Unknown fields are rejected so a misspelled field is reported instead of ignored.
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.api.ApiResponseModel[source]
Base class for typed application programming interface responses that are not reused library models.
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- biblicus.api.ArticleTextRequest
alias of
ArticleInput
- biblicus.api.ArticleTextResponse
alias of
TransformResult
- biblicus.api.ContextPackBuildBlocksRequest
alias of
ContextBlockBuildRequest
- biblicus.api.ContextPackBuildBlocksResponse
alias of
ContextBlockBuildResult
- class biblicus.api.GraphExtractRequest(*, corpus_path, extractor_id, extraction_snapshot, configuration_name='default', configuration_paths=<factory>, overrides=<factory>, max_items=None, item_timeout_seconds=60.0, item_retry_attempts=0, heartbeat_interval_seconds=10.0)[source]
Request to build a graph extraction snapshot.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
extractor_id (str) – Graph extractor identifier, for example
cooccurrence.extraction_snapshot (ExtractionSnapshotReference) – Extraction snapshot supplying item text.
configuration_name (str) – Human-readable configuration name that keys the snapshot identity.
configuration_paths (list[str]) – Configuration YAML paths; later files override earlier ones.
overrides (list[str]) –
key=valueoverrides applied after composing configurations.max_items (int or None) – Maximum extraction items to process, or None for all.
item_timeout_seconds (float) – Per-item timeout in seconds; zero or less disables the timeout.
item_retry_attempts (int) – Retries per item after a timeout or error.
heartbeat_interval_seconds (float) – Progress heartbeat interval; zero or less disables it.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
extractor_id (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (ExtractionSnapshotReference)
configuration_name (Annotated[str, MinLen(min_length=1)])
configuration_paths (List[str])
overrides (List[str])
max_items (int | None)
item_timeout_seconds (float)
item_retry_attempts (Annotated[int, Ge(ge=0)])
heartbeat_interval_seconds (float)
- configuration_name: str
- configuration_paths: List[str]
- corpus_path: str
- extraction_snapshot: ExtractionSnapshotReference
- extractor_id: str
- heartbeat_interval_seconds: float
- item_retry_attempts: int
- item_timeout_seconds: float
- max_items: int | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- overrides: List[str]
- exception biblicus.api.InvalidRequestError[source]
A request failed validation before any operation ran.
The command-line interface reports it as the
invalid_requestfailure with exit status 2.
- class biblicus.api.QueryRequest(*, corpus_path, query_text, snapshot_id=None, retriever_id=None, budget, reranker_id=None, minimum_score=None)[source]
Request to run a retrieval query against a built snapshot.
The query never builds a snapshot. When
snapshot_idis absent the corpus latest snapshot is used, and a corpus without one is an error.- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
query_text (str) – Query text.
snapshot_id (str or None) – Snapshot identifier, or None for the corpus latest snapshot.
retriever_id (str or None) – Retriever the snapshot must use, or None to accept the snapshot retriever.
budget (QueryBudget) – Evidence selection budget.
reranker_id (str or None) – Optional reranker applied after retrieval.
minimum_score (float or None) – Optional minimum score threshold applied after retrieval.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
query_text (str)
snapshot_id (Annotated[str | None, MinLen(min_length=1)])
retriever_id (Annotated[str | None, MinLen(min_length=1)])
budget (QueryBudget)
reranker_id (Annotated[str | None, MinLen(min_length=1)])
minimum_score (float | None)
- budget: QueryBudget
- corpus_path: str
- minimum_score: float | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- query_text: str
- reranker_id: str | None
- retriever_id: str | None
- snapshot_id: str | None
- class biblicus.api.ReindexRequest(*, corpus_path)[source]
Request to rebuild the corpus catalog from the on-disk corpus.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
- corpus_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.api.ReindexResponse(*, scanned, skipped, inserted, updated)[source]
Catalog rebuild statistics.
- Variables:
scanned (int) – Files examined.
skipped (int) – Files skipped.
inserted (int) – Catalog entries added.
updated (int) – Catalog entries refreshed.
- Parameters:
scanned (Annotated[int, Ge(ge=0)])
skipped (Annotated[int, Ge(ge=0)])
inserted (Annotated[int, Ge(ge=0)])
updated (Annotated[int, Ge(ge=0)])
- inserted: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- scanned: int
- skipped: int
- updated: int
- class biblicus.api.SteeringArtifactsRequest(*, corpus_path)[source]
Request to list stable artifact references.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
- corpus_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.api.SteeringExportRequest(*, corpus_path, classifier_id, topic_governance_snapshot_id=None)[source]
Request to export a stable steering bundle.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
classifier_id (str) – Topic classifier identifier used to load the accepted topic set.
topic_governance_snapshot_id (str or None) – Topic governance snapshot, or None for the latest pointer.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
classifier_id (Annotated[str, MinLen(min_length=1)])
topic_governance_snapshot_id (Annotated[str | None, MinLen(min_length=1)])
- classifier_id: str
- corpus_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- topic_governance_snapshot_id: str | None
- class biblicus.api.SteeringGraphSignalsRequest(*, corpus_path, classifier_id, graph_snapshot, steering_feedback_path=None)[source]
Request to emit topic-informed graph steering signals.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
classifier_id (str) – Topic classifier identifier used to load accepted topics.
graph_snapshot (str) – Graph snapshot reference in
extractor_id:snapshot_idform.steering_feedback_path (str or None) – Optional reviewed steering feedback JSON path.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
classifier_id (Annotated[str, MinLen(min_length=1)])
graph_snapshot (Annotated[str, MinLen(min_length=1)])
steering_feedback_path (Annotated[str | None, MinLen(min_length=1)])
- classifier_id: str
- corpus_path: str
- graph_snapshot: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- steering_feedback_path: str | None
- class biblicus.api.SteeringRenderSeedManifestRequest(*, input_path, output_path)[source]
Request to render an accepted steering topic set as a seed manifest.
- Variables:
input_path (str) – Accepted steering topic-set JSON input path.
output_path (str) – Destination seed-manifest path.
- Parameters:
input_path (Annotated[str, MinLen(min_length=1)])
output_path (Annotated[str, MinLen(min_length=1)])
- input_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- output_path: str
- class biblicus.api.SummarizeMapReduceRequest(*, extracted_text, source_uri='', reference_title='', original_title='', original_subtitle='', source_publication_date=None, source_is_archived=None, generated_title='', task, model)[source]
Request to map-reduce one bounded metadata field from extracted source text.
- Variables:
task (str) – Metadata field to generate.
model (str) – Pinned model identifier.
- Parameters:
extracted_text (Annotated[str, MinLen(min_length=1)])
source_uri (str)
reference_title (str)
original_title (str)
original_subtitle (str)
source_publication_date (date | None)
source_is_archived (bool | None)
generated_title (str)
task (Literal['title', 'subtitle', 'summary'])
model (Annotated[str, MinLen(min_length=1)])
- model: str
- model_config = {'extra': 'forbid', 'strict': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- task: Literal['title', 'subtitle', 'summary']
- biblicus.api.SummarizeMapReduceResponse
alias of
TransformResult
- class biblicus.api.TaxonomyDiscoverRequest(*, corpus_path, classifier_id, extraction_snapshot, steering_feedback_path=None)[source]
Request to discover candidate child taxonomy nodes under accepted root topics.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
classifier_id (str) – Topic classifier identifier used to collect root topic members.
extraction_snapshot (ExtractionSnapshotReference) – Extraction snapshot supplying item text.
steering_feedback_path (str or None) – Optional reviewed steering feedback JSON path.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
classifier_id (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (ExtractionSnapshotReference)
steering_feedback_path (Annotated[str | None, MinLen(min_length=1)])
- classifier_id: str
- corpus_path: str
- extraction_snapshot: ExtractionSnapshotReference
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- steering_feedback_path: str | None
- class biblicus.api.TaxonomyRecordRequest(*, corpus_path, input_path)[source]
Request to record an accepted taxonomy manifest.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
input_path (str) – Accepted taxonomy JSON input path.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
input_path (Annotated[str, MinLen(min_length=1)])
- corpus_path: str
- input_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.api.TopicClassifierProjectRequest(*, classifier_corpus_path, target_corpus_path, classifier_id, extraction_snapshot, project_all=False, item_ids=<factory>, review_threshold=0.35, top_k=5, record=False)[source]
Request to project a classifier from an authority corpus onto a target corpus.
- Variables:
classifier_corpus_path (str) – Corpus path holding classifier artifacts.
target_corpus_path (str) – Corpus path holding the items to classify.
classifier_id (str) – Topic classifier identifier.
extraction_snapshot (ExtractionSnapshotReference) – Target corpus extraction snapshot.
project_all (bool) – True to project onto every target item.
item_ids (list[str]) – Target item identifiers to project.
review_threshold (float) – Minimum confidence before review is recommended.
top_k (int) – Maximum ranked topic candidates to include.
record (bool) – True to persist projection predictions as audit records.
- Parameters:
classifier_corpus_path (Annotated[str, MinLen(min_length=1)])
target_corpus_path (Annotated[str, MinLen(min_length=1)])
classifier_id (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (ExtractionSnapshotReference)
project_all (bool)
item_ids (List[str])
review_threshold (float)
top_k (Annotated[int, Ge(ge=1)])
record (bool)
- classifier_corpus_path: str
- classifier_id: str
- extraction_snapshot: ExtractionSnapshotReference
- item_ids: List[str]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- project_all: bool
- record: bool
- review_threshold: float
- target_corpus_path: str
- top_k: int
- class biblicus.api.TopicClassifierTrainRequest(*, corpus_path, manifest_path, configuration_paths, overrides=<factory>, configuration_name='default', extraction_snapshot)[source]
Request to train a topic classifier model version.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
manifest_path (str) – Topic classifier seed manifest path.
configuration_paths (list[str]) – Configuration YAML paths; later files override earlier ones.
overrides (list[str]) –
key=valueoverrides applied after composing configurations.configuration_name (str) – Human-readable configuration name that keys the snapshot identity.
extraction_snapshot (ExtractionSnapshotReference) – Extraction snapshot supplying item text.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
manifest_path (Annotated[str, MinLen(min_length=1)])
configuration_paths (Annotated[List[str], MinLen(min_length=1)])
overrides (List[str])
configuration_name (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (ExtractionSnapshotReference)
- configuration_name: str
- configuration_paths: List[str]
- corpus_path: str
- extraction_snapshot: ExtractionSnapshotReference
- manifest_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- overrides: List[str]
- class biblicus.api.TopicContextRequest(*, corpus_path, topic_modeling_snapshot_id, max_topics=20, examples_per_topic=3, summary_model=None, include_outlier=False)[source]
Request to generate a research-agent topic context report.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
topic_modeling_snapshot_id (str) – Topic modeling snapshot to summarize.
max_topics (int) – Maximum non-outlier topic buckets to include.
examples_per_topic (int) – Maximum representative examples per topic.
summary_model (str or None) – Optional model for topic guidance and missing example summaries.
include_outlier (bool) – True to include the outlier topic.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
topic_modeling_snapshot_id (Annotated[str, MinLen(min_length=1)])
max_topics (Annotated[int, Ge(ge=1)])
examples_per_topic (Annotated[int, Ge(ge=1)])
summary_model (Annotated[str | None, MinLen(min_length=1)])
include_outlier (bool)
- corpus_path: str
- examples_per_topic: int
- include_outlier: bool
- max_topics: int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- summary_model: str | None
- topic_modeling_snapshot_id: str
- class biblicus.api.TopicIdentityRequest(*, corpus_path, current_run_id, previous_run_id=None, threshold=0.5, trends_snapshot_id=None)[source]
Request to match topic-model clusters to the previous run and assign stable identifiers.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
current_run_id (str) – Topic modeling snapshot to identify.
previous_run_id (str or None) – Previous topic modeling snapshot, or None for the first run.
threshold (float) – Minimum Jaccard overlap of member item identifiers for a match.
trends_snapshot_id (str or None) – Topic trends snapshot with weekly bins for the eight-week series.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
current_run_id (Annotated[str, MinLen(min_length=1)])
previous_run_id (Annotated[str | None, MinLen(min_length=1)])
threshold (Annotated[float, Gt(gt=0), Le(le=1)])
trends_snapshot_id (Annotated[str | None, MinLen(min_length=1)])
- corpus_path: str
- current_run_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- previous_run_id: str | None
- threshold: float
- trends_snapshot_id: str | None
- class biblicus.api.TopicTrendsRequest(*, corpus_path, topic_modeling_snapshot_id, classifier_id=None, windows=<factory>, as_of=None, rank_window='90d', bin=None, weeks=None)[source]
Request to run publication-date topic trend analysis.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
topic_modeling_snapshot_id (str) – Topic modeling snapshot to analyze.
classifier_id (str or None) – Optional topic classifier for canonical topic coverage.
windows (list[str]) – Publication windows such as
30d,90d,1y,all.as_of (str or None) – Publication date anchor, or None for the latest published date.
rank_window (str) – Window used to rank topics; must be one of
windows.bin (str or None) –
weeklyto add weekly counts per topic, or None for the trailing windows only.weeks (int or None) – Number of weekly bins, 8 when omitted. Requires
bin.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
topic_modeling_snapshot_id (Annotated[str, MinLen(min_length=1)])
classifier_id (Annotated[str | None, MinLen(min_length=1)])
windows (Annotated[List[str], MinLen(min_length=1)])
as_of (Annotated[str | None, MinLen(min_length=1)])
rank_window (Annotated[str, MinLen(min_length=1)])
bin (Literal['weekly'] | None)
weeks (Annotated[int | None, Ge(ge=1), Le(le=520)])
- as_of: str | None
- bin: Literal['weekly'] | None
- classifier_id: str | None
- corpus_path: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- rank_window: str
- topic_modeling_snapshot_id: str
- weeks: int | None
- windows: List[str]
- class biblicus.api.TypedEntitiesRequest(*, corpus_path, vocabulary_path, registry_path=None, backend='gliner2', threshold=0.7, item_ids=<factory>, extraction_snapshot=None, out_dir=None)[source]
Request to extract typed entities and relations from corpus items.
- Variables:
corpus_path (str) – Corpus path or uniform resource identifier.
vocabulary_path (str) – Closed vocabulary file (YAML or JSON).
registry_path (str or None) – Registry of accepted entities (JSON list), or None to propose every mention.
backend (str) –
gliner2for the registry matcher followed by GLiNER2, orregistry-only.threshold (float) – Score threshold for the backend, greater than 0 and at most 1.
item_ids (list[str]) – Items to process; empty for every item with extracted text.
extraction_snapshot (str or None) – Extraction snapshot as
extractor_id:snapshot_id, or None for the latest snapshot.out_dir (str or None) – Extra directory that receives
output.json, or None.
- Parameters:
corpus_path (Annotated[str, MinLen(min_length=1)])
vocabulary_path (Annotated[str, MinLen(min_length=1)])
registry_path (Annotated[str | None, MinLen(min_length=1)])
backend (Literal['gliner2', 'registry-only'])
threshold (Annotated[float, Gt(gt=0), Le(le=1)])
item_ids (List[str])
extraction_snapshot (Annotated[str | None, MinLen(min_length=1)])
out_dir (Annotated[str | None, MinLen(min_length=1)])
- backend: Literal['gliner2', 'registry-only']
- corpus_path: str
- extraction_snapshot: str | None
- item_ids: List[str]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- out_dir: str | None
- registry_path: str | None
- threshold: float
- vocabulary_path: str
- class biblicus.api.UrlTextFailureDetails(*, source_kind, strategy, content_type, prompt_version, attempts)[source]
Diagnostics a failed URL text extraction carries in the command failure
details.- Variables:
source_kind (str) –
web,pdforyoutube.strategy (str) – Fetch strategy that was applied.
content_type (str or None) – Fetched content type or None.
prompt_version (str) – Extraction contract version.
attempts (list[dict[str, Any]]) – Ordered fetch attempts made before the failure.
- Parameters:
source_kind (Literal['web', 'pdf', 'youtube'])
strategy (str)
content_type (str | None)
prompt_version (str)
attempts (List[Dict[str, Any]])
- attempts: List[Dict[str, Any]]
- content_type: str | None
- classmethod from_response(response)[source]
Copy the diagnostics out of a failed response.
- Parameters:
response (UrlTextResponse) – Failed URL text response.
- Returns:
The diagnostics.
- Return type:
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- prompt_version: str
- source_kind: Literal['web', 'pdf', 'youtube']
- strategy: str
- class biblicus.api.UrlTextRequest(*, source_uri, reference_title='', timeout_seconds=20.0)[source]
Request to extract readable text from a source URL.
- Variables:
source_uri (str) – Source URL.
reference_title (str) – Optional reference title context.
timeout_seconds (float) – Per-attempt fetch timeout in seconds.
- Parameters:
source_uri (Annotated[str, MinLen(min_length=1)])
reference_title (str)
timeout_seconds (Annotated[float, Gt(gt=0)])
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- reference_title: str
- source_uri: str
- timeout_seconds: float
- class biblicus.api.UrlTextResponse(*, status, source_kind, strategy, text, markdown, title, content_type, method=None, grobid=None, structured=None, prompt_version, attempts, error)[source]
Result of a URL text extraction.
Fields that the extraction did not produce are absent from the serialized JSON, so a failed extraction omits
method,grobidandstructured.- Variables:
status (str) –
okorfailed.source_kind (str) –
web,pdforyoutube.strategy (str) – Fetch strategy that was applied.
text (str) – Extracted plain text.
markdown (str) – Extracted markdown.
title (str or None) – Extracted title or None.
content_type (str or None) – Fetched content type or None.
method (str or None) – Extraction method or None.
grobid (dict[str, Any] or None) – Scholarly parse payload or None.
structured (dict[str, Any] or None) – Structured web payload or None.
prompt_version (str) – Extraction contract version.
attempts (list[dict[str, Any]]) – Ordered fetch attempts.
error (dict[str, Any] or None) – Structured failure or None.
- Parameters:
status (Literal['ok', 'failed'])
source_kind (Literal['web', 'pdf', 'youtube'])
strategy (str)
text (str)
markdown (str)
title (str | None)
content_type (str | None)
method (str | None)
grobid (Dict[str, Any] | None)
structured (Dict[str, Any] | None)
prompt_version (str)
attempts (List[Dict[str, Any]])
error (Dict[str, Any] | None)
- attempts: List[Dict[str, Any]]
- content_type: str | None
- error: Dict[str, Any] | None
- grobid: Dict[str, Any] | None
- markdown: str
- method: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- prompt_version: str
- source_kind: Literal['web', 'pdf', 'youtube']
- status: Literal['ok', 'failed']
- strategy: str
- structured: Dict[str, Any] | None
- text: str
- title: str | None
- class biblicus.api.WebMetadataRequest(*, source_uri='', html_content='', reference_title='', use_llm_fallback=False, model='', timeout_seconds=20.0)[source]
Request to extract reference metadata from a page URL or supplied HTML.
- Variables:
source_uri (str) – Page URL; fetched when
html_contentis absent.html_content (str) – Supplied HTML document, which takes precedence over fetching.
reference_title (str) – Optional title hint for model enrichment.
use_llm_fallback (bool) – True to enrich sparse heuristics with a model.
model (str) – Model identifier for enrichment.
timeout_seconds (float) – Fetch timeout in seconds.
- Parameters:
source_uri (str)
html_content (str)
reference_title (str)
use_llm_fallback (bool)
model (str)
timeout_seconds (Annotated[float, Gt(gt=0)])
- html_content: str
- model: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- reference_title: str
- require_source_uri_or_html_content()[source]
Require a page URL or supplied HTML.
- Returns:
The validated request.
- Return type:
- Raises:
ValueError – If both
source_uriandhtml_contentare blank.
- source_uri: str
- timeout_seconds: float
- use_llm_fallback: bool
- class biblicus.api.WebMetadataResponse(*, status, schemaVersion, sourceUri, title, subtitle, authors, authorsStructured, publicationDate, publicationDateRaw, updatedAt, structured, method, layers, rawMetadata, warnings, referenceCandidates, citationCount)[source]
Reference metadata extracted from a web page.
- Variables:
status (str) – Always
ok; failures are raised as errors.schemaVersion (str) – Metadata contract version.
sourceUri (str) – Page URL used for provenance.
title (str or None) – Page title or None.
subtitle (str or None) – Page subtitle or None.
authors (list[str]) – Author names.
authorsStructured (list[dict[str, Any]]) – Structured author records.
publicationDate (str or None) – Normalized publication date or None.
publicationDateRaw (str or None) – Publication date as found or None.
updatedAt (str or None) – Last update timestamp or None.
structured (dict[str, Any]) – Structured web payload.
method (str) – Extraction method.
layers (list[Any]) – Heuristic layers that contributed.
rawMetadata (dict[str, Any]) – Raw page metadata.
warnings (list[Any]) – Extraction warnings.
referenceCandidates (list[Any]) – Candidate cited references.
citationCount (int) – Number of structured citations.
- Parameters:
status (Literal['ok'])
schemaVersion (str)
sourceUri (str)
title (str | None)
subtitle (str | None)
authors (List[str])
authorsStructured (List[Dict[str, Any]])
publicationDate (str | None)
publicationDateRaw (str | None)
updatedAt (str | None)
structured (Dict[str, Any])
method (str)
layers (List[Any])
rawMetadata (Dict[str, Any])
warnings (List[Any])
referenceCandidates (List[Any])
citationCount (int)
- authors: List[str]
- authorsStructured: List[Dict[str, Any]]
- citationCount: int
- layers: List[Any]
- method: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- publicationDate: str | None
- publicationDateRaw: str | None
- rawMetadata: Dict[str, Any]
- referenceCandidates: List[Any]
- schemaVersion: str
- sourceUri: str
- status: Literal['ok']
- structured: Dict[str, Any]
- subtitle: str | None
- title: str | None
- updatedAt: str | None
- warnings: List[Any]
- biblicus.api.analyze_topic_context(request)[source]
Generate a research-agent topic context report.
- Parameters:
request (TopicContextRequest) – Validated request.
- Returns:
Topic context report.
- Return type:
TopicContextOutput
- biblicus.api.analyze_topic_trends(request)[source]
Run publication-date topic trend analysis.
- Parameters:
request (TopicTrendsRequest) – Validated request.
- Returns:
Topic trend report.
- Return type:
TopicTrendOutput
- biblicus.api.build_api_schemas()[source]
Build the JSON Schema for every request and response model.
Request schemas use validation mode and response schemas use serialization mode, so a response schema describes exactly the JSON the command prints.
- Returns:
Mapping of
<command>.requestand<command>.responseto a JSON Schema object.- Return type:
dict[str, dict[str, Any]]
- biblicus.api.context_pack_build_blocks(request)[source]
Build a context pack from ordered context blocks.
- Parameters:
request (ContextBlockBuildRequest) – Validated block build request.
- Returns:
Built context pack with per-block accounting.
- Return type:
- biblicus.api.extract_article_text(request)[source]
Select grounded article paragraphs from extracted source text.
- Parameters:
request (ArticleInput) – Validated article input.
- Returns:
Transform result, with
statusfailedand a sanitized error on failure.- Return type:
TransformResult
- biblicus.api.extract_typed_entities(request)[source]
Extract typed entities and relations from the extracted text of corpus items.
- Parameters:
request (TypedEntitiesRequest) – Validated request.
- Returns:
Typed entities and relations with evidence.
- Return type:
TypedEntitiesOutput
- Raises:
FileNotFoundError – If the vocabulary, registry or extraction snapshot is missing.
ValueError – If an input file is invalid or a requested item has no extracted text.
- biblicus.api.extract_url_text(request)[source]
Extract readable text from a source URL using the source-aware fetch strategy.
- Parameters:
request (UrlTextRequest) – Validated request.
- Returns:
Extraction result, with
statusfailedand a structured error on failure.- Return type:
- biblicus.api.extract_web_metadata(request)[source]
Extract reference metadata from supplied HTML or a fetched page.
- Parameters:
request (WebMetadataRequest) – Validated request.
- Returns:
Reference metadata.
- Return type:
- Raises:
ValueError – If extraction rejects the supplied page.
- biblicus.api.format_validation_error(command_label, error)[source]
Render a validation error as a single clear sentence naming each offending field.
- Parameters:
command_label (str) – Human-readable command name, for example
taxonomy discover.error (pydantic.ValidationError) – Validation error raised while building a request model.
- Returns:
Message describing every invalid field.
- Return type:
str
- biblicus.api.graph_extract(request, *, progress_callback=None)[source]
Build a graph extraction snapshot.
- Parameters:
request (GraphExtractRequest) – Validated request.
progress_callback (collections.abc.Callable or None) – Optional callback receiving progress event names and payloads.
- Returns:
Graph snapshot manifest.
- Return type:
- biblicus.api.open_request_corpus(corpus_path)[source]
Open the corpus named by a request.
- Parameters:
corpus_path (str) – Corpus path or uniform resource identifier.
- Returns:
Opened corpus.
- Return type:
- biblicus.api.query(request)[source]
Run a retrieval query and return structured evidence.
- Parameters:
request (QueryRequest) – Validated request.
- Returns:
Retrieval result with evidence.
- Return type:
- Raises:
ValueError – If no snapshot is available or the retriever does not match the snapshot.
- biblicus.api.reindex(request)[source]
Rebuild the corpus catalog from the on-disk corpus.
- Parameters:
request (ReindexRequest) – Validated request.
- Returns:
Catalog rebuild statistics.
- Return type:
- biblicus.api.steering_artifacts(request)[source]
List stable artifact references for an external steering worker.
- Parameters:
request (SteeringArtifactsRequest) – Validated request.
- Returns:
Artifact inventory.
- Return type:
SteeringArtifactInventory
- biblicus.api.steering_export(request)[source]
Export a stable steering bundle for an external application.
- Parameters:
request (SteeringExportRequest) – Validated request.
- Returns:
Steering export bundle.
- Return type:
SteeringExportBundle
- biblicus.api.steering_graph_signals(request)[source]
Emit topic-informed graph steering signals.
- Parameters:
request (SteeringGraphSignalsRequest) – Validated request.
- Returns:
Steering proposal bundle.
- Return type:
SteeringProposalBundle
- biblicus.api.steering_render_seed_manifest(request)[source]
Render an accepted steering topic set as a Biblicus seed manifest.
- Parameters:
request (SteeringRenderSeedManifestRequest) – Validated request.
- Returns:
Description of the rendered manifest.
- Return type:
SteeringRenderSeedManifestOutput
- biblicus.api.summarize_map_reduce(request)[source]
Map source evidence and reduce one metadata field.
- Parameters:
request (SummarizeMapReduceRequest) – Validated request.
- Returns:
Transform result, with
statusfailedand a sanitized error on failure.- Return type:
TransformResult
- biblicus.api.taxonomy_discover(request)[source]
Discover candidate child taxonomy nodes under accepted root topics.
- Parameters:
request (TaxonomyDiscoverRequest) – Validated request.
- Returns:
Discovery output.
- Return type:
TaxonomyDiscoveryOutput
- biblicus.api.taxonomy_record(request)[source]
Record an accepted taxonomy manifest.
- Parameters:
request (TaxonomyRecordRequest) – Validated request.
- Returns:
Recorded taxonomy snapshot description.
- Return type:
TaxonomyRecordOutput
- biblicus.api.topic_classifier_project(request)[source]
Project a topic classifier from an authority corpus onto a target corpus.
- Parameters:
request (TopicClassifierProjectRequest) – Validated request.
- Returns:
Projection output.
- Return type:
TopicClassifierProjectionOutput
- biblicus.api.topic_classifier_train(request)[source]
Train a topic classifier model version.
- Parameters:
request (TopicClassifierTrainRequest) – Validated request.
- Returns:
Training output.
- Return type:
TopicClassifierTrainOutput
- biblicus.api.validate_request(request_type, command_label, **fields)[source]
Build a request model from keyword fields and convert validation failures to clear errors.
- Parameters:
request_type (type[pydantic.BaseModel]) – Request model class to instantiate.
command_label (str) – Human-readable command name used in error messages.
fields (Any) – Field values for the request model.
- Returns:
Validated request model.
- Return type:
pydantic.BaseModel
- Raises:
InvalidRequestError – If the fields do not satisfy the request model.
Generation and drift checking for the published JSON Schemas of the typed application programming interface.
- biblicus.api.schemas.find_stale_api_schemas(directory)[source]
Compare the published schemas in a directory with the models and list every difference.
- Parameters:
directory (pathlib.Path) – Directory holding the published schema files.
- Returns:
Descriptions of missing, changed and unexpected files; empty when current.
- Return type:
list[str]
- biblicus.api.schemas.render_schema_text(schema)[source]
Render one schema as stable, human-diffable JSON text.
- Parameters:
schema (dict[str, Any]) – JSON Schema object.
- Returns:
Indented JSON with sorted keys and a trailing newline.
- Return type:
str
- biblicus.api.schemas.schema_file_name(schema_key)[source]
Map a schema key such as
taxonomy-discover.requestto its published file name.- Parameters:
schema_key (str) – Key from
biblicus.api.build_api_schemas().- Returns:
File name ending in
.schema.json.- Return type:
str
- biblicus.api.schemas.write_api_schemas(directory)[source]
Write every request and response schema into a directory.
- Parameters:
directory (pathlib.Path) – Destination directory, created when missing.
- Returns:
Names of the files written, sorted.
- Return type:
list[str]
The failure envelope that every typed Biblicus command prints when it fails.
A typed command is one that has a request model, a response model and a published schema in the
typed application programming interface. When such a command fails, the command-line interface
prints one CommandFailure as JSON on standard output and exits with a status that
separates a rejected request from a failed operation.
- class biblicus.command_failure.CommandFailure(*, status='failed', error)[source]
The single JSON object a typed command prints on standard output when it fails.
- Variables:
status (str) – Always
failed.error (CommandFailureError) – Code, message and details describing the failure.
- Parameters:
status (Literal['failed'])
error (CommandFailureError)
- error: CommandFailureError
- property exit_status: int
Exit status the command-line interface uses for this failure.
- Returns:
2when the failure code names a rejected request, otherwise1.- Return type:
int
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- status: Literal['failed']
- class biblicus.command_failure.CommandFailureError(*, code, message, details=<factory>)[source]
The error carried by a command failure.
- Variables:
code (str) – Stable machine-readable failure code.
message (str) – Human-readable explanation that never contains secrets or source text.
details (dict[str, Any]) – Optional structured diagnostics, empty when there are none.
- Parameters:
code (Annotated[str, MinLen(min_length=1)])
message (Annotated[str, MinLen(min_length=1)])
details (Dict[str, Any])
- code: str
- details: Dict[str, Any]
- message: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- exception biblicus.command_failure.CommandFailureException(*, code, message, details=None)[source]
Raised by a typed command to stop with a structured failure.
- Variables:
failure (CommandFailure) – The envelope the command-line interface prints.
- Parameters:
code (str)
message (str)
details (Dict[str, Any] | None)
- exception biblicus.command_failure.InvalidRequestError[source]
A request failed validation before any operation ran.
The command-line interface reports it as the
invalid_requestfailure with exit status 2.
- biblicus.command_failure.raise_for_error_payload(error_payload, diagnostics=None)[source]
Raise the structured failure described by a failed response model’s
errorobject.- Parameters:
error_payload (dict[str, Any]) – The
errormapping of a failed response, withcode,messageand optionallydetails.diagnostics (pydantic.BaseModel or None) – Typed diagnostics from the failed response, merged into
detailsbeneath any details the error already carries.
- Returns:
None.
- Return type:
None
- Raises:
CommandFailureException – Always, carrying the code, message and details.
Extraction
Text extraction snapshots for Biblicus.
- class biblicus.extraction.ExtractionConfigurationManifest(*, configuration_id, extractor_id, name, created_at, configuration=<factory>)[source]
Reproducible configuration for an extraction plugin snapshot.
- Variables:
configuration_id (str) – Deterministic configuration identifier.
extractor_id (str) – Extractor plugin identifier.
name (str) – Human-readable configuration name.
created_at (str) – International Organization for Standardization 8601 timestamp.
configuration (dict[str, Any]) – Extractor-specific configuration values.
- Parameters:
configuration_id (str)
extractor_id (str)
name (str)
created_at (str)
configuration (Dict[str, Any])
- configuration: Dict[str, Any]
- configuration_id: str
- created_at: str
- extractor_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- name: str
- class biblicus.extraction.ExtractionItemResult(*, item_id, status, final_text_relpath=None, final_metadata_relpath=None, final_stage_index=None, final_stage_extractor_id=None, final_producer_extractor_id=None, final_source_stage_index=None, error_type=None, error_message=None, stage_results=<factory>)[source]
Per-item result record for an extraction snapshot.
- Variables:
item_id (str) – Item identifier.
status (str) – Final result status, extracted, skipped, or errored.
final_text_relpath (str or None) – Relative path to the final extracted text artifact, when extracted.
final_metadata_relpath (str or None) – Relative path to the final metadata artifact, when present.
final_stage_index (int or None) – Pipeline stage index that produced the final text.
final_stage_extractor_id (str or None) – Extractor identifier of the stage that produced the final text.
final_producer_extractor_id (str or None) – Extractor identifier that produced the final text content.
final_source_stage_index (int or None) – Optional stage index that supplied the final text for selection-style extractors.
error_type (str or None) – Optional error type name when no extracted text was produced.
error_message (str or None) – Optional error message when no extracted text was produced.
stage_results (list[ExtractionStageResult]) – Per-stage results recorded for this item.
- Parameters:
item_id (str)
status (str)
final_text_relpath (str | None)
final_metadata_relpath (str | None)
final_stage_index (Annotated[int | None, Ge(ge=1)])
final_stage_extractor_id (str | None)
final_producer_extractor_id (str | None)
final_source_stage_index (Annotated[int | None, Ge(ge=1)])
error_type (str | None)
error_message (str | None)
stage_results (List[ExtractionStageResult])
- error_message: str | None
- error_type: str | None
- final_metadata_relpath: str | None
- final_producer_extractor_id: str | None
- final_source_stage_index: int | None
- final_stage_extractor_id: str | None
- final_stage_index: int | None
- final_text_relpath: str | None
- item_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- stage_results: List[ExtractionStageResult]
- status: str
- class biblicus.extraction.ExtractionSnapshotManifest(*, snapshot_id, configuration, corpus_uri, catalog_generated_at, created_at, items=<factory>, stats=<factory>)[source]
Immutable record describing an extraction snapshot.
- Variables:
snapshot_id (str) – Unique snapshot identifier.
configuration (ExtractionConfigurationManifest) – Configuration manifest for this snapshot.
corpus_uri (str) – Canonical uniform resource identifier for the corpus root.
catalog_generated_at (str) – Catalog timestamp used for the snapshot.
created_at (str) – International Organization for Standardization 8601 timestamp for snapshot creation.
items (list[ExtractionItemResult]) – Per-item results.
stats (dict[str, Any]) – Snapshot statistics.
- Parameters:
snapshot_id (str)
configuration (ExtractionConfigurationManifest)
corpus_uri (str)
catalog_generated_at (str)
created_at (str)
items (List[ExtractionItemResult])
stats (Dict[str, Any])
- catalog_generated_at: str
- configuration: ExtractionConfigurationManifest
- corpus_uri: str
- created_at: str
- items: List[ExtractionItemResult]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- stats: Dict[str, Any]
- class biblicus.extraction.ExtractionStageResult(*, stage_index, extractor_id, status, text_relpath=None, text_characters=0, producer_extractor_id=None, source_stage_index=None, confidence=None, metadata_relpath=None, error_type=None, error_message=None)[source]
Per-item result record for a single pipeline stage.
- Variables:
stage_index (int) – One-based pipeline stage index.
extractor_id (str) – Extractor identifier for the stage.
status (str) – Stage status, extracted, skipped, or errored.
text_relpath (str or None) – Relative path to the stage text artifact, when extracted.
text_characters (int) – Character count of the extracted text.
producer_extractor_id (str or None) – Extractor identifier that produced the text content.
source_stage_index (int or None) – Optional stage index that supplied the text for selection-style extractors.
confidence (float or None) – Optional confidence score from 0.0 to 1.0.
metadata_relpath (str or None) – Relative path to the stage metadata artifact, when present.
error_type (str or None) – Optional error type name for errored stages.
error_message (str or None) – Optional error message for errored stages.
- Parameters:
stage_index (Annotated[int, Ge(ge=1)])
extractor_id (str)
status (str)
text_relpath (str | None)
text_characters (Annotated[int, Ge(ge=0)])
producer_extractor_id (str | None)
source_stage_index (Annotated[int | None, Ge(ge=1)])
confidence (Annotated[float | None, Ge(ge=0.0), Le(le=1.0)])
metadata_relpath (str | None)
error_type (str | None)
error_message (str | None)
- confidence: float | None
- error_message: str | None
- error_type: str | None
- extractor_id: str
- metadata_relpath: str | None
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- producer_extractor_id: str | None
- source_stage_index: int | None
- stage_index: int
- status: str
- text_characters: int
- text_relpath: str | None
- biblicus.extraction.build_extraction_snapshot(corpus, *, extractor_id, configuration_name, configuration, force=False, max_workers=1)[source]
Build an extraction snapshot for a corpus using the pipeline extractor.
- Parameters:
corpus (Corpus) – Corpus to extract from.
extractor_id (str) – Extractor plugin identifier (must be
pipeline).configuration_name (str) – Human-readable configuration name.
configuration (dict[str, Any]) – Extractor configuration mapping.
force (bool) – Whether to reprocess items even if artifacts already exist.
max_workers (int) – Maximum number of concurrent workers.
- Returns:
Extraction snapshot manifest describing the build.
- Return type:
- Raises:
KeyError – If the extractor identifier is unknown.
ValueError – If the extractor configuration is invalid.
OSError – If the snapshot directory or artifacts cannot be written.
ExtractionSnapshotFatalError – If the extractor is not the pipeline.
- biblicus.extraction.create_extraction_configuration_manifest(*, extractor_id, name, configuration)[source]
Create a deterministic extraction configuration manifest.
- Parameters:
extractor_id (str) – Extractor plugin identifier.
name (str) – Human configuration name.
configuration (dict[str, Any]) – Extractor configuration.
- Returns:
Configuration manifest.
- Return type:
- biblicus.extraction.create_extraction_snapshot_manifest(corpus, *, configuration)[source]
Create a new extraction snapshot manifest for a corpus.
- Parameters:
corpus (Corpus) – Corpus associated with the snapshot.
configuration (ExtractionConfigurationManifest) – Configuration manifest.
- Returns:
Snapshot manifest.
- Return type:
- biblicus.extraction.load_or_build_extraction_snapshot(corpus, *, extractor_id, configuration_name, configuration, max_workers=1)[source]
Load an extraction snapshot if it exists or build it when missing.
- Parameters:
corpus (Corpus) – Corpus to extract from.
extractor_id (str) – Extractor plugin identifier (must be
pipeline).configuration_name (str) – Human-readable configuration name.
configuration (dict[str, Any]) – Extractor configuration mapping.
max_workers (int) – Maximum number of concurrent workers.
- Returns:
Extraction snapshot manifest describing the build.
- Return type:
- biblicus.extraction.write_extracted_metadata_artifact(*, snapshot_dir, item, metadata)[source]
Write an extracted metadata artifact for an item into the snapshot directory.
- Parameters:
snapshot_dir (Path) – Extraction snapshot directory.
item (CatalogItem) – Catalog item being extracted.
metadata (dict[str, Any]) – Metadata dictionary to persist.
- Returns:
Relative path to the stored metadata artifact, or None if empty.
- Return type:
str or None
- biblicus.extraction.write_extracted_text_artifact(*, snapshot_dir, item, text)[source]
Write an extracted text artifact for an item into the snapshot directory.
- Parameters:
snapshot_dir (Path) – Extraction snapshot directory.
item (CatalogItem) – Catalog item being extracted.
text (str) – Extracted text.
- Returns:
Relative path to the stored text artifact.
- Return type:
str
- biblicus.extraction.write_extraction_latest_pointer(*, extractor_dir, manifest)[source]
Persist the latest pointer for an extractor.
- Parameters:
extractor_dir (Path) – Extractor directory containing snapshots.
manifest (ExtractionSnapshotManifest) – Snapshot manifest used for the pointer.
- Returns:
None.
- Return type:
None
- biblicus.extraction.write_extraction_snapshot_manifest(*, snapshot_dir, manifest)[source]
Persist an extraction snapshot manifest to a snapshot directory.
- Parameters:
snapshot_dir (Path) – Extraction snapshot directory.
manifest (ExtractionSnapshotManifest) – Snapshot manifest to write.
- Returns:
None.
- Return type:
None
- biblicus.extraction.write_pipeline_stage_metadata_artifact(*, snapshot_dir, stage_index, extractor_id, item, metadata)[source]
Write a pipeline stage metadata artifact for an item.
- Parameters:
snapshot_dir (Path) – Extraction snapshot directory.
stage_index (int) – One-based pipeline stage index.
extractor_id (str) – Extractor identifier for the stage.
item (CatalogItem) – Catalog item being extracted.
metadata (dict[str, Any]) – Metadata dictionary to persist.
- Returns:
Relative path to the stored stage metadata artifact, or None if empty.
- Return type:
str or None
- biblicus.extraction.write_pipeline_stage_text_artifact(*, snapshot_dir, stage_index, extractor_id, item, text)[source]
Write a pipeline stage text artifact for an item.
- Parameters:
snapshot_dir (Path) – Extraction snapshot directory.
stage_index (int) – One-based pipeline stage index.
extractor_id (str) – Extractor identifier for the stage.
item (CatalogItem) – Catalog item being extracted.
text (str) – Extracted text content.
- Returns:
Relative path to the stored stage text artifact.
- Return type:
str
Text extraction plugins for Biblicus.
Graph
Graph extraction snapshots for Biblicus.
- biblicus.graph.extraction.build_graph_snapshot(corpus, *, extractor_id, configuration_name, configuration, extraction_snapshot, max_items=None, item_timeout_seconds=60.0, item_retry_attempts=0, heartbeat_interval_seconds=10.0, progress_callback=None)[source]
Build a graph extraction snapshot for a corpus.
- Parameters:
corpus (Corpus) – Corpus to process.
extractor_id (str) – Graph extractor identifier.
configuration_name (str) – Human configuration name.
configuration (dict[str, Any]) – Extractor configuration values.
extraction_snapshot (ExtractionSnapshotReference) – Extraction snapshot reference.
max_items (int or None) – Optional maximum number of extraction items to process.
item_timeout_seconds (float or None) – Optional per-item extraction timeout in seconds. Non-positive values disable timeout enforcement.
item_retry_attempts (int) – Number of retries after a failed extraction attempt.
heartbeat_interval_seconds (float) – Progress heartbeat cadence in seconds.
progress_callback (collections.abc.Callable or None) – Optional callback for progress events.
- Returns:
Graph snapshot manifest.
- Return type:
- biblicus.graph.extraction.create_graph_configuration_manifest(*, extractor_id, name, configuration)[source]
Create a deterministic graph extraction configuration manifest.
- Parameters:
extractor_id (str) – Graph extractor identifier.
name (str) – Human configuration name.
configuration (dict[str, Any]) – Extractor configuration.
- Returns:
Configuration manifest.
- Return type:
- biblicus.graph.extraction.create_graph_id(*, extractor_id, configuration)[source]
Create a deterministic graph identifier from extractor and configuration.
- Parameters:
extractor_id (str) – Graph extractor identifier.
configuration (dict[str, Any]) – Extractor configuration.
- Returns:
Graph identifier.
- Return type:
str
- biblicus.graph.extraction.create_graph_snapshot_manifest(corpus, *, configuration, extraction_snapshot, graph_id)[source]
Create a new graph snapshot manifest for a corpus.
- Parameters:
corpus (Corpus) – Corpus associated with the snapshot.
configuration (GraphConfigurationManifest) – Configuration manifest.
extraction_snapshot (ExtractionSnapshotReference) – Extraction snapshot reference.
graph_id (str) – Graph identifier.
- Returns:
Graph snapshot manifest.
- Return type:
- biblicus.graph.extraction.export_graph_snapshot(corpus, *, snapshot)[source]
Export graph snapshot contents from Neo4j as portable JSON records.
- Parameters:
corpus (Corpus) – Corpus containing the snapshot manifest.
snapshot (GraphSnapshotReference) – Graph snapshot reference.
- Returns:
Portable graph snapshot export.
- Return type:
- biblicus.graph.extraction.latest_graph_snapshot_reference(corpus, *, extractor_id=None)[source]
Return the most recent graph snapshot reference.
- Parameters:
corpus (Corpus) – Corpus containing the snapshots.
extractor_id (str or None) – Optional extractor identifier filter.
- Returns:
Latest graph snapshot reference or None when no snapshots exist.
- Return type:
GraphSnapshotReference or None
- biblicus.graph.extraction.list_graph_snapshots(corpus, *, extractor_id=None)[source]
List graph snapshots stored under the corpus.
- Parameters:
corpus (Corpus) – Corpus containing the snapshots.
extractor_id (str or None) – Optional extractor identifier filter.
- Returns:
Summary list entries for each snapshot.
- Return type:
list[GraphSnapshotListEntry]
- biblicus.graph.extraction.load_graph_snapshot_manifest(corpus, *, extractor_id, snapshot_id)[source]
Load a graph snapshot manifest from the corpus.
- Parameters:
corpus (Corpus) – Corpus containing the snapshot.
extractor_id (str) – Graph extractor identifier.
snapshot_id (str) – Graph snapshot identifier.
- Returns:
Parsed snapshot manifest.
- Return type:
- Raises:
FileNotFoundError – If the manifest file does not exist.
ValueError – If the manifest data is invalid.
- biblicus.graph.extraction.resolve_graph_snapshot_reference(corpus, *, raw)[source]
Resolve a graph snapshot reference from a raw string.
- Parameters:
corpus (Corpus) – Corpus containing the snapshots.
raw (str) – Raw snapshot reference.
- Returns:
Parsed graph snapshot reference.
- Return type:
- biblicus.graph.extraction.write_graph_latest_pointer(*, extractor_dir, manifest)[source]
Persist the latest pointer for a graph extractor.
- Parameters:
extractor_dir (Path) – Extractor directory containing snapshots.
manifest (GraphSnapshotManifest) – Snapshot manifest used for the pointer.
- Returns:
None.
- Return type:
None
- biblicus.graph.extraction.write_graph_snapshot_manifest(*, snapshot_dir, manifest)[source]
Persist a graph snapshot manifest to a snapshot directory.
- Parameters:
snapshot_dir (Path) – Graph snapshot directory.
manifest (GraphSnapshotManifest) – Snapshot manifest to write.
- Returns:
None.
- Return type:
None
Graph extraction models for Biblicus.
- class biblicus.graph.models.GraphConfigurationManifest(*, configuration_id, extractor_id, name, created_at, configuration=<factory>)[source]
Reproducible configuration for a graph extraction snapshot.
- Variables:
configuration_id (str) – Deterministic configuration identifier.
extractor_id (str) – Graph extractor identifier.
name (str) – Human-readable configuration name.
created_at (str) – International Organization for Standardization 8601 timestamp.
configuration (dict[str, Any]) – Extractor-specific configuration values.
- Parameters:
configuration_id (str)
extractor_id (str)
name (str)
created_at (str)
configuration (Dict[str, Any])
- configuration: Dict[str, Any]
- configuration_id: str
- created_at: str
- extractor_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- name: str
- class biblicus.graph.models.GraphEdge(*, schema_version=1, edge_id, src, dst, edge_type, weight=1.0, properties=<factory>)[source]
Edge record extracted from a corpus item.
- Variables:
edge_id (str) – Deterministic edge identifier.
src (str) – Source node identifier.
dst (str) – Destination node identifier.
edge_type (str) – Edge type identifier.
weight (float) – Edge weight.
properties (dict[str, Any]) – Edge-specific properties.
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
edge_id (Annotated[str, MinLen(min_length=1)])
src (Annotated[str, MinLen(min_length=1)])
dst (Annotated[str, MinLen(min_length=1)])
edge_type (Annotated[str, MinLen(min_length=1)])
weight (float)
properties (Dict[str, Any])
- dst: str
- edge_id: str
- edge_type: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- properties: Dict[str, Any]
- src: str
- weight: float
- class biblicus.graph.models.GraphExportEdge(*, schema_version=1, extractor_id, snapshot_id, graph_id, extraction_snapshot, item_id, edge_id, src, dst, edge_type, weight=1.0, properties=<factory>)[source]
Portable edge record exported from a graph snapshot.
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
graph_id (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (Annotated[str, MinLen(min_length=1)])
item_id (Annotated[str, MinLen(min_length=1)])
edge_id (Annotated[str, MinLen(min_length=1)])
src (Annotated[str, MinLen(min_length=1)])
dst (Annotated[str, MinLen(min_length=1)])
edge_type (Annotated[str, MinLen(min_length=1)])
weight (float)
properties (Dict[str, Any])
- dst: str
- edge_id: str
- edge_type: str
- extraction_snapshot: str
- extractor_id: str
- graph_id: str
- item_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- properties: Dict[str, Any]
- snapshot_id: str
- src: str
- weight: float
- class biblicus.graph.models.GraphExportNode(*, schema_version=1, extractor_id, snapshot_id, graph_id, extraction_snapshot, item_id, node_id, node_type, label, properties=<factory>)[source]
Portable node record exported from a graph snapshot.
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
graph_id (Annotated[str, MinLen(min_length=1)])
extraction_snapshot (Annotated[str, MinLen(min_length=1)])
item_id (Annotated[str, MinLen(min_length=1)])
node_id (Annotated[str, MinLen(min_length=1)])
node_type (Annotated[str, MinLen(min_length=1)])
label (Annotated[str, MinLen(min_length=1)])
properties (Dict[str, Any])
- extraction_snapshot: str
- extractor_id: str
- graph_id: str
- item_id: str
- label: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- node_id: str
- node_type: str
- properties: Dict[str, Any]
- snapshot_id: str
- class biblicus.graph.models.GraphExtractionItemSummary(*, item_id, node_count=0, edge_count=0, status, error_message=None, error_reason=None, duration_ms=None, attempts=None)[source]
Summary record for a single item in a graph extraction snapshot.
- Variables:
item_id (str) – Corpus item identifier.
node_count (int) – Number of nodes written for the item.
edge_count (int) – Number of edges written for the item.
status (str) – Result status.
error_message (str or None) – Optional error message.
error_reason (str or None) – Optional normalized error reason.
duration_ms (int or None) – Item processing duration in milliseconds.
attempts (int or None) – Number of extraction attempts for this item.
- Parameters:
item_id (Annotated[str, MinLen(min_length=1)])
node_count (Annotated[int, Ge(ge=0)])
edge_count (Annotated[int, Ge(ge=0)])
status (Annotated[str, MinLen(min_length=1)])
error_message (str | None)
error_reason (str | None)
duration_ms (Annotated[int | None, Ge(ge=0)])
attempts (Annotated[int | None, Ge(ge=1)])
- attempts: int | None
- duration_ms: int | None
- edge_count: int
- error_message: str | None
- error_reason: str | None
- item_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- node_count: int
- status: str
- class biblicus.graph.models.GraphExtractionResult(*, schema_version=1, item_id, nodes=<factory>, edges=<factory>, metadata=<factory>)[source]
Graph extraction output for a single item.
- Variables:
- Parameters:
- item_id: str
- metadata: Dict[str, Any]
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class biblicus.graph.models.GraphNode(*, schema_version=1, node_id, node_type, label, properties=<factory>)[source]
Node record extracted from a corpus item.
- Variables:
node_id (str) – Deterministic node identifier.
node_type (str) – Node type identifier.
label (str) – Human-readable label.
properties (dict[str, Any]) – Node-specific properties.
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
node_id (Annotated[str, MinLen(min_length=1)])
node_type (Annotated[str, MinLen(min_length=1)])
label (Annotated[str, MinLen(min_length=1)])
properties (Dict[str, Any])
- label: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- node_id: str
- node_type: str
- properties: Dict[str, Any]
- class biblicus.graph.models.GraphSchemaModel(*, schema_version=1)[source]
Base model for graph extraction schemas with strict validation.
- Variables:
schema_version (int) – Graph schema version.
- Parameters:
schema_version (Annotated[int, Ge(ge=1)])
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- schema_version: int
- class biblicus.graph.models.GraphSnapshotExport(*, snapshot, manifest, nodes=<factory>, edges=<factory>, stats=<factory>)[source]
Portable export payload for a graph extraction snapshot.
- Parameters:
snapshot (GraphSnapshotReference)
manifest (GraphSnapshotManifest)
nodes (List[GraphExportNode])
edges (List[GraphExportEdge])
stats (Dict[str, Any])
- edges: List[GraphExportEdge]
- manifest: GraphSnapshotManifest
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- nodes: List[GraphExportNode]
- snapshot: GraphSnapshotReference
- stats: Dict[str, Any]
- class biblicus.graph.models.GraphSnapshotListEntry(*, extractor_id, snapshot_id, graph_id, configuration_id, configuration_name, catalog_generated_at, created_at, stats=<factory>)[source]
Summary entry for a graph extraction snapshot stored in a corpus.
- Variables:
extractor_id (str) – Graph extractor identifier.
snapshot_id (str) – Graph snapshot identifier.
graph_id (str) – Deterministic graph identifier.
configuration_id (str) – Deterministic configuration identifier.
configuration_name (str) – Human-readable configuration name.
catalog_generated_at (str) – Catalog timestamp used for the snapshot.
created_at (str) – International Organization for Standardization 8601 timestamp for snapshot creation.
stats (dict[str, Any]) – Snapshot statistics.
- Parameters:
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
graph_id (Annotated[str, MinLen(min_length=1)])
configuration_id (Annotated[str, MinLen(min_length=1)])
configuration_name (Annotated[str, MinLen(min_length=1)])
catalog_generated_at (Annotated[str, MinLen(min_length=1)])
created_at (Annotated[str, MinLen(min_length=1)])
stats (Dict[str, object])
- catalog_generated_at: str
- configuration_id: str
- configuration_name: str
- created_at: str
- extractor_id: str
- graph_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- stats: Dict[str, object]
- class biblicus.graph.models.GraphSnapshotManifest(*, snapshot_id, graph_id, configuration, corpus_uri, catalog_generated_at, extraction_snapshot, created_at, stats=<factory>)[source]
Immutable record describing a graph extraction snapshot.
- Variables:
snapshot_id (str) – Unique snapshot identifier.
graph_id (str) – Deterministic graph identifier.
configuration (GraphConfigurationManifest) – Configuration manifest for this snapshot.
corpus_uri (str) – Canonical uniform resource identifier for the corpus root.
catalog_generated_at (str) – Catalog timestamp used for the snapshot.
extraction_snapshot (str) – Extraction snapshot reference.
created_at (str) – International Organization for Standardization 8601 timestamp for snapshot creation.
stats (dict[str, Any]) – Snapshot statistics.
- Parameters:
snapshot_id (str)
graph_id (str)
configuration (GraphConfigurationManifest)
corpus_uri (str)
catalog_generated_at (str)
extraction_snapshot (str)
created_at (str)
stats (Dict[str, Any])
- catalog_generated_at: str
- configuration: GraphConfigurationManifest
- corpus_uri: str
- created_at: str
- extraction_snapshot: str
- graph_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- stats: Dict[str, Any]
- class biblicus.graph.models.GraphSnapshotReference(*, extractor_id, snapshot_id)[source]
Reference to a graph extraction snapshot.
- Variables:
extractor_id (str) – Graph extractor identifier.
snapshot_id (str) – Graph snapshot identifier.
- Parameters:
extractor_id (Annotated[str, MinLen(min_length=1)])
snapshot_id (Annotated[str, MinLen(min_length=1)])
- as_string()[source]
Serialize the reference as a single string.
- Returns:
Reference in the form extractor_id:snapshot_id.
- Return type:
str
- extractor_id: str
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- snapshot_id: str
- biblicus.graph.models.parse_graph_snapshot_reference(value)[source]
Parse a graph snapshot reference in the form extractor_id:snapshot_id.
- Parameters:
value (str) – Raw reference string.
- Returns:
Parsed graph snapshot reference.
- Return type:
- Raises:
ValueError – If the reference is not well formed.
Neo4j graph storage helpers for Biblicus.
- class biblicus.graph.neo4j.Neo4jSettings(uri, username, password, database, auto_start, container_name, docker_image, http_port, bolt_port)[source]
Configuration values for Neo4j connectivity and lifecycle.
- Variables:
uri – Neo4j connection URI.
username – Neo4j username.
password – Neo4j password.
database – Optional Neo4j database name.
auto_start – Whether to auto-start Neo4j via Docker.
container_name – Docker container name for auto-start.
docker_image – Docker image for auto-start.
http_port – HTTP port for Neo4j UI.
bolt_port – Bolt port for Neo4j driver connections.
- Parameters:
uri (str)
username (str)
password (str)
database (str | None)
auto_start (bool)
container_name (str)
docker_image (str)
http_port (int)
bolt_port (int)
- auto_start: bool
- bolt_port: int
- container_name: str
- database: str | None
- docker_image: str
- http_port: int
- password: str
- uri: str
- username: str
- biblicus.graph.neo4j.clear_graph_records(*, driver, settings, corpus_id, graph_id, extraction_snapshot)[source]
Remove graph nodes and edges for a corpus graph snapshot.
- Parameters:
driver – Neo4j driver instance.
settings (Neo4jSettings) – Resolved Neo4j settings.
corpus_id (str) – Corpus identifier.
graph_id (str) – Graph identifier.
extraction_snapshot (str) – Extraction snapshot reference string.
- Return type:
None
- biblicus.graph.neo4j.create_neo4j_driver(settings)[source]
Create a Neo4j driver, waiting for availability when auto-start is enabled.
- Parameters:
settings (Neo4jSettings) – Resolved Neo4j settings.
- Returns:
Neo4j driver instance.
- Return type:
neo4j.Driver
- Raises:
ValueError – If the Neo4j driver dependency is missing.
- biblicus.graph.neo4j.ensure_neo4j_running(settings)[source]
Ensure the Neo4j container is running when auto-start is enabled.
- Parameters:
settings (Neo4jSettings) – Resolved Neo4j settings.
- Returns:
None.
- Return type:
None
- Raises:
ValueError – If Docker is unavailable or the container cannot be started.
- biblicus.graph.neo4j.read_graph_records(*, driver, settings, corpus_id, graph_id, extraction_snapshot)[source]
Read graph nodes and edges for a corpus graph snapshot from Neo4j.
- Returns:
Mapping with
nodesandedgesrecord lists.- Parameters:
settings (Neo4jSettings)
corpus_id (str)
graph_id (str)
extraction_snapshot (str)
- Return type:
dict[str, list[dict[str, object]]]
- biblicus.graph.neo4j.resolve_neo4j_settings(*, config=None)[source]
Resolve Neo4j settings from environment or user configuration.
- Parameters:
config (BiblicusUserConfig or None) – Optional pre-loaded user configuration.
- Returns:
Resolved Neo4j settings.
- Return type:
- biblicus.graph.neo4j.write_graph_records(*, driver, settings, corpus_id, graph_id, extraction_snapshot, item_id, nodes, edges)[source]
Persist graph nodes and edges to Neo4j.
- Parameters:
driver (neo4j.Driver) – Neo4j driver instance.
settings (Neo4jSettings) – Resolved Neo4j settings.
corpus_id (str) – Corpus identifier.
graph_id (str) – Graph identifier.
extraction_snapshot (str) – Extraction snapshot reference.
item_id (str) – Corpus item identifier.
nodes (Iterable[biblicus.graph.models.GraphNode]) – Iterable of graph nodes.
edges (Iterable[biblicus.graph.models.GraphEdge]) – Iterable of graph edges.
- Returns:
None.
- Return type:
None
Graph extractor registry for Biblicus.