Biblicus

Make your documents usable by your assistant, then decide later how you will search and retrieve them.

Retrieval quality upgrades

This document describes the retrieval quality upgrades available in Biblicus. It is a reference for how retrieval quality is expressed in runs and how to interpret the signals in artifacts and evidence.

Goals

Available upgrades

1) Tuned lexical baseline

Biblicus exposes the knobs you use to shape lexical relevance without losing determinism:

Example configuration (SQLite full text search):

python -m biblicus build --corpus corpora/demo --backend sqlite-full-text-search \
  --config chunk_size=200 \
  --config chunk_overlap=50 \
  --config snippet_characters=120 \
  --config ngram_min=1 \
  --config ngram_max=2

2) Reranking stage

The optional rerank stage rescoring keeps retrieval quality transparent. It re-scores a bounded candidate set and records rerank scores alongside retrieve scores in evidence metadata.

Example configuration:

python -m biblicus build --corpus corpora/demo --backend sqlite-full-text-search \
  --config rerank_enabled=true \
  --config rerank_model=cross-encoder \
  --config rerank_top_k=10

3) Hybrid retrieval

Hybrid retrieval combines lexical and vector signals. It expands candidate pools for each component backend, fuses scores with explicit weights, and then applies the final budget.

Example configuration:

python -m biblicus build --corpus corpora/demo --backend hybrid \
  --config lexical_retriever=sqlite-full-text-search \
  --config embedding_retriever=vector \
  --config lexical_weight=0.7 \
  --config embedding_weight=0.3

Evidence items record both stage scores in stage_scores and preserve the hybrid weights in the run metadata so evaluation can interpret how the fused ranking was produced.

Evaluation guidance

Evaluation keeps the retrieval stages explicit and makes comparisons easy:

Interpreting evidence signals

Evidence returned by retrieval snapshots includes a stage label and optional stage_scores map:

Use these fields to understand how a candidate moved through the pipeline and why it ranked where it did.

Budget awareness

Budgets shape every retrieval comparison:

When you compare backends, keep budgets constant and note any candidate expansion in hybrid runs so fused rankings are drawn from comparable pools.

Evidence tracing checklist

Non-goals

Summary

Retrieval quality upgrades in Biblicus keep determinism intact while making scoring richer and more interpretable. Start with tuned lexical baselines, add reranking when you need sharper relevance, and reach for hybrid retrieval when you want to balance lexical precision with semantic similarity signals. Evaluate each change with the same dataset and budget so improvements remain credible and reproducible.