Biblicus

Make your documents usable by your assistant, then decide later how you will search and retrieve them.

Context packs

A context pack is the text that your application sends to a large language model.

Biblicus keeps two things separate:

This separation makes retrieval repeatable and testable, while keeping context formatting as an explicit surface you can change, compare, and evaluate.

Minimal policy

The minimal policy is: join evidence text blocks with a separator.

In Python:

from biblicus.context import ContextPackPolicy, build_context_pack

policy = ContextPackPolicy(join_with="\n\n")
context_pack = build_context_pack(result, policy=policy)
print(context_pack.text)

Output structure

Context pack building returns a structured result you can inspect:

{
  "text": "item_id: ...",
  "evidence_count": 2,
  "blocks": [
    {
      "evidence_item_id": "ITEM_ID",
      "text": "item_id: ITEM_ID\nsource_uri: ...",
      "metadata": {
        "item_id": "ITEM_ID",
        "source_uri": "file:///...",
        "score": 0.42,
        "stage": "retrieve"
      }
    }
  ]
}

blocks keeps a per-evidence record so you can debug how the final text was assembled.

Before and after example

Given two evidence blocks, compare how different policies change the output:

policy = ContextPackPolicy(join_with="\n\n", ordering="rank", include_metadata=False)
context_pack = build_context_pack(result, policy=policy)
print(context_pack.text)

With metadata enabled and score ordering:

policy = ContextPackPolicy(join_with="\n\n", ordering="score", include_metadata=True)
context_pack = build_context_pack(result, policy=policy)
print(context_pack.text)

The first output keeps the original ranking and clean text blocks. The second output reorders by score and adds explicit metadata lines for inspection.

Policy surfaces

Context pack policies make ordering and formatting explicit.

Ordering

Use ordering to control how evidence blocks are arranged before joining:

Metadata inclusion

Set include_metadata=True to prepend metadata to each block. Metadata includes:

Character budgets

Character budgets drop trailing blocks until the context pack fits the specified limit. This keeps context shaping deterministic without relying on a tokenizer.

In Python:

from biblicus.context import CharacterBudget, ContextPackPolicy, fit_context_pack_to_character_budget

policy = ContextPackPolicy(join_with="\n\n", ordering="score", include_metadata=True)
fitted = fit_context_pack_to_character_budget(context_pack, policy=policy, character_budget=CharacterBudget(max_characters=500))
print(fitted.text)

Command-line interface

The command-line interface can build a context pack from a retrieval result by reading JavaScript Object Notation from standard input.

biblicus query --corpus corpora/example --query "primary button style preference" \\
  | biblicus context-pack build --ordering score --include-metadata --max-characters 500

Reproducibility checklist

What context pack building does

Common pitfalls

Token budgets

Fitting context to a token budget is a separate concern. Token counting depends on a specific tokenizer and may vary by model.

Biblicus treats token budgeting as a separate stage so it can be configured, tested, and evaluated independently from retrieval and text formatting.

In Python:

from biblicus.context import (
    ContextPackPolicy,
    TokenBudget,
    fit_context_pack_to_token_budget,
)

fitted_context_pack = fit_context_pack_to_token_budget(
    context_pack,
    policy=policy,
    token_budget=TokenBudget(max_tokens=500),
)
print(fitted_context_pack.text)

Ordered context blocks

Some systems assemble context from doctrine, summaries, or memory sections before budgeting. Biblicus supports that by accepting an ordered list of context blocks instead of a retrieval result.

In Python:

from biblicus.context import (
    ContextBlock,
    ContextBlockBuildRequest,
    ContextSectionBudget,
    build_context_pack_from_blocks,
)

request = ContextBlockBuildRequest(
    blocks=[
        ContextBlock(block_id="doctrine-1", section="doctrine", text="Mission paragraph", required=True),
        ContextBlock(block_id="memory-1", section="desk_memory", text="Recent assignment summary"),
        ContextBlock(block_id="memory-2", section="desk_memory", text="Older assignment summary"),
    ],
    max_tokens=20,
    section_budgets=[
        ContextSectionBudget(section="doctrine", share=0.25),
        ContextSectionBudget(section="desk_memory", share=0.75),
    ],
)
result = build_context_pack_from_blocks(request)
print(result.text)

The command-line interface can build the same structure from standard input:

cat context-blocks.json | biblicus context-pack build-blocks

The result includes final text plus the included and dropped blocks, section token counts, and applied budgets.