Testing and coverage
Behavior specifications are written in features/*.feature and executed with Behave.
Coverage is measured across the src/biblicus/ package, excluding vendored third-party libraries, and a Hypertext Markup Language report is written to reports/htmlcov/index.html.
Run tests
Run everything (unit tests, baseline behavior specs, and integration scenarios):
python scripts/run_all_tests.py
The integration phase requires an OpenAI API key. The runner reads it from .biblicus/config.yml
under openai.api_key or from the OPENAI_API_KEY environment variable.
Run the test suite without integration downloads:
python scripts/test.py
Run the test suite including integration scenarios that download public test data at runtime:
python scripts/test.py --integration
Run the test suite including optical character recognition integration scenarios:
python scripts/test.py --integration --ocr
Run the test suite including Unstructured integration scenarios:
python scripts/test.py --integration --unstructured
Run only integration scenarios with a configured OpenAI API key:
OPENAI_API_KEY="..." BIBLICUS_RUN_MARKOV_DEMO=1 python -m behave features/integration_*.feature
Coverage expectations
Biblicus targets 100% coverage for src/biblicus/. Any uncovered behavior should be removed or turned into an explicit
error with a specification.
Integration datasets
Integration scenarios are tagged @integration.
The repository does not include downloaded content. Integration scripts download content into a corpus path you choose and then ingest it for a test run.
- AG News dataset:
scripts/download_ag_news.py - Portable Document Format samples:
scripts/download_pdf_samples.py - Image samples:
scripts/download_image_samples.py - Mixed modality samples:
scripts/download_mixed_samples.py - Audio samples:
scripts/download_audio_samples.py
Optional integrations in tests
Some integrations require credentials, such as speech to text.
Those are covered by unit-style behavior specifications using fake libraries, not by integration scenarios.
Docling integration scenarios require the Docling optional dependency (and its model downloads).
Unstructured integration scenarios require the Unstructured optional dependency and Poppler
(pdfinfo must be available on PATH).
Optical character recognition integration scenarios are tagged @ocr and are excluded unless you pass --ocr.
Unstructured integration scenarios are tagged @unstructured and are excluded unless you pass --unstructured.
Common pitfalls
- Running integration tests without optional dependencies installed.
- Forgetting to set API keys when testing speech-to-text integrations.
- Comparing coverage reports generated from different code revisions.