# User configuration

Biblicus supports a small user configuration file for optional integrations.

This is separate from corpus configuration. A corpus is a folder you can copy and share. User configuration usually contains machine-specific settings such as credentials.

## Where it looks

Biblicus looks for user configuration in two places, in this order.

1. `~/.biblicus/config.yml`
2. `./.biblicus/config.yml`

If both files exist, the local configuration overrides the home configuration.

## File format

The configuration file is YAML and is parsed using the `dotyaml` approach (YAML with optional environment variable interpolation).

## Reproducibility notes

User configuration is machine-specific. Do not bake secrets into corpora or datasets, and avoid storing user config
files in source control.

## Example: OpenAI speech to text

Create a config file with an OpenAI API key.

You can start from the included example configuration file:

- Copy `.biblicus/config.example.yml` to `~/.biblicus/config.yml`, or
- Copy `.biblicus/config.example.yml` to `./.biblicus/config.yml`

`~/.biblicus/config.yml`:

```yaml
openai:
  api_key: YOUR_KEY_HERE
```

The OpenAI speech to text extractor also supports the `OPENAI_API_KEY` environment variable. Environment takes precedence over configuration.

## Example: Deepgram speech to text

Create a config file with a Deepgram API key.

`~/.biblicus/config.yml`:

```yaml
deepgram:
  api_key: YOUR_KEY_HERE
```

The Deepgram speech to text extractor also supports the `DEEPGRAM_API_KEY` environment variable. Environment takes precedence over configuration.

## Example: Aldea speech to text

Create a config file with an Aldea API key.

`~/.biblicus/config.yml`:

```yaml
aldea:
  api_key: YOUR_KEY_HERE
```

The Aldea speech to text extractor also supports the `ALDEA_API_KEY` environment variable. Environment takes precedence over configuration.

## Source profiles (remote collections and corpora)

Biblicus supports **multiple source profiles** so you can connect to many accounts and providers at once.
Profiles live in `~/.biblicus/config.yml` or `./.biblicus/config.yml` (repo root).

Example profiles:

```yaml
sources:
  - name: azure-prod
    kind: azure-blob
    connection_string: YOUR_CONNECTION_STRING
    account_name: YOUR_ACCOUNT_NAME
  - name: s3-archive
    kind: s3
    access_key_id: YOUR_KEY_ID
    secret_access_key: YOUR_SECRET
    session_token: YOUR_SESSION_TOKEN
    region: us-east-1
    endpoint_url: https://s3.us-east-1.amazonaws.com
```

Environment variables override configuration when present:

- `AZURE_STORAGE_CONNECTION_STRING`
- `AZURE_STORAGE_ACCOUNT`
- `AZURE_STORAGE_KEY`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN`
- `AWS_REGION`

## Example: Neo4j graph extraction

Graph extraction uses a Neo4j backend. Biblicus can auto-start a local Neo4j Docker container if it is not already running.

Install the Neo4j Python driver before running graph extraction:

```
python -m pip install neo4j
```

If you use NLP-based graph extractors (for example `ner-entities` or `dependency-relations`), install the matching
Biblicus extra for the extractor. The `ner` extra includes spaCy and the default `en_core_web_sm` model.

`~/.biblicus/config.yml`:

```yaml
neo4j:
  uri: bolt://localhost:7687
  username: neo4j
  password: testpassword
  auto_start: true
  container_name: biblicus-neo4j
  docker_image: neo4j:5
  http_port: 7474
  bolt_port: 7687
```

Environment variables override configuration when present:

- `NEO4J_URI`
- `NEO4J_USERNAME`
- `NEO4J_PASSWORD`
- `NEO4J_DATABASE`
- `BIBLICUS_NEO4J_AUTO_START`
- `BIBLICUS_NEO4J_CONTAINER_NAME`
- `BIBLICUS_NEO4J_IMAGE`
- `BIBLICUS_NEO4J_HTTP_PORT`
- `BIBLICUS_NEO4J_BOLT_PORT`

## GROBID PDF extraction

URL text extraction converts PDFs with a GROBID service. Biblicus reads GROBID settings from environment variables only:

- `BIBLICUS_GROBID_URL`: base URL of the service. Default `http://127.0.0.1:8070`.
- `BIBLICUS_GROBID_AUTO_START`: whether Biblicus launches a local Docker container when no service is reachable at a localhost URL. Default **enabled** (any value other than `1`, `true`, `yes` or `on` disables it).
- `BIBLICUS_GROBID_CONTAINER_NAME`: name of the auto-started container. Default `biblicus-grobid`.
- `BIBLICUS_GROBID_DOCKER_IMAGE`: image for the auto-started container. Default `grobid/grobid:0.9.1-crf`, which publishes both `linux/arm64` and `linux/amd64` (the previous default was amd64 only and is killed for lack of memory under emulation on Apple Silicon).
- `BIBLICUS_GROBID_MEMORY_LIMIT`: Docker memory limit (also used as the swap limit) for the auto-started container. Default `5g`. GROBID peaks near 4.4 GiB on a long PDF and a 3g limit is killed at startup, because the Wapiti models load outside the Java heap.
- `BIBLICUS_GROBID_CONSOLIDATE_HEADER`: ask GROBID to consolidate the header against CrossRef. Default off.
- `BIBLICUS_GROBID_CONSOLIDATE_CITATIONS`: ask GROBID to consolidate citations against CrossRef. Default off.

The auto-started container is published on `127.0.0.1` only (not reachable from the network), restarts unless stopped, and runs with `JAVA_OPTS=-Xms512m -Xmx2g`.

Consolidation is off by default because it calls CrossRef during extraction. On a long PDF this hits CrossRef rate limits, can take several minutes (past the 180 second request timeout), and makes the extracted text depend on an external service. Enable it only when you accept that cost.

## Common pitfalls

- Saving secrets in the corpus directory instead of a user config file.
- Forgetting that local configuration overrides home configuration.
- Expecting user configuration to be copied with a corpus.
