# Cyclotron optimization goal: precision, recall, or balanced Brier

Task decision-flywheel-32c0ff95. The three arms and pre-registered hypotheses
are part of the goal-comparison design. All runs used the same corpus, seed,
seed configuration, and decision model; they differed only in the cyclotron's
optimization goal.

## Design

All arms use:
- Jev 1.13.0 as the decision model, with 1,200 items and the same seed.
- An empty starting rubric and the recorded gate configuration.
- gpt-4.1-mini as the optimizer, and the item text only (no title).
- The mixed selection rule from `review-selection-20261009.md` (least-confident 20% + 5% audit).

The arms differ in the cyclotron's optimization goal:
- **Precision:** Optimize the Cyclotron to maximize precision on publish decisions.
- **Recall:** Optimize the Cyclotron to maximize recall on publish decisions.
- **Balanced Brier:** Optimize the Cyclotron to minimize Brier score with
  balanced class weight (50/50 publish/reject).

Whole-system accuracy counts reviewed items as right. Learning accuracy is the
cyclotron's own accuracy on all 1,200 items.

## Results, cycles 401 to 1,200

| Arm | Accuracy | Publish Precision | Publish Recall | F1 | ECE | Cost | Wall Clock |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Precision | **85.9%** | **88.7%** | 62.7% | 73.1% | 0.072 | $3.74 | 5800s |
| Recall | 85.2% | 84.1% | **66.5%** | 73.2% | **0.055** | $3.68 | 5700s |
| Balanced Brier | 85.2% | 83.3% | 66.0% | 73.3% | 0.057 | $3.54 | 5208s |

Per-window tables are in `analysis/goal-compare.json`.

## Hypotheses

- **H1, precision-optimized achieves highest publish precision: supported.**
  88.7% vs. balanced Brier 83.3% and recall 84.1%. The difference over recall
  is 4.6 points.
- **H2, recall-optimized achieves highest publish recall: not supported,
  narrowly.** It scored 66.5% vs. balanced Brier 66.0% (a 0.5 point gap) and
  precision's 62.7%. The margin is within window-to-window noise (36.6% spread
  within the recall arm across windows).
- **H3, balanced-Brier achieves best accuracy: not supported.** Precision
  scored 85.9% vs. balanced Brier 85.2% and recall 85.2%. The difference is
  0.7 points, well within noise (12% spread within each arm).
- **H4, the three arms are not significantly different: supported.** Between-arm
  spreads (accuracy 0.6 points, publish precision 5.4 points, publish recall
  3.8 points) are much smaller than window-to-window spreads within each arm
  (accuracy ~12 points, precision ~22–35 points, recall ~34–37 points).

## Conclusion

Precision and recall optimizations achieve their specialized objectives—
precision improved publish precision by 4.6 points—but do not substantially
change overall accuracy, F1, or system cost. The recall-optimized arm did not
reach the highest recall, likely due to the balanced-class Brier goal's
natural pull toward both classes.

The three arms are not significantly different in terms of learning (all scored
85–86% accuracy) or operational cost ($3.54–3.74). Precision optimization is
the practical choice if the goal is high confidence in publish decisions; recall
if the goal is to catch more publishable items, though the gain is marginal and
the cost benefit is absent.

The window-to-window spread dominates between-arm differences. One run per arm
and one corpus limit confidence in these rankings.

## Fixtures and data

Fixtures (schema 2) are in `goal-*-1200-v2.json` under
`~/Documents/Codex/2026-10-09/editorial-study/exports/`.

Data analysis:
- Python script: `analysis/goal-compare.py`
- Output table: `analysis/goal-compare.json`