primus.score_rubric_consistency module

Score-version rubric consistency checks.

This module owns the lightweight preflight that asks whether the score code for a specific ScoreVersion appears consistent with that same version’s rubric text. The result is designed to be persisted on Evaluation.parameters and displayed as operator context before RCA.

class primus.score_rubric_consistency.ScoreRubricConsistencyRequest(scorecard_identifier: 'str', score_identifier: 'str', score_version_id: 'str', rubric_text: 'str', score_code: 'str', item_text: 'str' = '')

Bases: object

__init__(scorecard_identifier: str, score_identifier: str, score_version_id: str, rubric_text: str, score_code: str, item_text: str = '') → None

item_text: str = ''

rubric_text: str

score_code: str

score_identifier: str

score_version_id: str

scorecard_identifier: str

class primus.score_rubric_consistency.ScoreRubricConsistencyResult(scorecard_identifier: 'str', score_identifier: 'str', score_version_id: 'str', status: 'str', paragraph: 'str', checked_at: 'str', model: 'str', diagnostics: 'Dict[str, Any]')

Bases: object

__init__(scorecard_identifier: str, score_identifier: str, score_version_id: str, status: str, paragraph: str, checked_at: str, model: str, diagnostics: Dict[str, Any]) → None

checked_at: str

diagnostics: Dict[str, Any]

model: str

paragraph: str

score_identifier: str

score_version_id: str

scorecard_identifier: str

status: str

to_parameters_payload() → Dict[str, Any]

class primus.score_rubric_consistency.ScoreRubricConsistencyService(*, invoke_model: Callable[[str, str], str] | None = None, model: str = 'gpt-5-mini-2025-08-07', semantic_authority: Any = None, openai_client_factory: Callable[[], Any] | None = None, token_counter: Callable[[str, str], int] | None = None, max_input_tokens: int = 36000, max_output_tokens: int = 2000)

Bases: object

Generate a concise score-code vs rubric consistency assessment.

DEFAULT_MODEL = 'gpt-5-mini-2025-08-07'

MAX_ATTEMPTS = 2

MAX_INPUT_TOKENS = 36000

MAX_OUTPUT_TOKENS = 2000

VALID_STATUSES = {'consistent', 'inconclusive', 'potential_conflict'}

__init__(*, invoke_model: Callable[[str, str], str] | None = None, model: str = 'gpt-5-mini-2025-08-07', semantic_authority: Any = None, openai_client_factory: Callable[[], Any] | None = None, token_counter: Callable[[str, str], int] | None = None, max_input_tokens: int = 36000, max_output_tokens: int = 2000)

generate(request: ScoreRubricConsistencyRequest) → ScoreRubricConsistencyResult

generate_from_api(*, client: Any, scorecard_identifier: str, score_identifier: str, score_id: str, score_version_id: str, item_text: str = '') → ScoreRubricConsistencyResult

primus.score_rubric_consistency.fetch_score_version_for_consistency(client: Any, score_version_id: str) → Dict[str, Any]

primus.score_rubric_consistency.merge_consistency_result_into_parameters(parameters: Any, result: ScoreRubricConsistencyResult) → Dict[str, Any]