Skip to content

Deterministic CSR Regression Gate

Report-only CI gate — not a leaderboard metric

The Constraint Satisfaction Rate (CSR) harness is a deterministic, per-PR regression gate (bit-identical every run — no LLM judge, no embeddings, no Redis module; identical on Redis and Valkey). Its RSR (standard/adversarial) and Adversarial Gap numbers are report-only signals for catching regressions, not a "higher is better" leaderboard score.

Run date: 2026-07-03T09:37:37.974131+00:00
Python: 3.12.13
Platform: macOS-26.3.1-arm64-arm-64bit

Summary

Metric Value
Cases 8
Errors 0
RSR (standard) 0.8750
RSR (adversarial) 0.2750
Adversarial Gap 0.6000

Gap alert: Adversarial Gap >= 0.3 — likely keyword-dependent retrieval. Expected of the pure-BM25 lexical path (healthy signature); this is NOT the #409 query-blindness signature.

Query-blindness flags (the #409 signature — identical rankings + standard CSR < 0.5): composite_control_409

Per primitive

Primitive n CSR (std) CSR (adv)
coverage 1 1.0000 0.3333
exclusion 1 1.0000 0.3333
lexical_recall 1 1.0000 0.2500
ordering 1 1.0000 0.3333
precision 1 1.0000 0.5000
query_independence 2 0.5000 0.1000
recency 1 1.0000 0.2500

Per case

Case Primitive Status CSR (std) CSR (adv) Rankings identical Query-blind
lexical_recall_basic lexical_recall ok 1.0000 0.2500 False False
ordering_score_separated ordering ok 1.0000 0.3333 False False
recency_none_older recency ok 1.0000 0.2500 False False
coverage_all_topic_facts coverage ok 1.0000 0.3333 False False
exclusion_unrelated exclusion ok 1.0000 0.3333 False False
precision_multi_topic precision ok 1.0000 0.5000 False False
query_blind_409 query_independence ok 1.0000 0.2000 False False
composite_control_409 query_independence ok 0.0000 0.0000 True True

Notes

  • Deterministic: lexical/BM25 + composite control only — no LLM judge, no embedding model, no Redis module. Identical on Redis and Valkey.
  • A LARGE Adversarial Gap on the lexical path = keyword dependence (healthy). Query-blindness (#409) = identical rankings for both queries AND a low standard CSR.
  • executed_path is recorded per run in the JSON; a composite fallback can never masquerade as a lexical number.