popoto.extraction.decision_log¶
popoto.extraction.decision_log
¶
The write-side audit store for auditable extraction (M3).
Every candidate the generator produces gets a row here, and every row ends
in exactly one terminal state -- firewall_drop | accept | reject |
withhold (issue #562). Extraction precision and recall become computable
offline from these rows alone: "audit by construction".
Two properties carry the whole design, and both are structural rather than conventional -- neither depends on a caller remembering to do the right thing.
1. The row is identified by the candidate, not by a mint-on-save id.
agent_id, turn_id and candidate_id are all KeyFields, so
the composite Redis key is the candidate's identity and re-saving the
same tuple transitions that row in place. Do not "fix" this by copying
the sibling :class:~popoto.recipes.provenance_journal.JournalEntry, which
uses entry_id = AutoKeyField(): an AutoKeyField mints a brand-new
row on every save, which is right for an append-only journal and fatal
here. It would leave two rows behind every pending -> terminal
transition, make the terminal-write guard read an empty row and never
refuse, and make list_pending return already-reconciled rows forever.
AutoKeyField is forbidden on :class:DecisionRecord.
Two consequences of composite keying worth knowing before you read a key by eye or by position:
- KeyFields join alphabetically, not in declaration order
(
models/base.py:284-301), so the Redis key isDecisionRecord:<agent_id>:<candidate_id>:<turn_id>--candidate_idin the middle. DB_keyescapes colons and glob characters inside values (models/db_key.py:86-88), so acandidate_idoft-41:sent:0renders ast/-41{:}sent{:}0. That is correct; do not "fix" it.
2. The decision log is written before every irreversible side effect. A candidate never reaches a side effect that has no row already describing it:
firewall_drop/reject/withholdhave no downstream side effect, so their first write is also their terminal write.acceptis the only two-phase path: :meth:DecisionLog.write_pendingcommits a non-terminalpendingrow before assembly callsProvenanceJournal.append(), and :meth:DecisionLog.write_terminaltransitions that same row afterwards.pendingis not a fifth terminal state -- it is a visible unfinished write, which is the opposite of a silent drop. A survivingpendingrow means a process died mid-assembly.
Recovering stale pending rows is manual in v1 -- nothing sweeps,
expires or alerts on them, and decision rows deliberately carry no TTL
(a TTL would delete the audit evidence this module exists to keep). The
operator recipe is:
log = DecisionLog()
for row in log.list_pending("agent-7"): # oldest-first
print(row.turn_id, row.candidate_id, row.written_at)
# then re-invoke the auditable path for each stale (agent_id, turn_id);
# assembly's identity probe reconciles it.
A periodic age-keyed sweep, an alert threshold and any dashboard over this
log are M9 (#568) follow-ons, not v1. Detail rows ship unbounded on
purpose: the right retention horizon cannot be known until M9 consumes the
log, so v1 declines to guess a number. There is no LTRIM here and no
Defaults cap constant for this log.
CLAIM_KEY_PREFIX = 'popoto:m3:claim'
module-attribute
¶
Prefix for the ephemeral assembly-claim keys.
A separate, extraction-owned keyspace from the decision rows. Claim keys carry a TTL and hold no audit content, so losing one costs at most a reprobe -- never a record.
CLAIM_RELEASE_LUA = "\n-- KEYS[1] = the claim key, ARGV[1] = this runner's token\nif redis.call('GET', KEYS[1]) == ARGV[1] then\n return redis.call('DEL', KEYS[1])\nend\nreturn 0\n"
module-attribute
¶
Token-checked release. A runner never deletes a claim it lost.
SUMMARY_KEY_PREFIX = 'popoto:m3:summary'
module-attribute
¶
Prefix for the per-turn compact summary hash.
The summary is a convenience index, not a completeness fallback -- detail rows are unbounded and remain the sole source of truth. It exists so per-turn terminal-state counts are O(1) instead of a scan over every candidate row for the turn. Any summary/detail disagreement is resolved in favour of the detail rows.
TURN_SUMMARY_REBUILD_LUA = "\n-- Recompute one turn's summary hash from its detail rows (operator repair).\n-- KEYS[1] = the per-turn summary hash\n-- KEYS[2] = the agent_id KeyField index Set, KEYS[3] = the turn_id one\n-- ARGV = the terminal state names (not hard-coded here)\n--\n-- The row keys are read from SINTER KEYS[2] KEYS[3] rather than declared in\n-- KEYS: an undeclared-key access, fine on standalone Redis and Valkey and\n-- not Redis Cluster-safe (neither is the decision log, whose summary and row\n-- keys share no hash tag). One EVAL, so TERMINAL_WRITE_LUA calls serialize\n-- around it. Keep the 'state:' / 'reason:' prefixes in step with\n-- DecisionLog.write_terminal's summary_fields.\nlocal terminal = {}\nfor i = 1, #ARGV do\n terminal[ARGV[i]] = true\nend\nlocal counts = {}\nlocal order = {}\nlocal function bump(field)\n if counts[field] == nil then\n counts[field] = 0\n order[#order + 1] = field\n end\n counts[field] = counts[field] + 1\nend\nfor _, row in ipairs(redis.call('SINTER', KEYS[2], KEYS[3])) do\n local packed_state = redis.call('HGET', row, 'state')\n if packed_state then\n local state = cmsgpack.unpack(packed_state)\n if terminal[state] then\n local packed_reason = redis.call('HGET', row, 'reason_code')\n bump('state:' .. state)\n bump('reason:' .. (packed_reason and cmsgpack.unpack(packed_reason) or ''))\n end\n end\nend\nredis.call('DEL', KEYS[1])\nlocal out = {}\nfor _, field in ipairs(order) do\n redis.call('HSET', KEYS[1], field, counts[field])\n out[#out + 1] = field\n out[#out + 1] = counts[field]\nend\nreturn out\n"
module-attribute
¶
Repair script for a summary hash that disagrees with its rows.
Hashes written before the #811 fix (a terminal-to-terminal write), and
hashes written before the #822 fix (a pending write over a terminal row,
which used to be a plain save that left the row's counts in place).
DecisionRecord
¶
Bases: Model
One candidate's decision row, keyed by the candidate's identity.
agent_id + turn_id + candidate_id form the composite key,
so writing the same tuple twice transitions one row rather than
creating two. See the module docstring for why AutoKeyField is
forbidden here and for the alphabetical key-join order.
Every field below is a plain (non-indexed) field on purpose. Popoto
routes IndexedFieldMixin fields out of the base HSET and into
INDEX_SWAP_LUA (models/base.py:1470-1480), so an indexed
state would be invisible to the guard script's HGET and its
index would be silently desynchronised by the guard's HSET.
list_pending filters in Python instead; it is an operator recovery
reader, not a hot path.
Attributes:
| Name | Type | Description |
|---|---|---|
agent_id |
Owning agent. KeyField. |
|
turn_id |
The turn the candidate was generated from. KeyField. |
|
candidate_id |
|
|
state |
A :class: |
|
reason_code |
A :class: |
|
generator_rule |
Which deterministic rule produced the candidate, so offline metrics can break results down per rule. |
|
span_start |
Candidate span start offset in the turn text. |
|
span_end |
Candidate span end offset in the turn text. |
|
text_hash |
SHA-256 of the candidate text. A digest is safe here
because nothing scans a decision row; the |
|
entry_id |
The journal entry id, set on a terminal |
|
detail_code |
Free-form diagnostic string, written only by trusted
code. See :attr: |
|
written_at |
Unix timestamp, stamped on every write -- both the
|
Source code in src/popoto/extraction/decision_log.py
is_terminal
property
¶
True when :attr:state is one of the four terminal states.
DecisionLog
¶
Writer/reader over :class:DecisionRecord rows.
Stateless -- every method takes the identity it operates on -- so one
instance can serve any agent. On Redis/Valkey all writes go through core
commands and Lua only; on Postgres (when DecisionRecord's backend is
not Redis) each method is one SQL statement through the backend, with the
same semantics, and no Redis command is issued.
Example::
log = DecisionLog()
log.write_terminal(
agent_id="agent-7",
candidate=candidate,
state=Verdict.FIREWALL_DROP,
reason_code=ReasonCode.PRE_LLM_CANDIDATE_BLOCK,
)
Source code in src/popoto/extraction/decision_log.py
421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 | |
CONFLICT_REFUSED = 'terminal_conflict_refused'
class-attribute
instance-attribute
¶
detail_code recorded when a terminal write is refused.
detail_code is a free-form diagnostic string, not an enum: it
carries three structurally different payloads by design -- this fixed
literal, an exception class name on assembly_failed, and
",".join(entry_ids) on ambiguous_reconciliation. That does not
weaken the enums-only constraint on the model's output: state and
reason_code are genuine single-value enums, and detail_code is
written exclusively by trusted code, never by the LLM.
row_key(agent_id, turn_id, candidate_id)
staticmethod
¶
The Redis key for one candidate's row.
Built through DB_key rather than by string formatting, so the
alphabetical join order and the colon/glob escaping stay in one
place (the model) instead of being re-derived by every caller.
Source code in src/popoto/extraction/decision_log.py
summary_key(agent_id, turn_id)
staticmethod
¶
claim_key(agent_id, turn_id, candidate_id)
staticmethod
¶
The ephemeral assembly-claim key for one candidate.
Carries a TTL and holds no audit content -- losing it costs at most a re-probe, never a record.
Source code in src/popoto/extraction/decision_log.py
key_field_index_keys(agent_id, turn_id, candidate_id)
staticmethod
¶
The KeyField secondary index Sets for one row's key values.
Built with KeyFieldMixin's own key builder rather than by
formatting the $KeyF: pattern here, so the guard script SADDs
into exactly the Sets on_save would have.
Source code in src/popoto/extraction/decision_log.py
write_pending(agent_id, candidate, reason_code=ReasonCode.ACCEPTED)
¶
Commit the non-terminal pending row for an accepted candidate.
Phase 1 of the two-phase accept path. This write is committed
before ProvenanceJournal.append() is called, never
pipelined with it -- that ordering is what guarantees no candidate
can reach an irreversible side effect with zero decision-log rows.
pending is never counted into the per-turn summary, which
aggregates terminal states only. A pending write over a row that
is already terminal (an unclaimed non-accept write can land
between :meth:assemble's row read and this call) therefore takes
that row's counts back out, in the same atomic step as the row write:
PENDING_WRITE_LUA on Redis, while Postgres derives the summary
from the rows on read. Either way :meth:turn_summary stays the
rollup of the rows (#822). Like the plain save it replaced, the write
is unguarded: it overwrites any row.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
agent_id
|
str
|
Owning agent. Never |
required |
candidate
|
Candidate
|
The candidate being assembled. |
required |
reason_code
|
Union[ReasonCode, str]
|
The verdict's reason code, carried forward. |
ACCEPTED
|
Returns:
| Type | Description |
|---|---|
DecisionRecord
|
The saved :class: |
Source code in src/popoto/extraction/decision_log.py
write_terminal(agent_id, candidate, state, reason_code, entry_id='', detail_code='')
¶
Write a terminal state through the guarded script.
This is the only way any terminal state is written. Every
terminal state routes through it, including the pre-LLM
firewall_drop that cannot conflict in practice -- there is no
fast path to bypass.
The guard applies uniformly rather than only to non-accept
writes. Applying it to accept too costs nothing and removes the
last conditional branch: the legitimate pending -> accept
transition is never refused (a pending row is not accept
with an entry_id), and the only accept write it does refuse
is a duplicate assembly of an already-assembled candidate, where
refusing is correct.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
agent_id
|
str
|
Owning agent. |
required |
candidate
|
Candidate
|
The candidate being decided. |
required |
state
|
Union[Verdict, str]
|
A terminal :class: |
required |
reason_code
|
Union[ReasonCode, str]
|
A :class: |
required |
entry_id
|
str
|
Journal entry id. Required on |
''
|
detail_code
|
str
|
Free-form trusted diagnostic string. |
''
|
Returns:
| Type | Description |
|---|---|
bool
|
|
bool
|
refused because the row is already terminal |
bool
|
|
bool
|
existing row stands and its |
bool
|
attr: |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/popoto/extraction/decision_log.py
604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 | |
acquire_claim(agent_id, turn_id, candidate_id)
¶
Claim a candidate for assembly. One atomic op, not read-then-act.
Without this, two runners over the same candidate -- a duplicated
delivery racing a crash-retry, both seeing a surviving pending
-- can both probe, both find nothing (neither has committed
append() yet), and both append. That produces two permanent
journal entries, and the journal cannot catch it: append()
takes no idempotency key and AppendOnlyViolation fires only when
a record's Redis key already exists, which is never true for a fresh
AutoKeyField append.
SET ... NX PX is one round trip and a core command on both Redis
and Valkey. It is chosen over WATCH/MULTI compare-and-set
deliberately: WATCH needs a dedicated connection held across the
transaction plus a retry loop, which does not compose with Popoto's
shared pool. Both are Valkey-safe; SET NX is smaller.
Returns:
| Type | Description |
|---|---|
Optional[str]
|
This runner's token if the claim was won, else |
Optional[str]
|
runner holding |
Optional[str]
|
no row transition -- it no-ops and leaves the candidate to |
Optional[str]
|
the winner. |
Source code in src/popoto/extraction/decision_log.py
release_claim(agent_id, turn_id, candidate_id, token)
¶
Release a claim this runner still owns.
Token-checked so a runner can never delete a claim it no longer owns -- if the TTL expired and another runner re-claimed, the DEL would otherwise hand that runner's candidate to a third.
Source code in src/popoto/extraction/decision_log.py
assemble(agent_id, candidate, journal, speaker=None, topic_tags=(), resolution=None)
¶
Assemble one accepted candidate into the provenance journal.
The full ordering, which is the reason this is one function rather than a few helpers a caller sequences by hand: claim -> read row -> (probe) -> pending -> append -> terminal transition -> release.
Dedup is owned here and cannot be delegated to the journal:
append() accepts no idempotency key, JournalEntry.entry_id
is an AutoKeyField, and M1 is append-only with no delete path,
so a duplicate entry would be permanent.
The four-case probe on the existing row:
- terminal
acceptwith anentry_id-- already assembled, skip entirely. - any other terminal state -- already decided, skip.
pending-- a retry of an interrupted assembly. Reconcile by candidate identity (see below) before considering a re-append.- absent -- fresh candidate. Write
pending, thenappend().
Reconciliation matches the cand:{candidate_id} subject tag, not
verbatim text. Text matching is unsound here and the plan
withdrew it: verbatim is not unique per candidate within a turn
by construction -- a repeated sentence, or a sentence span whose
text equals an entity-lifted span, produces two candidates with
identical verbatim, and a pending row could reconcile onto
the other candidate's entry and record the wrong entry_id.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
agent_id
|
str
|
Owning agent. Required and never |
required |
candidate
|
Candidate
|
The accepted candidate. |
required |
journal
|
Any
|
The |
required |
speaker
|
Optional[str]
|
Attribution, passed through to the entry. |
None
|
topic_tags
|
Sequence[str]
|
Extra subject tags. The |
()
|
resolution
|
Optional[Resolution]
|
The M4 (#563) :class: |
None
|
Returns:
| Type | Description |
|---|---|
Optional[str]
|
The journal |
Optional[str]
|
one, else |
Optional[str]
|
failed -- in which case a terminal row records why). |
Source code in src/popoto/extraction/decision_log.py
803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 | |
get(agent_id, turn_id, candidate_id)
¶
Read one candidate's row, or None if it has none yet.
Source code in src/popoto/extraction/decision_log.py
list_for_agent(agent_id)
¶
Every row for an agent, in no particular order.
Reads by key pattern rather than through
DecisionRecord.query.filter(agent_id=...) on purpose, and
this is load-bearing rather than a style choice. Exact-match
KeyField filtering resolves through secondary index Sets that
KeyFieldMixin.on_save maintains
(fields/key_field_mixin.py:422+), and the guarded terminal
write is a deliberate ORM bypass -- it writes the row hash from
Lua, so on_save never runs and those index Sets are never
built. A candidate whose first write is terminal (every
firewall_drop / reject / withhold) would therefore be
invisible to filter() while sitting in Redis in full. Scanning
the keyspace has no such dependency.
Uses SCAN rather than KEYS so an unbounded log cannot block
the server. This is an operator/analysis reader, not a hot path.
All of the above is the Redis path. On Postgres the table's own
indexes cover every row however it was written, so this is an
indexed WHERE agent_id = ... query through the ORM.
Source code in src/popoto/extraction/decision_log.py
list_pending(agent_id, older_than=None)
¶
Stale pending rows for an agent, oldest-first.
The operator recovery reader (see the module docstring). A thin
reader over existing rows -- it adds no keyspace, and decision rows
carry no TTL, so nothing here deletes audit evidence. It reads through
:meth:list_for_agent, so it works on either backend.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
agent_id
|
str
|
Owning agent. |
required |
older_than
|
Optional[float]
|
If given, only rows whose |
None
|
Returns:
| Type | Description |
|---|---|
List[DecisionRecord]
|
Matching rows sorted ascending by |
Source code in src/popoto/extraction/decision_log.py
turn_summary(agent_id, turn_id)
¶
The per-turn compact summary as {field: count}.
A convenience index over the detail rows: a rollup of the rows'
current terminal states. Each candidate contributes one
state:<s> and one reason:<r> count (an empty reason is
reason:) under the state and reason its row holds now, so a
candidate that moves between terminal states is counted under its new
state only. Zero counts are absent and state:pending never
appears. Correctness never depends on it -- if it ever disagrees with
the detail rows, the detail rows are right.
Redis keeps a hash that TERMINAL_WRITE_LUA maintains atomically
with the row; Postgres derives it on read with an indexed
GROUP BY and cannot drift. Every Redis write that changes a row's
state updates the hash in the same script as the row:
TERMINAL_WRITE_LUA for terminal writes and PENDING_WRITE_LUA
for :meth:write_pending, which takes back the counts of a terminal
row it overwrites (#822). Both backends therefore return the same
dict for the same sequence of writes, in any interleaving. A hash
written by an earlier version can still disagree with its rows; see
:meth:rebuild_turn_summary.
Source code in src/popoto/extraction/decision_log.py
rebuild_turn_summary(agent_id, turn_id)
¶
Recompute a turn's summary from its detail rows and return it.
The repair for Redis summary hashes that disagree with their rows,
which current code never produces: hashes written before the #811
fix (a terminal-to-terminal write that left the old state counted),
and before the #822 fix (a pending write over a terminal row was
a plain save that left the row's counts in place). Run it per turn
once every process runs the fixed version. During a rolling upgrade
an old process can still write either shape, and a new process's
take-back can then remove a count the hash does not hold for that
row, so a mixed fleet's hash may be off in either direction until
it is rebuilt.
One EVAL (TURN_SUMMARY_REBUILD_LUA), so concurrent terminal
writes serialize around it. An operator call, never on the hot path.
On Postgres the summary is derived on read and cannot drift, so this
returns :meth:turn_summary unchanged.
Source code in src/popoto/extraction/decision_log.py
compute_metrics(agent_id, gold_labels)
¶
Precision/recall/F1 for one agent, from the log alone.
Reads decision-log rows and a caller-supplied gold-label mapping and nothing else -- no journal read, no LLM, no other live Redis state. That isolation is the point: it is what makes "extraction quality is computable offline" true rather than aspirational, and a test re-runs this with the journal keyspace flushed to prove it.
A row counts as a positive prediction when its state is accept.
Rows with no gold label are excluded from precision/recall (they are
still counted in the breakdowns), so a partially-labelled corpus
does not silently score as a pile of false positives.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
agent_id
|
str
|
Owning agent. |
required |
gold_labels
|
Dict[str, bool]
|
|
required |
Returns:
| Name | Type | Description |
|---|---|---|
A |
Metrics
|
class: |
Source code in src/popoto/extraction/decision_log.py
1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 | |
Metrics
dataclass
¶
Offline extraction quality, computed from decision-log rows alone.
Attributes:
| Name | Type | Description |
|---|---|---|
precision |
float
|
|
recall |
float
|
|
f1 |
float
|
Harmonic mean of the two, 0.0 when undefined. |
true_positives |
int
|
Accepted and gold-labelled accept. |
false_positives |
int
|
Accepted and gold-labelled reject. |
false_negatives |
int
|
Not accepted but gold-labelled accept. |
per_reason_code |
Dict[str, int]
|
|
per_generator_rule |
Dict[str, Dict[str, int]]
|
|
Source code in src/popoto/extraction/decision_log.py
AuditableExtractionConfig
dataclass
¶
Opt-in configuration for the auditable extraction path.
Passing auditable_extraction=None (the default) is the only thing
the existing path ever sees, and its behavior is byte-for-byte
unchanged.
Deliberately carries no numeric knobs:
Defaults.M3_ASSEMBLY_CLAIM_TTL_MS is a pinned in-repo constant, not
a config field, per the repo's magic-number rule.
Attributes:
| Name | Type | Description |
|---|---|---|
verdict_provider |
Any
|
Anything callable as
|
journal |
Any
|
The :class: |
resolution_provider |
Any
|
Anything callable as
|
Source code in src/popoto/extraction/decision_log.py
hash_candidate_text(text)
¶
SHA-256 of the candidate text, for tamper-evidence on the row.