ObservationProtocol¶

Outcome-driven memory effects — the application layer reports how the agent used retrieved memories, and the ORM applies effects atomically.

Overview¶

ObservationProtocol provides three lifecycle hooks for passive behavioral inference on memory models:

on_read(instance) — fired when a query hydrates an instance. Delegates to AccessTrackerMixin.
on_surfaced(instances, reason) — fired when a proactive system pushes memories into agent context. Creates RecallProposal entries.
on_context_used(instances, outcome_map) — fired when the application reports how the agent responded. Applies effects based on outcome.

Outcomes¶

Five outcomes drive different effects:

Outcome	Confidence	Cycles	Pressure	Access	Predictions
`acted`	Corroborate (signal=0.9)	Strengthen (factor=1.2)	Resolve	Confirm	Auto-resolve (error=0.1)
`dismissed`	—	Weaken (factor=0.8)	—	Discard	Auto-resolve (error=0.5)
`deferred`	—	—	Keeps building	Discard	—
`contradicted`	Contradict (signal=0.1)	Aggressively weaken (factor=0.5)	Auto-discharge if confidence < 0.1	Discard	Auto-resolve (error=0.9)
`used`	—	—	—	Confirm	Auto-resolve (error=0.3)

`"used"` vs `"deferred"`¶

"used" and "deferred" are the two outcomes that do not emit a strength signal, but they are observably different:

"deferred" — agent set the memory aside without reading it. Staged reads are discarded (no confirmed-read trace). Pending predictions are left unresolved.
"used" — agent read and reasoned over the memory but did not cite it in the response. Staged reads are confirmed via AccessTrackerMixin.confirm_access(). Predictions are auto-resolved with a moderate error (0.3). No confidence, cycle, or decay signal is emitted.

Use "used" when the memory informed the agent's reasoning without appearing directly in the output — a common case that "acted" overcounts and "deferred" undercounts.

outcome_map = {
    memory1.db_key.redis_key: "acted",    # appeared in response
    memory2.db_key.redis_key: "used",     # informed reasoning, not cited
    memory3.db_key.redis_key: "dismissed",
    # memory4 not in map → defaults to "deferred"
}
ObservationProtocol.on_context_used(memories, outcome_map)

See Metacognitive Layer for the full effects comparison table.

Unsaved instances in a batch¶

on_context_used iterates every instance in instances and applies effects one at a time. An instance that has never been saved has no redis_key, so calling a model method on it (touch, confirm_access, strengthen_cycle, resolve_pressure, weaken_cycle) raises TypeError. Every applier — acted, dismissed, contradicted, and used (deferred has no method calls to guard) — catches that TypeError (and ValueError) around each call and moves on, so:

an unsaved member is skipped for that effect: it never raises out of on_context_used, and it never aborts the batch;
every other, saved instance in the same call still gets its full effects, applied through the same pipeline and execute()d as normal;
the model methods themselves are unchanged — calling touch(), confirm_access(), strengthen_cycle(), weaken_cycle(), or resolve_pressure() directly on an unsaved instance still raises TypeError, exactly as before. The degradation is a property of ObservationProtocol, not of the model methods it calls — a caller invoking those methods outside the protocol gets no such safety net.

Migrating custom outcomes¶

If you were using a custom "echoed" outcome (or any bespoke label between "used" and "dismissed"), map it to "used" when the agent reasoned over the memory or to "dismissed" when the overlap was coincidental; on_context_used() raises ValueError on unknown labels, so coerce to a valid outcome before calling.

Usage¶

from popoto.fields.observation import ObservationProtocol

# After agent processes memories:
outcome_map = {
    memory1.db_key.redis_key: "acted",
    memory2.db_key.redis_key: "dismissed",
    memory3.db_key.redis_key: "contradicted",
}
ObservationProtocol.on_context_used(memories, outcome_map)

Instances not in the outcome_map default to "deferred".

Proactive Surfacing¶

When a proactive system pushes memories into agent context:

ObservationProtocol.on_surfaced(memories, reason="proactive")

This creates RecallProposal entries in a Redis sorted set for tracking. Proposals expire after 1 hour (configurable via RecallProposal.DEFAULT_TTL).

Tuning Constants¶

All constants are configurable via Defaults:

from popoto.fields.constants import Defaults

Defaults.ACTED_CONFIDENCE_SIGNAL = 0.9
Defaults.CONTRADICTED_CONFIDENCE_SIGNAL = 0.1
Defaults.ACTED_CYCLE_STRENGTHEN_FACTOR = 1.2
Defaults.DISMISSED_CYCLE_WEAKEN_FACTOR = 0.8
Defaults.CONTRADICTED_CYCLE_WEAKEN_FACTOR = 0.5
Defaults.AUTO_DISCHARGE_CONFIDENCE_THRESHOLD = 0.1

Constant	Default	Optimal Range	Notes
`ACTED_CONFIDENCE_SIGNAL`	0.9	[0.5, 1.0]	Insensitive within range
`CONTRADICTED_CONFIDENCE_SIGNAL`	0.1	[0.05, 0.3]	Insensitive within range
`ACTED_CYCLE_STRENGTHEN_FACTOR`	1.2	[1.0, 2.0]	CLIFF EFFECT below 1.0
`DISMISSED_CYCLE_WEAKEN_FACTOR`	0.8	[0.3, 1.0]	Insensitive within range
`CONTRADICTED_CYCLE_WEAKEN_FACTOR`	0.5	[0.3, 0.8]	Insensitive within range
`AUTO_DISCHARGE_CONFIDENCE_THRESHOLD`	0.1	[0.05, 0.3]	Insensitive within range

Effects Matrix¶

Each row lists what the field/mixin does for each outcome. — means no effect.

Effect	acted	used	dismissed	deferred	contradicted
ConfidenceField	strengthen	—	—	—	weaken
CyclicDecayField	strengthen	—	weaken	—	weaken (aggressive)
DecayingSortedField	touch	—	—	—	—
AccessTracker	confirm	confirm	discard	discard	discard
PredictionLedger	auto-resolve	moderate err	auto-resolve	—	auto-resolve
ValidityField	—	—	—	—	close + chain (opt-in, see below)

Supporting notes:

DecayingSortedField: acted calls touch() to refresh the decay clock.
AccessTrackerMixin: acted and used call confirm_access(); dismissed, deferred, and contradicted call discard_staged_access().
CyclicDecayField: acted strengthens cycles and resolves pressure; dismissed and contradicted weaken cycles (contradicted more aggressively). These adjustments are durable — an ordinary save() preserves the learned amplitude rather than resetting it to the model's declared default, so repeated outcomes accumulate. See Learned amplitudes persist across saves. One ordering caveat: if you pass a shared pipeline to both apply_outcome() and save(), queue the save() first, or the amplitude adjustment is discarded when the pipeline executes.
ConfidenceField: acted corroborates; contradicted contradicts.
PredictionLedgerMixin: acted, used, dismissed, and contradicted auto-resolve pending predictions with appropriate error values (used maps to moderate error Defaults.PL_AUTO_RESOLVE_USED).
ValidityField (issue #580): a bare contradicted outcome is a no-op for this field — on_context_used has no slot in outcome_map for a second, correcting instance. To close the contradicted record's interval and chain it to its correction, tag the contradicted instance with the private _superseded_by attribute before reporting it:
```
stale._superseded_by = corrected
ObservationProtocol.on_context_used(
    [stale], {stale.db_key.redis_key: "contradicted"}
)
```
This routes through _apply_supersession, which delegates to SupersessionProtocol.invalidate — the same closure-and-chain primitive used by ValidityField and SupersessionProtocol. It is a strict no-op unless the model declares a ValidityField and _superseded_by is set; on every other model, or without the attribute, contradicted behaves exactly as the rest of this table describes.

Outcomes Now Reach Forgetting¶

Reporting an outcome used to affect ranking and cycle amplitudes but never the rate at which a memory left the corpus: a memory dismissed ten times decayed exactly as fast as one acted on ten times, because decay_rate was a field-level constant shared by every record. The only asymmetry was acted, which calls touch() and so slows effective decay by resetting the clock — with no matching mechanism by which dismissal could accelerate it.

The confidence this protocol writes now feeds two further consumers:

Decay rate. DecayingSortedField and CyclicDecayField derive a per-record effective decay rate from confidence (eff = decay_rate * 2 ^ (s * 2 * (c0 - c))), so a contradicted memory both ranks lower and fades faster. See Confidence-Modulated Decay.
Forget eligibility. MemoryLifecycle forgets a low-confidence idle record once it has at least FORGET_MIN_EVIDENCE observations behind it, tombstoning rather than deleting so the decision stays reversible.

The full loop: on_context_used(..., "dismissed") → confidence drops → next retrieval ranks the memory lower → next lifecycle tick tombstones it. Nothing new is required at the call site; the outcome reporting deployments already do is what drives it.

Neutrality is preserved for corpora with no outcome data at all — a record that has never been reported on scores byte-identically to pre-modulation Popoto.

RecallProposal¶

Internal ORM infrastructure for tracking proactively surfaced memories.

Key pattern: $RP:{ClassName}:pending:{partition} (sorted set scored by surfaced_at)
Lifecycle: pending -> acted | used | dismissed | deferred | contradicted | expired
TTL: 3600s (1 hour). Unresolved proposals are treated as deferred.

from popoto.fields.observation import RecallProposal

# Get pending proposals
pending = RecallProposal.get_pending(Memory, partition="default")

# Expire stale proposals
expired = RecallProposal.expire_stale(Memory, ttl=3600)