`popoto.fields.decaying_sorted_field`¶

`popoto.fields.decaying_sorted_field` ¶

DecayingSortedField — time-weighted scoring via Lua decay computation.

This module provides a SortedField subclass where records lose relevance over time following power-law decay: base_score * elapsed_days^(-decay_rate).

The sorted set stores timestamps as scores. A Lua script computes decay-ranked results at query time, reading base scores from each member's model hash via cmsgpack.

Design

Timestamps are always stored as scores (auto_now=True behavior)
Decay computation happens server-side in Lua (no round trips)
Base scores come from a companion field on the same model hash
When base_score_field is None, all items have equal base score (1.0)

Example

class Memory(Model): key = UniqueKeyField() content = StringField() strength = FloatField(default=1.0) last_accessed = DecayingSortedField( decay_rate=0.5, base_score_field="strength", )

Query top-10 memories by decayed relevance¶

top = Memory.query.top_by_decay("last_accessed", n=10)

Refresh a memory's decay clock without full save¶

memory.touch("last_accessed")

`DecayingSortedField` ¶

Bases: SortedFieldMixin, Field

A SortedField subclass where records lose relevance over time.

Stores timestamps as sorted set scores. A Lua script computes decay-ranked results at query time using power-law decay: decayed_score = base_score * elapsed_days ^ (-decay_rate)

With decay_rate=0.5, a record scores 1.0 after 1 day, 0.5 after 4 days, and 0.1 after 100 days.

Ranking is deterministic: equal-scored members are ordered by member key (redis_key) ascending, byte-wise, broken inside the Lua script before top-N truncation.

Parameters:

Name	Description	Default
`decay_rate`	Controls how fast scores drop. Higher = faster decay. Defaults to `Defaults.DECAY_RATE` (0.1). Must be > 0.	required
`base_score_field`	Name of a companion field whose value multiplies the decay curve. When None, base score is 1.0.	required
`confidence_modulation_field`	Controls confidence-modulated decay (issue #491). `None` (default) auto-detects a single `ConfidenceField` on the model; a `str` names one explicitly; `False` disables modulation for this field. Modulation is also globally disabled by `Defaults.DECAY_CONFIDENCE_MODULATION_ENABLED = False`, which needs no model-code edit (deploy-level kill switch). Every disabled path is a byte-identical no-op.	required
`partition_by`	Partition the sorted set by key field values. Inherited from SortedFieldMixin.	required

Source code in src/popoto/fields/decaying_sorted_field.py

class DecayingSortedField(SortedFieldMixin, Field):
    """A SortedField subclass where records lose relevance over time.

    Stores timestamps as sorted set scores. A Lua script computes
    decay-ranked results at query time using power-law decay:
        decayed_score = base_score * elapsed_days ^ (-decay_rate)

    With decay_rate=0.5, a record scores 1.0 after 1 day, 0.5 after
    4 days, and 0.1 after 100 days.

    Ranking is deterministic: equal-scored members are ordered by member
    key (redis_key) ascending, byte-wise, broken inside the Lua script
    before top-N truncation.

    Args:
        decay_rate: Controls how fast scores drop. Higher = faster decay.
            Defaults to ``Defaults.DECAY_RATE`` (0.1). Must be > 0.
        base_score_field: Name of a companion field whose value multiplies
            the decay curve. When None, base score is 1.0.
        confidence_modulation_field: Controls confidence-modulated decay
            (issue #491). ``None`` (default) auto-detects a single
            ``ConfidenceField`` on the model; a ``str`` names one explicitly;
            ``False`` disables modulation for this field. Modulation is also
            globally disabled by ``Defaults.DECAY_CONFIDENCE_MODULATION_ENABLED
            = False``, which needs no model-code edit (deploy-level kill
            switch). Every disabled path is a byte-identical no-op.
        partition_by: Partition the sorted set by key field values.
            Inherited from SortedFieldMixin.
    """

    def __init__(self, **kwargs):
        decay_rate = kwargs.pop("decay_rate", None)
        self.decay_rate = decay_rate if decay_rate is not None else Defaults.DECAY_RATE
        self.base_score_field = kwargs.pop("base_score_field", None)
        self.confidence_modulation_field = kwargs.pop(
            "confidence_modulation_field", None
        )
        if not (
            self.confidence_modulation_field is None
            or self.confidence_modulation_field is False
            or isinstance(self.confidence_modulation_field, str)
        ):
            raise ModelException(
                "confidence_modulation_field must be None (auto-detect), False "
                "(disabled), or the str name of a ConfidenceField (got "
                f"{self.confidence_modulation_field!r})"
            )
        # Per-model-class resolution cache. Keyed by model class, NOT by the
        # kill switch -- the switch is re-read on every call so toggling it at
        # runtime takes effect immediately.
        self._confidence_modulation_cache: dict[Any, Any] = {}

        if self.decay_rate <= 0:
            raise ModelException(
                "decay_rate must be > 0 (got {})".format(self.decay_rate)
            )

        # Force type=float and auto_now=True behavior
        kwargs["type"] = float
        kwargs["auto_now"] = True
        kwargs["sorted"] = True
        super().__init__(**kwargs)

`resolve_confidence_modulation_field(model_class, field, field_name)` ¶

Resolve which ConfidenceField modulates field's decay.

Resolution order (first match wins):

Defaults.DECAY_CONFIDENCE_MODULATION_ENABLED is False -> off. This is the deploy-level kill switch: it disables modulation without any model-code edit, for adopters who cannot edit model definitions.
confidence_modulation_field=False -> off (per-field opt-out).
confidence_modulation_field="name" -> that field, or ModelException if it is missing or is not a ConfidenceField.
confidence_modulation_field=None (default) -> auto-detect over model_class._meta.fields. Exactly one ConfidenceField is used; zero means off; two or more means off plus a warning naming the candidates. Guessing between two confidence signals would silently pick a ranking policy the adopter never chose.

Returns:

Name	Type	Description
`tuple`	`Any`	`(confidence_field_name, confidence_field)`, or
	`Any`	`(None, None)` when modulation is off.

Source code in src/popoto/fields/decaying_sorted_field.py

def resolve_confidence_modulation_field(
    model_class: Any, field: Any, field_name: str
) -> tuple[Any, Any]:
    """Resolve which ``ConfidenceField`` modulates ``field``'s decay.

    Resolution order (first match wins):

    1. ``Defaults.DECAY_CONFIDENCE_MODULATION_ENABLED is False`` -> off.
       This is the deploy-level kill switch: it disables modulation without
       any model-code edit, for adopters who cannot edit model definitions.
    2. ``confidence_modulation_field=False`` -> off (per-field opt-out).
    3. ``confidence_modulation_field="name"`` -> that field, or ``ModelException``
       if it is missing or is not a ``ConfidenceField``.
    4. ``confidence_modulation_field=None`` (default) -> auto-detect over
       ``model_class._meta.fields``. Exactly one ``ConfidenceField`` is used;
       zero means off; two or more means off plus a warning naming the
       candidates. Guessing between two confidence signals would silently pick
       a ranking policy the adopter never chose.

    Returns:
        tuple: ``(confidence_field_name, confidence_field)``, or
        ``(None, None)`` when modulation is off.
    """
    from .confidence_field import ConfidenceField

    if not Defaults.DECAY_CONFIDENCE_MODULATION_ENABLED:
        return None, None

    spec = getattr(field, "confidence_modulation_field", None)
    if spec is False:
        return None, None

    cache = getattr(field, "_confidence_modulation_cache", None)
    if cache is not None and model_class in cache:
        return cache[model_class]

    fields = getattr(getattr(model_class, "_meta", None), "fields", {}) or {}

    if isinstance(spec, str):
        target = fields.get(spec)
        if target is None:
            raise ModelException(
                f"confidence_modulation_field='{spec}' on "
                f"{model_class.__name__}.{field_name} names a field that does "
                f"not exist on {model_class.__name__}. Available fields: "
                f"{', '.join(sorted(fields)) or '(none)'}"
            )
        if not isinstance(target, ConfidenceField):
            raise ModelException(
                f"confidence_modulation_field='{spec}' on "
                f"{model_class.__name__}.{field_name} must name a "
                f"ConfidenceField, but '{spec}' is a "
                f"{type(target).__name__}"
            )
        resolved: tuple[Any, Any] = (spec, target)
    else:
        candidates = [
            (name, f) for name, f in fields.items() if isinstance(f, ConfidenceField)
        ]
        if len(candidates) == 1:
            resolved = candidates[0]
        elif not candidates:
            resolved = (None, None)
        else:
            logger.warning(
                "%s.%s: confidence-modulated decay disabled -- %d ConfidenceFields "
                "found (%s). Pass confidence_modulation_field='<name>' to choose "
                "one, or False to silence this.",
                model_class.__name__,
                field_name,
                len(candidates),
                ", ".join(name for name, _ in candidates),
            )
            resolved = (None, None)

    if cache is not None:
        cache[model_class] = resolved
    return resolved

`confidence_modulation_args(model_class, field, field_name, *, filters=None, model_instance=None)` ¶

Build (confidence_hash_key, s, c0) for a decay EVAL.

c0 is the resolved field's own initial_confidence -- never a hard-coded 0.5. It is both the absent-value default and the centering constant, so an adopter running initial_confidence=0.3 still gets a bit-exactly neutral score for a record with no evidence.

Parameters:

Name	Type	Description	Default
`model_class`	`Any`	The Model class being queried.	required
`field`	`Any`	The DecayingSortedField / CyclicDecayField instance.	required
`field_name`	`str`	Name of that field.	required
`filters`	`Optional[dict[Any, Any]]`	Query filter mapping, used to satisfy the ConfidenceField's `partition_by`. Missing partition filters raise QueryException.	`None`
`model_instance`	`Any`	A saved instance to read partition values off of, for callers (the metacognitive proxy) that have records but no filters.	`None`

Returns:

Type	Description
`tuple[str, str, str]`	tuple[str, str, str]: `(key, s, c0)`; `MODULATION_DISABLED` when off.

Raises:

Type	Description
`QueryException`	The ConfidenceField is partitioned by fields the query does not filter on, so no single `:data` hash covers the result set. Silently disabling modulation here would make the ranking quietly wrong instead of loudly unsupported.

Source code in src/popoto/fields/decaying_sorted_field.py

def confidence_modulation_args(
    model_class: Any,
    field: Any,
    field_name: str,
    *,
    filters: Optional[dict[Any, Any]] = None,
    model_instance: Any = None,
) -> tuple[str, str, str]:
    """Build ``(confidence_hash_key, s, c0)`` for a decay ``EVAL``.

    ``c0`` is the resolved field's own ``initial_confidence`` -- never a
    hard-coded ``0.5``. It is both the absent-value default and the centering
    constant, so an adopter running ``initial_confidence=0.3`` still gets a
    bit-exactly neutral score for a record with no evidence.

    Args:
        model_class: The Model class being queried.
        field: The DecayingSortedField / CyclicDecayField instance.
        field_name: Name of that field.
        filters: Query filter mapping, used to satisfy the ConfidenceField's
            ``partition_by``. Missing partition filters raise QueryException.
        model_instance: A saved instance to read partition values off of, for
            callers (the metacognitive proxy) that have records but no filters.

    Returns:
        tuple[str, str, str]: ``(key, s, c0)``; ``MODULATION_DISABLED`` when off.

    Raises:
        QueryException: The ConfidenceField is partitioned by fields the query
            does not filter on, so no single ``:data`` hash covers the result
            set. Silently disabling modulation here would make the ranking
            quietly wrong instead of loudly unsupported.
    """
    # models.query.QueryException, NOT exceptions.QueryException: the two are
    # distinct classes, and this must match what ConfidenceField's own partition
    # guard (confidence_field.py:259) and top_by_decay already raise.
    from ..models.query import QueryException

    conf_name, conf_field = resolve_confidence_modulation_field(
        model_class, field, field_name
    )
    if conf_field is None:
        return MODULATION_DISABLED

    strength = Defaults.DECAY_CONFIDENCE_MODULATION_STRENGTH
    if not strength:
        return MODULATION_DISABLED

    if model_instance is not None:
        data_hash_key = conf_field.get_data_hash_key(model_instance, conf_name)
    else:
        filters = filters or {}
        missing = [pf for pf in conf_field.partition_by if pf not in filters]
        if missing:
            raise QueryException(
                f"Confidence-modulated decay on "
                f"{model_class.__name__}.{field_name} reads ConfidenceField "
                f"'{conf_name}', which is partitioned by "
                f"{', '.join(conf_field.partition_by)}. "
                f"Query must include filter(s) for: {', '.join(missing)}. "
                f"Alternatively set confidence_modulation_field=False on "
                f"'{field_name}' to disable modulation."
            )
        partition_values = {pf: filters[pf] for pf in conf_field.partition_by}
        data_hash_key = conf_field.get_data_hash_key_from_values(
            model_class, conf_name, **partition_values
        )

    return (data_hash_key, str(strength), str(conf_field.initial_confidence))

popoto.fields.decaying_sorted_field¶