Skip to content

popoto.backends.postgres.codec

popoto.backends.postgres.codec

jsonb encoding for the collection fields on Postgres (#759 M1.1).

ListField, DictField, SetField and TupleField are stored as jsonb -- never msgpack (plan ยง8). The contract is that a value comes back exactly as it comes back from Redis, so the encoding reproduces what msgpack plus popoto's tagged-dict registry (TYPE_ENCODER_DECODERS) round-trip, and nothing more:

  • The top level is typed by the column: a list/tuple/set field is a JSON array, re-typed on the way out by the field's own type (on Redis the registry tags a top-level tuple or set for the same reason).
  • Nested values behave as msgpack does: a nested tuple comes back a list; a tagged dict ({"__Decimal__": True, "as_encodable": ...}, written by CappedListProxy.push and capped-list saves per element) decodes to its type at any depth, as decode_custom_types does as msgpack's object hook; and a value msgpack cannot pack (a nested set, Decimal, datetime...) raises TypeError here too.
  • What JSON lacks and msgpack has is tagged so it survives: bytes (base64), a non-finite float and a dict with a non-str key. These three tags are this module's own and never reach Redis.

Pure: no network, no psycopg.

encode_json_element(value)

One element of a capped ListField: tagged through popoto's type registry first, exactly as _encode_list_element does for Redis's per-element RPUSH/LPUSH.

Source code in src/popoto/backends/postgres/codec.py 
66
67
68
69
70
71
72
73
74
def encode_json_element(value: Any) -> Any:
    """One element of a capped ``ListField``: tagged through popoto's type
    registry first, exactly as ``_encode_list_element`` does for Redis's
    per-element ``RPUSH``/``LPUSH``."""
    from ...models.encoding import TYPE_ENCODER_DECODERS

    if value is not None and type(value) in TYPE_ENCODER_DECODERS:
        value = TYPE_ENCODER_DECODERS[type(value)].encoder(value)
    return _encode_nested(value)

encode_json(py_type, value, *, capped=False)

The JSON document stored for a collection field's value.

Source code in src/popoto/backends/postgres/codec.py 
77
78
79
80
81
82
83
84
85
86
def encode_json(py_type: type, value: Any, *, capped: bool = False) -> Any:
    """The JSON document stored for a collection field's ``value``."""
    if value is None:
        return None
    data = getattr(value, "_data", value)  # a CappedListProxy holds a list
    if capped:
        return [encode_json_element(v) for v in data]
    if py_type in (list, tuple, set) and isinstance(data, (list, tuple, set)):
        return [_encode_nested(v) for v in data]
    return _encode_nested(data)

decode_json(py_type, value)

A stored JSON document back as the field's Python value.

Source code in src/popoto/backends/postgres/codec.py 
115
116
117
118
119
120
121
122
123
124
def decode_json(py_type: type, value: Any) -> Any:
    """A stored JSON document back as the field's Python value."""
    if value is None:
        return None
    decoded = _decode_nested(value)
    if py_type is tuple and isinstance(decoded, list):
        return tuple(decoded)
    if py_type is set and isinstance(decoded, list):
        return set(decoded)
    return decoded