popoto.fields.shortcuts¶
popoto.fields.shortcuts
¶
Shortcut Field Classes for Popoto Redis ORM.
This module provides convenient, type-safe field classes that simplify model
definitions by pre-configuring the base Field class with common Python types.
Rather than writing Field(type=int), users can write IntField().
Design Philosophy
Popoto follows Django's ORM conventions to provide a familiar API for Python developers. These shortcut fields serve three purposes:
- Readability:
price = FloatField()clearly communicates intent - Type Safety: Prevents accidental type mismatches at field definition
- IDE Support: Better autocomplete and type hints than generic Field
Architecture
The shortcuts are organized into three categories:
-
Type Fields (IntField, StringField, etc.): Simple type wrappers that pre-set the
typeparameter. These are thin convenience layers. -
Key Fields (KeyField, UniqueKeyField, AutoKeyField): Control how model instances are identified and indexed in Redis. These determine the Redis key structure and enable efficient lookups.
-
Sorted Fields (SortedField, SortedKeyField): Enable range queries by maintaining Redis Sorted Sets alongside model data.
Redis Storage Considerations
All Python types are serialized to bytes via msgpack before storage. The type parameter primarily affects validation and deserialization, not storage format. Complex types (list, dict, set) serialize correctly but cannot be used as KeyFields since Redis keys must be simple strings.
Example
class Product(Model): sku = UniqueKeyField() # Primary identifier name = StringField() # Basic string storage price = FloatField() # Numeric, could use SortedField for range queries tags = ListField(default=[]) # Complex type storage created = DateField() # Temporal data
See Also
- field.py: Base Field class implementation
- key_field_mixin.py: KeyField indexing behavior
- sorted_field_mixin.py: Range query support via Sorted Sets
IntField
¶
Bases: Field
A Field that stores int values.
Use IntField when you need whole number storage with automatic type validation. For range queries (e.g., "find products with quantity > 10"), consider SortedField(type=int) instead.
Example
class Inventory(Model): product_id = UniqueKeyField() quantity = IntField(default=0) reorder_threshold = IntField(null=True)
Source code in src/popoto/fields/shortcuts.py
FloatField
¶
Bases: Field
A Field that stores float values.
Use FloatField for decimal values where exact precision is not critical. For financial data or when precision matters, prefer DecimalField. For range queries, consider SortedField(type=float).
Example
class Sensor(Model): sensor_id = UniqueKeyField() temperature = FloatField() humidity = FloatField(null=True)
Source code in src/popoto/fields/shortcuts.py
DecimalField
¶
Bases: Field
A Field that stores Decimal values.
Use DecimalField for financial data, currency, or any values where floating-point precision errors are unacceptable. The Decimal type preserves exact decimal representation through serialization.
Note: DecimalField can be used with SortedField for range queries, though the underlying Redis score uses float conversion.
Example
class Transaction(Model): tx_id = UniqueKeyField() amount = DecimalField(null=False) tax_rate = DecimalField(default=Decimal("0.0825"))
Source code in src/popoto/fields/shortcuts.py
StringField
¶
Bases: Field
A Field that stores str values (same as the base Field default).
StringField is functionally equivalent to Field() since str is the default type. It exists for explicitness and consistency with other typed fields.
For strings that identify model instances, use KeyField or UniqueKeyField instead, which enable efficient lookups and query filtering.
Example
class Article(Model): slug = UniqueKeyField() # Identifier - use KeyField title = StringField(max_length=200) content = StringField() # No practical limit
Source code in src/popoto/fields/shortcuts.py
BooleanField
¶
Bases: Field
A Field that stores bool values.
BooleanField stores Python booleans with full type validation. Note that by default null=True, so the field can hold three states: True, False, or None. Set null=False for strict two-state booleans.
Example
class Feature(Model): name = UniqueKeyField() enabled = BooleanField(default=False, null=False) deprecated = BooleanField(default=False)
Source code in src/popoto/fields/shortcuts.py
BytesField
¶
Bases: Field
A Field that stores bytes values.
Use BytesField for binary content like images, encoded data, or any content that should not be interpreted as text. The bytes are stored as-is via msgpack serialization.
Example
class BinaryCache(Model): key = UniqueKeyField() data = BytesField() checksum = BytesField(null=True)
Source code in src/popoto/fields/shortcuts.py
CappedListProxy
¶
A list-like wrapper for capped ListField values.
Provides a push() method for direct Redis LPUSH + LTRIM operations without requiring a full read/write cycle. Behaves like a regular list for all other operations.
The proxy holds a reference to the model instance, field name, and max_length to compute the correct Redis list key and enforce the cap.
Attributes:
| Name | Type | Description |
|---|---|---|
_data |
The underlying Python list. |
|
_model_instance |
The Model instance this proxy belongs to. |
|
_field_name |
The name of the ListField on the model. |
|
_max_length |
Maximum number of items to keep in the list. |
Source code in src/popoto/fields/shortcuts.py
231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 | |
push(value)
¶
Prepend a value to the capped list using LPUSH + LTRIM.
This method writes directly to Redis without requiring a full model save. The value is serialized with msgpack individually.
The Redis write and local state update are kept in sync: if the Redis pipeline succeeds but the local update fails, the local state is rolled back to match what Redis held before the push, and the exception is re-raised so the caller knows something went wrong.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
The value to prepend. Can be any msgpack-serializable type. |
required |
Raises:
| Type | Description |
|---|---|
ModelException
|
If the model instance has not been saved yet (no _redis_key available). |
Source code in src/popoto/fields/shortcuts.py
ListField
¶
Bases: Field
A Field that stores list values.
ListField stores ordered collections that serialize via msgpack. The list contents can be any msgpack-serializable types.
When max_length is set, the list is stored in a separate Redis list
key ({model_db_key}::field_name) instead of in the model hash. This
enables efficient push() operations using LPUSH + LTRIM without
reading the full list.
When max_length is not set, the list is stored atomically in the
model's Redis hash as before (backward compatible).
Note: ListField cannot be used as a KeyField since lists cannot be converted to Redis key strings.
Example
class ShoppingCart(Model): user_id = UniqueKeyField() items = ListField(default=[]) recently_viewed = ListField(null=True)
class EventLog(Model): session_id = UniqueKeyField() events = ListField(max_length=100) # Capped at 100 items
Source code in src/popoto/fields/shortcuts.py
388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 | |
is_valid(field, value, null_check=True, **kwargs)
classmethod
¶
Validate list field values, accepting CappedListProxy as valid.
Source code in src/popoto/fields/shortcuts.py
format_value_pre_save(field_value, **kwargs)
¶
For capped lists, return the value unchanged.
The actual data is written to a separate Redis list key in on_save(). The value is excluded from the hash in encode_popoto_model_obj().
Source code in src/popoto/fields/shortcuts.py
on_save(model_instance, field_name, field_value, pipeline=None, **kwargs)
classmethod
¶
Write capped list data to a separate Redis list key.
For capped lists (max_length is set), this replaces the list at the Redis list key using DEL + RPUSH in a pipeline. For uncapped lists, delegates to the base Field.on_save() (no-op).
Source code in src/popoto/fields/shortcuts.py
on_delete(model_instance, field_name, field_value, pipeline=None, **kwargs)
classmethod
¶
Delete the Redis list key when the model is deleted.
For capped lists, removes the separate Redis list key. For uncapped lists, delegates to the base Field.on_delete() (no-op).
Source code in src/popoto/fields/shortcuts.py
DictField
¶
Bases: Field
A Field that stores dict values.
DictField stores key-value mappings via msgpack serialization. Useful for flexible schema-less data, metadata, or configuration. The entire dict is stored atomically with the model.
Note: DictField cannot be used as a KeyField. For Redis Hash operations, consider using the Redis client directly or separate fields.
Example
class UserPreferences(Model): user_id = UniqueKeyField() settings = DictField(default={}) metadata = DictField(null=True)
Source code in src/popoto/fields/shortcuts.py
SetField
¶
Bases: Field
A Field that stores set values.
SetField stores unordered collections of unique values. The set is serialized via msgpack and stored atomically with the model instance. This is distinct from Redis Sets - the data is embedded in the model.
Note: SetField cannot be used as a KeyField. For Redis Set operations (unions, intersections), use the Redis client directly.
Example
class Article(Model): slug = UniqueKeyField() tags = SetField(default=set()) categories = SetField(null=True)
Source code in src/popoto/fields/shortcuts.py
TupleField
¶
Bases: Field
A Field that stores tuple values.
TupleField stores immutable ordered sequences. The tuple is serialized via msgpack and stored atomically. Useful for fixed-structure data like coordinates (x, y) or version numbers (major, minor, patch).
Note: TupleField cannot be used as a KeyField. For geographic coordinates, consider GeoField which provides spatial queries.
Example
class Release(Model): name = UniqueKeyField() version = TupleField() # (major, minor, patch) coordinates = TupleField(null=True) # (lat, lng)
Source code in src/popoto/fields/shortcuts.py
DateField
¶
Bases: Field
A Field that stores datetime.date values.
DateField stores Python date objects. Dates are serialized and can be filtered in queries. For date range queries like "find all events after 2024-01-01", use SortedField(type=date) instead.
For datetime values with time components, use DatetimeField (defined in datetime_field.py with timezone handling).
Example
class Event(Model): event_id = UniqueKeyField() event_date = DateField(null=False) registration_deadline = DateField(null=True)
Source code in src/popoto/fields/shortcuts.py
TimeField
¶
Bases: Field
A Field that stores datetime.time values.
TimeField stores Python time objects representing time of day. Useful for schedules, recurring events, or time-only data. For full datetime values, use DatetimeField instead.
Example
class Store(Model): store_id = UniqueKeyField() opening_time = TimeField(null=False) closing_time = TimeField(null=False)
Source code in src/popoto/fields/shortcuts.py
KeyField
¶
Bases: KeyFieldMixin, Field
A field that forms part of the model's Redis key and enables query filtering.
KeyField is the foundation of Popoto's identity and query system. When you define KeyFields on a model, their values become part of the Redis key used to store that instance. This enables:
- Direct lookups: Model.query.get(field=value) uses the key directly
- Pattern matching: Queries can use startswith, endswith, contains
- Set-based filtering: Each unique value maintains a Redis Set of instance keys
All KeyField values together enforce a unique-together constraint.
Supports filter lookups: exact, __isnull, __contains,
__startswith, __endswith, __in.
Design Decision
KeyField allows null values and duplicate single-field values by default. Use UniqueKeyField for strict uniqueness, or AutoKeyField for auto-generated IDs.
Example
class BandMember(Model): band = KeyField() # Part of composite key role = KeyField() # Part of composite key name = StringField()
Redis key: "BandMember:BLACKPINK:vocalist"¶
lisa = BandMember(band="BLACKPINK", role="vocalist", name="Lisa")
Query filtering uses the KeyField's Redis Set index¶
BandMember.query.filter(band="BLACKPINK") # Returns all BLACKPINK members BandMember.query.filter(band__startswith="BLACK") # Pattern matching
See Also
- UniqueKeyField: Enforces single-field uniqueness
- AutoKeyField: Auto-generates unique identifiers
- KeyFieldMixin: Implementation of indexing behavior
Source code in src/popoto/fields/shortcuts.py
UniqueKeyField
¶
Bases: KeyField
A KeyField with a per-value uniqueness constraint. Cannot be null.
UniqueKeyField guarantees that no two model instances can have the same value for this field. Unlike KeyField (which only enforces unique_together across all key fields), UniqueKeyField enforces uniqueness on this single field alone.
Constraints
- Cannot be null (unique=True + null=True is logically inconsistent)
- Cannot set unique=False (use KeyField instead)
Use Cases
- Primary identifiers: usernames, SKUs, email addresses
- Slugs or permalinks
- Any field that must be globally unique
Example
class User(Model): username = UniqueKeyField() # Unique across all users email = UniqueKeyField() # Also unique across all users display_name = StringField()
Direct lookup by unique field¶
user = User.query.get(username="johndoe")
See Also
- KeyField: For non-unique key fields
- AutoKeyField: For auto-generated unique identifiers
Source code in src/popoto/fields/shortcuts.py
AutoKeyField
¶
Bases: AutoFieldMixin, UniqueKeyField
A UniqueKeyField whose value is auto-generated (UUID-based).
AutoKeyField automatically generates a unique identifier (UUID hex) when a model instance is created. This is Popoto's equivalent of Django's AutoField or an auto-incrementing primary key.
Automatically added to models that define no KeyField of their own.
Key Behavior
- Generates a 32-character UUID hex string by default
- Value is set at instance creation, before first save
- Cannot be null, cannot be non-unique
- If no KeyFields are defined on a model, Popoto automatically adds an AutoKeyField named "_auto_key"
Design Decision
Uses UUID4 (random) rather than auto-increment for two reasons: 1. Distributed safety: No coordination needed across Redis instances 2. Privacy: IDs don't reveal creation order or count
Example
class LogEntry(Model): # Explicit AutoKeyField entry_id = AutoKeyField() message = StringField()
class SimpleModel(Model): # No KeyFields defined - Popoto adds "_auto_key" automatically data = StringField()
entry = LogEntry(message="Hello") entry.save() print(entry.entry_id) # e.g., "a1b2c3d4e5f6..."
See Also
- AutoFieldMixin: UUID generation implementation
- UniqueKeyField: Base class providing uniqueness
Source code in src/popoto/fields/shortcuts.py
SortedField
¶
Bases: SortedFieldMixin, Field
A field backed by a Redis sorted set for fast range queries.
SortedField maintains a parallel Redis Sorted Set index, enabling O(log N) range queries on numeric and temporal values. Without SortedField, range filtering would require scanning all instances.
Supports filter lookups: __gt, __gte, __lt, __lte.
Must be a numeric type (int, float, Decimal, date, or datetime).
Supported Types
- int, float, Decimal (used directly as Redis scores)
- date, datetime, time (converted to numeric via ordinal/timestamp)
Constraints
- Cannot be null (Sorted Sets require a score for every member)
- Must be a numeric or temporal type
Partitioning (partition_by): For large datasets, you can partition the Sorted Set by another field's value using partition_by. This improves performance by reducing set size, but requires the partition field in all queries.
Example
class Product(Model): sku = UniqueKeyField() price = SortedField(type=float)
Range query - O(log N) using ZRANGEBYSCORE¶
cheap_products = Product.query.filter(price__lte=10.0) mid_range = Product.query.filter(price__gte=10.0, price__lte=50.0)
With partitioning for better scalability¶
class Product(Model): sku = UniqueKeyField() category = KeyField() price = SortedField(type=float, partition_by='category')
Must include partition field¶
Product.query.filter(category='electronics', price__lte=100.0)
See Also
- SortedFieldMixin: Range query implementation details
- SortedKeyField: Combines SortedField with KeyField behavior
Source code in src/popoto/fields/shortcuts.py
SortedKeyField
¶
Bases: SortedFieldMixin, KeyFieldMixin, Field
A field that is both a KeyField and a SortedField.
SortedKeyField participates in the model's Redis key (like KeyField) while also maintaining a Sorted Set index for range queries (like SortedField). This enables both direct lookups and efficient range filtering on the same field.
Use Case
When you need to both identify instances by a numeric value AND perform range queries on it. For example, a timestamp-based ID where you need both direct access and "find all after time X" queries.
Inheritance Order
The MRO is [SortedFieldMixin, KeyFieldMixin, Field]. Both mixin behaviors are active: KeyField's Set-based lookups and SortedField's range queries.
Example
class TimestampedEvent(Model): # Timestamp serves as both key and enables range queries timestamp = SortedKeyField(type=float) event_type = KeyField() data = DictField()
Direct lookup via key¶
event = TimestampedEvent.query.get(timestamp=1609459200.0, event_type="login")
Range query via sorted index¶
recent = TimestampedEvent.query.filter(timestamp__gte=1609459200.0)
See Also
- KeyField: For key-only fields without range queries
- SortedField: For range queries without key participation
Source code in src/popoto/fields/shortcuts.py
IndexedField
¶
Bases: IndexedFieldMixin, Field
A non-key field with Set-based secondary indexing for exact-match queries.
IndexedField maintains a Redis Set index for the field value, enabling
efficient filter() queries without making the field part of the
model's Redis key (identity).
This decouples querying from identity: you can filter on status,
email, or category without those fields affecting the Redis
storage key.
Supports filter lookups: exact, __in, __isnull,
__startswith, __endswith.
Example
class User(Model): user_id = AutoKeyField() status = IndexedField(type=str) role = IndexedField(type=str, null=True)
User.query.filter(status="active") User.query.filter(role__in=["admin", "moderator"]) User.query.filter(status__startswith="act")
See Also
- UniqueField: IndexedField with uniqueness enforcement
- KeyField: For fields that should be part of the Redis key
Source code in src/popoto/fields/shortcuts.py
UniqueField
¶
Bases: IndexedFieldMixin, Field
An indexed non-key field with a per-value uniqueness constraint.
UniqueField combines secondary indexing with uniqueness enforcement. It guarantees that no two model instances can have the same value for this field, while keeping it separate from the model's Redis key.
Cannot be null (unique + null is logically inconsistent).
Example
class User(Model): user_id = AutoKeyField() email = UniqueField(type=str)
user1 = User.create(email="alice@example.com") user2 = User.create(email="alice@example.com") # Raises ModelException
See Also
- IndexedField: For indexed fields without uniqueness
- UniqueKeyField: For unique fields that are part of the Redis key