---
name: modeling-nosql-data
description: Models data for document and key-value stores — access-pattern-first design, embed vs reference decisions, controlled denormalization, Redis key design, cursor pagination, and document schema versioning. Use when the user asks to design a MongoDB or DynamoDB schema, model documents or collections, structure Redis keys, decide between embedding and referencing, or migrate a relational model to NoSQL. Do not use for relational schema design or for SQL databases with JSON columns, which follow relational rules first.
---

<!--
Author: Simon-Pierre Boucher
Contact: contact@spboucher.ai
-->

# Modeling NoSQL Data

## When to use / when NOT to use
- **Use for:** designing document models (MongoDB, Firestore, DynamoDB) and key-value layouts (Redis, DynamoDB keys).
- **Do NOT use for:** relational schemas (`designing-database-schemas`) — and a Postgres table with a JSONB column is still relational: normalize first, JSONB for the truly schemaless remainder.

## Core rules

1. **Queries first, schema second.** List every access pattern (screen/endpoint → fields → frequency) BEFORE shaping documents — the inverse of relational design. A model that can't be listed against its queries isn't done.
2. **Embed vs reference decision rule:**
   - **Embed** when data is 1:few, read together, and updated together (order + its line items).
   - **Reference** when 1:many grows unbounded, is shared by many parents, or updates independently (user ← posts; product ← reviews).
   - ✅ `{order_id, items: [{sku, qty}]}` — ❌ embedding a user's entire post history in the user document.
3. **Denormalize on purpose, with ONE source of truth per fact.** Copying `author_name` into posts is fine — but document where the canonical value lives and how copies get refreshed (on write, or accept staleness with a TTL).
4. **Design far below size limits.** MongoDB caps documents at 16MB; treat ~1MB as the design ceiling. Unbounded arrays are the failure smell — bucket them (one document per day/100 items) or reference.
5. **Redis/KV keys: colon-namespaced, predictable, with a TTL policy.**
   - ✅ `user:42:session`, `cache:product:9f3e` + `EXPIRE`
   - ❌ `data_42_final`, keys without owner/type, cache keys with no TTL.
6. **Paginate with cursors, never offsets.** Sort by an indexed, unique (or tie-broken) field and continue from the last seen value: `find({created_at: {$lt: cursor}}).limit(20)`.
7. **Version documents.** Every document carries `schema_version: 3`; readers handle N and N-1, writers upgrade on write (lazy migration).
8. **Model transactions around aggregates.** Put data that must change atomically in ONE document; cross-document transactions exist but are the escape hatch, not the design.

## Workflow

1. Write the access-pattern table: operation, fields needed, reads/sec vs writes/sec, consistency need.
2. Group into aggregates (what changes together) → those become documents; everything else becomes references or copies.
3. For each copy created by denormalization, record the source of truth and the refresh rule.
4. Define keys/indexes per access pattern (compound indexes matching sort + filter; Redis key format + TTL per type).
5. Validate: walk EVERY access pattern from step 1 against the model — each must resolve to one indexed query or one key lookup. Any pattern needing a scan or N+1 fetches → reshape and re-check.

## Edge cases & failure modes
- **Many-to-many** → reference both ways only if both directions are queried; otherwise store the relation on the side that's queried.
- **Hot documents** (one doc absorbing all writes, e.g. a global counter) → shard the key (`counter:{0..15}`) and sum on read.
- **Search across fields** → document stores are poor at ad-hoc search; pair with a search index rather than contorting the model.
- **Strong consistency needed across entities** → that's the signal you may be in relational territory; say so instead of forcing it.
- **Relational→NoSQL migration** → do NOT port tables 1:1 into collections; restart from access patterns (rule 1).

## References
Worked examples (blog, orders, sessions), bucket pattern, cursor pagination code, Redis key catalog: see [references/patterns.md](references/patterns.md).