# name: modeling-nosql-data description: Models data for document and key-value stores — access-pattern-first design, embed vs reference decisions, controlled denormalization, Redis key design, cursor pagination, and document schema versioning. Use when the user asks to design a MongoDB or DynamoDB schema, model documents or collections, structure Redis keys, decide between embedding and referencing, or migrate a relational model to NoSQL. Do not use for relational schema design or for SQL databases with JSON columns, which follow relational rules first.

# Modeling NoSQL Data

# When to use / when NOT to use

• Use for: designing document models (MongoDB, Firestore, DynamoDB) and key-value layouts (Redis, DynamoDB keys).
• Do NOT use for: relational schemas (designing-database-schemas) — and a Postgres table with a JSONB column is still relational: normalize first, JSONB for the truly schemaless remainder.

# Core rules

1. Queries first, schema second. List every access pattern (screen/endpoint → fields → frequency) BEFORE shaping documents — the inverse of relational design. A model that can't be listed against its queries isn't done.
2. Embed vs reference decision rule:
   • Embed when data is 1:few, read together, and updated together (order + its line items).
   • Reference when 1:many grows unbounded, is shared by many parents, or updates independently (user ← posts; product ← reviews).
   • ✅ {order_id, items: [{sku, qty}]} — ❌ embedding a user's entire post history in the user document.
3. Denormalize on purpose, with ONE source of truth per fact. Copying author_name into posts is fine — but document where the canonical value lives and how copies get refreshed (on write, or accept staleness with a TTL).
4. Design far below size limits. MongoDB caps documents at 16MB; treat ~1MB as the design ceiling. Unbounded arrays are the failure smell — bucket them (one document per day/100 items) or reference.
5. Redis/KV keys: colon-namespaced, predictable, with a TTL policy.
   • ✅ user:42:session, cache:product:9f3e + EXPIRE
   • ❌ data_42_final, keys without owner/type, cache keys with no TTL.
6. Paginate with cursors, never offsets. Sort by an indexed, unique (or tie-broken) field and continue from the last seen value: find({created_at: {$lt: cursor}}).limit(20).
7. Version documents. Every document carries schema_version: 3; readers handle N and N-1, writers upgrade on write (lazy migration).
8. Model transactions around aggregates. Put data that must change atomically in ONE document; cross-document transactions exist but are the escape hatch, not the design.

# Workflow

1. Write the access-pattern table: operation, fields needed, reads/sec vs writes/sec, consistency need.
2. Group into aggregates (what changes together) → those become documents; everything else becomes references or copies.
3. For each copy created by denormalization, record the source of truth and the refresh rule.
4. Define keys/indexes per access pattern (compound indexes matching sort + filter; Redis key format + TTL per type).
5. Validate: walk EVERY access pattern from step 1 against the model — each must resolve to one indexed query or one key lookup. Any pattern needing a scan or N+1 fetches → reshape and re-check.

# Edge cases & failure modes

• Many-to-many → reference both ways only if both directions are queried; otherwise store the relation on the side that's queried.
• Hot documents (one doc absorbing all writes, e.g. a global counter) → shard the key (counter:{0..15}) and sum on read.
• Search across fields → document stores are poor at ad-hoc search; pair with a search index rather than contorting the model.
• Strong consistency needed across entities → that's the signal you may be in relational territory; say so instead of forcing it.
• Relational→NoSQL migration → do NOT port tables 1:1 into collections; restart from access patterns (rule 1).

# References

Worked examples (blog, orders, sessions), bucket pattern, cursor pagination code, Redis key catalog: see references/patterns.md.