spb/ultra-sharp-agent-skills Public

1---2name: modeling-nosql-data3description: Models data for document and key-value stores — access-pattern-first design, embed vs reference decisions, controlled denormalization, Redis key design, cursor pagination, and document schema versioning. Use when the user asks to design a MongoDB or DynamoDB schema, model documents or collections, structure Redis keys, decide between embedding and referencing, or migrate a relational model to NoSQL. Do not use for relational schema design or for SQL databases with JSON columns, which follow relational rules first.4---56<!--7Author: Simon-Pierre Boucher8Contact: contact@spboucher.ai9-->1011# Modeling NoSQL Data1213## When to use / when NOT to use14- **Use for:** designing document models (MongoDB, Firestore, DynamoDB) and key-value layouts (Redis, DynamoDB keys).15- **Do NOT use for:** relational schemas (`designing-database-schemas`) — and a Postgres table with a JSONB column is still relational: normalize first, JSONB for the truly schemaless remainder.1617## Core rules18191. **Queries first, schema second.** List every access pattern (screen/endpoint → fields → frequency) BEFORE shaping documents — the inverse of relational design. A model that can't be listed against its queries isn't done.202. **Embed vs reference decision rule:**21   - **Embed** when data is 1:few, read together, and updated together (order + its line items).22   - **Reference** when 1:many grows unbounded, is shared by many parents, or updates independently (user ← posts; product ← reviews).23   - ✅ `{order_id, items: [{sku, qty}]}` — ❌ embedding a user's entire post history in the user document.243. **Denormalize on purpose, with ONE source of truth per fact.** Copying `author_name` into posts is fine — but document where the canonical value lives and how copies get refreshed (on write, or accept staleness with a TTL).254. **Design far below size limits.** MongoDB caps documents at 16MB; treat ~1MB as the design ceiling. Unbounded arrays are the failure smell — bucket them (one document per day/100 items) or reference.265. **Redis/KV keys: colon-namespaced, predictable, with a TTL policy.**27   - ✅ `user:42:session`, `cache:product:9f3e` + `EXPIRE`28   - ❌ `data_42_final`, keys without owner/type, cache keys with no TTL.296. **Paginate with cursors, never offsets.** Sort by an indexed, unique (or tie-broken) field and continue from the last seen value: `find({created_at: {$lt: cursor}}).limit(20)`.307. **Version documents.** Every document carries `schema_version: 3`; readers handle N and N-1, writers upgrade on write (lazy migration).318. **Model transactions around aggregates.** Put data that must change atomically in ONE document; cross-document transactions exist but are the escape hatch, not the design.3233## Workflow34351. Write the access-pattern table: operation, fields needed, reads/sec vs writes/sec, consistency need.362. Group into aggregates (what changes together) → those become documents; everything else becomes references or copies.373. For each copy created by denormalization, record the source of truth and the refresh rule.384. Define keys/indexes per access pattern (compound indexes matching sort + filter; Redis key format + TTL per type).395. Validate: walk EVERY access pattern from step 1 against the model — each must resolve to one indexed query or one key lookup. Any pattern needing a scan or N+1 fetches → reshape and re-check.4041## Edge cases & failure modes42- **Many-to-many** → reference both ways only if both directions are queried; otherwise store the relation on the side that's queried.43- **Hot documents** (one doc absorbing all writes, e.g. a global counter) → shard the key (`counter:{0..15}`) and sum on read.44- **Search across fields** → document stores are poor at ad-hoc search; pair with a search index rather than contorting the model.45- **Strong consistency needed across entities** → that's the signal you may be in relational territory; say so instead of forcing it.46- **Relational→NoSQL migration** → do NOT port tables 1:1 into collections; restart from access patterns (rule 1).4748## References49Worked examples (blog, orders, sessions), bucket pattern, cursor pagination code, Redis key catalog: see [references/patterns.md](references/patterns.md).50