<!--
Author: Simon-Pierre Boucher
Contact: contact@spboucher.ai
-->

# NoSQL Data Modeling — Patterns

## Contents
- Access-pattern table template
- Embed vs reference — worked examples
- Bucket pattern for unbounded arrays
- Cursor pagination
- Redis key catalog
- Schema versioning / lazy migration
- Gotchas

## Access-pattern table template

| # | Operation | Fields needed | Freq | Consistency |
|---|---|---|---|---|
| 1 | Load order page | order, items, shipping status | 500 r/s | read-your-writes |
| 2 | Add item to cart | cart items | 50 w/s | strong (same doc) |
| 3 | Seller sales list | order ids, totals by seller | 5 r/s | eventual ok |

Every pattern must map to one indexed query or key lookup in the final model.

## Embed vs reference — worked examples

```js
// EMBED: order + line items (1:few, read & updated together, bounded)
{
  _id: "ord_81",
  user_id: "u_42",             // reference — user updates independently
  user_name: "Alice",          // denormalized copy; truth: users collection,
                               // refreshed: never (historical snapshot is correct here)
  status: "shipped",
  items: [ { sku: "A-1", qty: 2, unit_price_cents: 1250 } ],
  schema_version: 2
}

// REFERENCE: user ← posts (unbounded growth)
{ _id: "u_42", name: "Alice", schema_version: 1 }
{ _id: "p_777", author_id: "u_42", author_name: "Alice",   // copy, refreshed on user rename via async job
  title: "...", created_at: ISODate("2026-08-01") }
// Index to serve "posts by user, newest first":
db.posts.createIndex({ author_id: 1, created_at: -1 })
```

## Bucket pattern for unbounded arrays

```js
// Instead of one sensor doc with an ever-growing readings[] array:
{
  _id: "sensor_9:2026-08-05",       // one bucket per sensor per day
  sensor_id: "sensor_9",
  day: "2026-08-05",
  count: 1440,
  readings: [ { t: "00:00", v: 20.1 }, /* ≤ 1440 */ ]
}
// Bounded documents, efficient range reads, no 16MB ceiling risk.
```

## Cursor pagination

```js
// Page 1:
db.posts.find({ author_id: uid })
        .sort({ created_at: -1, _id: -1 })      // _id breaks timestamp ties
        .limit(20)
// Next page — cursor = (created_at, _id) of last item:
db.posts.find({
  author_id: uid,
  $or: [
    { created_at: { $lt: cursor.created_at } },
    { created_at: cursor.created_at, _id: { $lt: cursor._id } }
  ]
}).sort({ created_at: -1, _id: -1 }).limit(20)
// Offsets (skip) degrade linearly and break when rows shift — never for feeds.
```

## Redis key catalog

Document every key family in one table; every cache key has a TTL.

| Key pattern | Type | TTL | Notes |
|---|---|---|---|
| `user:{id}:session` | hash | 30d sliding | auth token data |
| `cache:product:{id}` | string(json) | 300s | invalidate on product write |
| `rate:{ip}:{minute}` | int (INCR) | 120s | rate limiting window |
| `queue:emails` | list | none | worker queue (durable store elsewhere) |
| `counter:orders:{0..15}` | int | none | sharded hot counter; SUM on read |

```
SET cache:product:9f3e '{"name":...}' EX 300
INCR rate:203.0.113.7:202608051211
EXPIRE rate:203.0.113.7:202608051211 120 NX
```

## Schema versioning / lazy migration

```js
// Reader handles current and previous version:
function readUser(doc) {
  if (doc.schema_version === 1) {
    doc.full_name = doc.name;          // v1 → v2 shape
    doc.schema_version = 2;
  }
  return doc;
}
// Writer persists upgraded shape on next write ("lazy migration").
// Backfill job optional once v1 read-rate ≈ 0.
```

## Gotchas

- MongoDB's 16MB limit includes field names — long repeated keys in big arrays
  waste real space; short names matter at scale.
- An index on `{a: 1, b: 1}` serves filters on `a` and `a+b`, NOT `b` alone
  (prefix rule) — order compound indexes by equality → sort → range.
- `$lookup` (joins) run on unsharded/local data paths and get slow fast — a
  frequent `$lookup` is a modeling smell: embed or copy instead.
- DynamoDB: model EVERYTHING around partition key + sort key up front; there is
  no ad-hoc query escape hatch, only GSIs (each with its own cost).
- Redis `KEYS pattern` blocks the server — always `SCAN` in production.
- Firestore charges and limits per document read — bucket smallness matters
  differently: many tiny docs can cost more than fewer medium ones.
- Eventual consistency of denormalized copies must be a product decision
  ("name may be stale ≤5 min"), never an accident.