name: shipping-with-ci-cd description: Designs CI/CD pipelines that ship safely — fail-fast stages, build-once artifact promotion, rolling/blue-green/canary deploy strategies, automated rollback, and backward-compatible migrations. Use when the user asks to set up or fix a CI/CD pipeline, GitHub Actions or GitLab CI workflow, deployment process, release automation, rollback strategy, or asks how to deploy to staging and production. Do not use for writing the tests themselves (testing-backend-services) or authoring Dockerfiles (containerizing-services).
Shipping with CI/CD
When to use / when NOT to use
- Use for: pipeline design/review, deploy strategy choice, environment promotion, release/rollback automation.
- Do NOT use for: test authoring (testing-backend-services), Dockerfile content (containerizing-services), or infra provisioning.
Core rules
- Main is always deployable. Trunk-based development by default: short-lived branches, merge behind a green pipeline. A red main is the team's top priority.
- Fail fast, in cost order. Stage sequence: lint/typecheck → unit tests → build → integration tests → security scan → deploy. Cheap checks first so failures cost seconds, not minutes.
- Build once, promote the artifact. The exact image/binary tested in staging is byte-identical in production — tag by commit SHA.
- ✅
svc:3f2a91cpromoted staging → prod - ❌ Rebuilding "the same" code per environment (different deps, different artifact).
- ✅
- Pick the deploy strategy by risk, not fashion. Rolling is the default; blue-green when you need instant rollback; canary (1–5% traffic, then ramp) for high-risk changes. Anything beyond rolling must justify its infra cost.
- Rollback is automated, not heroic. Health checks/SLO probes gate each deploy step; regression triggers automatic rollback to the previous artifact. If rollback requires a human running commands from memory, it isn't a rollback plan.
- Migrations deploy before code and stay one version backward-compatible. Old code must run against the new schema (expand → deploy → contract). Never couple a destructive migration to the deploy that needs it.
- Secrets come from the platform (environment/secret manager, OIDC cloud auth) — never committed in pipeline YAML, never echoed in logs.
- Keep the pipeline under ~10 minutes commit-to-verdict. Parallelize test shards, cache dependencies; a slow pipeline silently kills trunk-based flow.
Workflow
- Map stages in cost order (rule 2); wire caching for dependency steps.
- Emit one artifact tagged with the commit SHA; push to the registry once.
- Define environments (staging auto-deploys on main; production gated by promotion of the same artifact).
- Add deploy gates: health-check verification after each batch, automatic rollback on failure (rule 5).
- Wire migrations as a separate pre-deploy step with a rehearsed rollback (rule 6).
- Validate: run a deliberately failing commit (lint error) and confirm the pipeline stops at stage 1; deploy a canary/rolling change and kill an instance mid-deploy — confirm zero failed requests and automatic recovery.
Edge cases & failure modes
- Flaky tests → quarantine tagged-flaky tests to a non-blocking stage the same day; a pipeline people retry until green is a pipeline nobody trusts.
- Hotfix while main is red → fix forward on main; the deployable-main rule makes dedicated hotfix branches unnecessary.
- Long-running migration locks a hot table → split into batched backfills; see db-skills/managing-database-migrations.
- Rollback needed but migration was destructive → this is a rule-6 violation; restore path is a backup drill, so keep contract phases a release behind.
References
Pipeline templates and gotchas: see references/patterns.md.