Service Deployment and On-Call Runbooks
Create service-specific deployment, incident, maintenance, and rollback runbooks with observable checks and explicit stop conditions.
Before & after
Same prompt, same agent — the only difference is the skill.
Here's an on-call runbook structure for a payments service: **Service Overview**: What the service does and its dependencies **Deployment**: How to deploy and roll back **Common Issues**: Known problems and their fixes **Monitoring**: Dashboards and alerts to watch **Escalation**: Who to contact and when **Incident Response**: Steps to take during an outage For a payments service specifically, you'd want to pay attention to: - Transaction failures and reconciliation - Third-party payment provider outages - Idempotency and duplicate charges Would you like me to fill in any of these sections?
RUNBOOK — Payments Service · env: production
Templated structure, payments-specific where it counts.
SERVICE
Owns: charge creation, refunds, provider webhooks
Critical deps: Stripe (provider), Postgres (ledger), Redis (idempotency)
Blast radius if down: no new charges. Existing subscriptions unaffected
until renewal. This framing tells on-call how hard to page.
DEPLOY
Command, health-check URL, expected green signal.
ROLLBACK — the section on-call actually opens at 3am, so it's first:
1. Payments is ledger-critical. NEVER roll back if a migration
changed the ledger schema — roll FORWARD with a fix. Rolling
back a schema on a payments DB is how you double-charge.
2. For code-only changes: <exact rollback command>
3. After rollback: verify idempotency keys still resolve (Redis)
INCIDENT RESPONSE — by symptom, not generic steps
Charges failing broadly
→ Check Stripe status page FIRST. Provider outage is the most
common cause and nothing you do fixes it — comms, don't debug.
Duplicate charges reported
→ Idempotency layer. Check Redis health and key TTLs. This is the
one that generates refunds and angry customers — highest urgency.
Webhook backlog
→ Provider events queuing. Check consumer lag; replay is safe
BECAUSE the ledger is idempotent — confirm that before replaying.
MAINTENANCE
Reconciliation job schedule, what a healthy run looks like, what a
mismatch means (investigate, never auto-correct a ledger).
MONITORING
Dashboards: charge success rate, provider latency, idempotency hit
rate, ledger reconciliation delta. Alert thresholds per metric.
ESCALATION
L1 on-call → L2 payments owner (15 min no-resolve) → finance lead if
money is miscounted, not just if the service is down.
THE PAYMENTS-SPECIFIC RULE ON EVERY PAGE
When unsure, favor NOT charging over maybe-double-charging. A failed
charge retries clean; a double charge is a refund, a support ticket, and
a trust hit.
Want the deploy/rollback commands filled in for your actual pipeline?About this skill
“Check the logs” is not enough when a production service is failing. Turn service details into runbooks that tell an on-call engineer what to inspect, what result to expect, when to act, and when to stop or escalate.
What it does
- Documents deployments with target verification, release checks, observation windows, and explicit abort conditions.
- Builds incident procedures around observable symptoms and diagnostic branches instead of unsupported guesses about root cause.
- Separates application rollback from database recovery, including compatibility checks and irreversible migrations.
- Defines maintenance procedures with entry conditions, bounded execution, recovery paths, and completion checks.
- Reviews existing runbooks for dangerous commands, missing dependencies, vague success criteria, and stale assumptions.
- Walks through realistic incidents to expose missing access, circular dependencies, and steps that fail when the service is unavailable.
What you get
A service operations packet covering deployment, incident response, rollback, and maintenance, or a focused procedure for the operation you need. Missing facts are marked explicitly, with proposed defaults separated from confirmed production instructions.
Best for
Engineering teams preparing a service for production, platform engineers standardizing operational procedures, and on-call owners replacing incomplete or outdated runbooks.
SKILL.md preview
---
name: service-deployment-and-on-call-runbooks
description: Create service-specific deployment, incident response, maintenance, and rollback runbooks using observable gates, symptom-driven branches, and application/database compatibility checks. Use when the user says "write a runbook for this service", "document our deployment and rollback", "prepare this service for on-call", "our runbook just says check the logs", "my deployments keep failing and nobody knows when to roll back", "review this incident procedure", or "what should the on-call engineer do first?".
---
# Service Deployment and On-Call Runbooks
… (sign up to view the full skill)More development skills
View all Development skills →Chaos Experiment and Game Day Planner
Plans chaos experiments and game days: hypothesis, blast radius, abort conditions, tool manifests, rollback, and post-run review.
Claude Code Cost and Latency Routing
Configure Claude Code Router with model tiers, explicit fallbacks, compatibility checks, and a cost-aware validation plan.
CI/CD Pipeline Builder with Gated Deploys
Generates CI/CD workflows from your repo's real stack: cached fast-fail checks, pinned actions, and staging-to-prod deploys with gates.
Terraform Modules and Safe Deployments
Get reusable Terraform modules, safe state migrations, explicit provider wiring, and CI gates tied to reviewed plans.