Prompt Governance
Manages the lifecycle of production prompts at scale — versioning, A/B testing, registries, regression prevention, and eval pipelines.
Before & after
Same prompt, same agent — the only difference is the skill.
Prompt management at scale is definitely complex. You'd want to version your prompts so you can track changes and roll back if needed. A/B testing prompts is interesting—you'd need a way to measure the output quality. That could mean customer satisfaction scores, user ratings, or some automated evaluation metric. You'd probably want a prompt registry to keep track of all your prompts. Testing infrastructure is important. You might need a dashboard to monitor performance. There are a lot of moving parts here. Would you like to think through what metrics would indicate a better prompt?
PROMPT GOVERNANCE FRAMEWORK: CUSTOMER SUPPORT
PROMPT REGISTRY (centralized management)
store/prompts/support-reply-v1.yaml → Created 2026-07-15
store/prompts/support-reply-v2.yaml → Created 2026-07-28
Metadata: author, version, date_deployed, rollback_target
A/B TEST SETUP
→ Control (v1): Original prompt (deployed 60 days, baseline behavior)
→ Treatment (v2): "Empathy-first" variant (prioritize customer emotion + solution)
→ Traffic split: 50/50 random assignment per incoming ticket
→ Duration: 14 days minimum (min 500 responses per variant)
QUALITY METRICS (automated evaluation)
→ Response time: Under 30 seconds (measure latency)
→ Customer satisfaction: NPS follow-up via 1-5 emoji (post-reply)
→ Solution rate: Ticket closed in 1 reply vs requires followup
→ Tone detection: Analyze sentiment—apologetic, confident, dismissive
EVAL PIPELINE
Each response scored by:
→ Relevance: Addresses customer question (0-100 score via Claude-eval)
→ Tone: Matches brand voice (rule-based: empathy words, jargon avoided)
→ Length: Under 200 words (penalty if verbose)
→ Actionability: Contains next steps or deadline (binary check)
DATA COLLECTION
→ Every prompt call logged: {prompt_version, ticket_id, response_text, created_at}
→ Manual review: 10% of each variant reviewed by support lead
→ Survey: "Did this answer help?" link in every reply (target: 40% response rate)
RESULTS THRESHOLD
→ Win condition: Treatment avg score > Control by 3+ points (p < 0.05)
→ Minimum sample: 500 responses per variant
→ If v2 wins: Deploy to 100% traffic, create v1-to-v2 rollback snapshot
→ If v1 wins: Archive v2, start new experiment
DEPLOYMENT
→ Winner deploys within 3 hours of test completion
→ Rollback: 1-command revert to previous version (under 5 min)
→ Post-launch monitoring: Track key metrics for 7 daysAbout this skill
name: prompt-governance description: Use when Use when managing prompts in production at scale: versioning prompts, running A/B tests on prompts, building prompt registries, preventing prompt regressions, or creating eval pipelines for product...
Prompt Governance
Use when managing prompts in production at scale: versioning prompts, running A/B tests on prompts, building prompt registries, preventing prompt regressions, or creating eval pipelines for production AI features. Triggers: 'manage prompts in production', 'prompt versioning', 'prompt regression', 'prompt A/B test', 'prompt registry', 'eval pipeline'. NOT for writing or improving individual prompts (use senior-prompt-engineer). NOT for RAG pipeline design (use rag-architect). NOT for LLM cost reduction (use llm-cost-optimizer).
What you get
- Public GitHub repo (alirezarezvani/claude-skills)
- the prompt-governance skill folder with SKILL.md. Part of a 337-skill / 30-agent / 70-command install.
Customize your output
- Fork the repo and adapt the skill's instructions and references to your workflow.
Example output
Activates automatically when your request matches Prompt Governance; chains with the other skills, agents, and commands in the collection.
Best for
Creators, builders, and teams using Claude Code.
SKILL.md preview
---
name: prompt-governance
description: Use this skill when managing prompts in production at scale - versioning prompts, running A/B tests, building prompt registries, preventing regressions, or creating eval pipelines for production AI features.
version: 1.0.0
category: Development / Engineering
author: AgentVolt
license: proprietary
tags:
- development
- engineering
---
# Prompt Governance
Manages the lifecycle of production prompts at scale — versioning, A/B testing, registries, regression prevention, and eval pipelines.
## When to use
… (sign up to view the full skill)More development skills
View all Development skills →Generate
Scaffold code, tests, and boilerplate from a short spec.
Performance Profiler
Systematic performance profiling for Node.js, Python, and Go applications.
Threat Detection
Use when hunting for threats in an environment, analyzing IOCs, or detecting behavioral anomalies in telemetry.
Agent Harness
Turns a domain folder of skills into a bounded agentic loop: compile a goal into a task plan, execute with the domain's tools, verify every task, retry within a budget.