Self-Improving Skill Overnight Loop
Run an automated overnight research loop on a Claude skill: try a tweak, run binary assertions, keep or revert based on pass rate, and iterate.
Before & after
Same prompt, same agent — the only difference is the skill.
Automated self-improvement of a prompt or skill is an interesting idea. The general approach would be: - Have the agent propose a change - Test the change against some examples - Keep it if it improves, revert if it doesn't - Repeat You'd need a way to measure "better" objectively, which is the hard part. Binary pass/fail tests on example outputs could work. You'd also want version control so you can revert cleanly. Be cautious about letting it run fully unattended, since automated changes can drift in unexpected ways. What skill are you trying to improve?
Yes — a research loop that measures "better" with binary assertions, so it can decide keep-or-revert without you. Here's the loop and the one thing that makes it safe. THE LOOP 1. Claude reads the skill.md 2. Proposes ONE change 3. Runs 5 test prompts through the changed skill 4. Scores each output against 25 true/false assertions (125 checks) 5. ALL pass → git commit. ANY fail → git reset, try a different change 6. Repeat until interrupted or a perfect score holds WHY BINARY ASSERTIONS ARE THE WHOLE TRICK "Is this output better" is subjective and un-loopable. "Does the output contain a dated action item — true/false" is checkable, and 25 of those give you a hard pass-rate the loop can act on without a human. Vague scoring is what makes self-improvement drift; binary scoring is what makes it converge. THE SAFETY MECHANISM — git as the undo Every kept change is a commit; every failed change is a reset. The skill can NEVER end up worse than its last passing state, because a regression is reverted before the next iteration. That's what makes it safe to run overnight — the floor is fixed. WHAT YOU GET The read-propose-test-score-commit loop, a 25-assertion scoring harness, git-gated keep/revert, and no-permission iteration until interrupted. THE INPUT THAT DECIDES QUALITY The 25 assertions. Write them to capture what "good" actually means for this skill and the loop optimizes toward it; write them loosely and it optimizes toward nothing. Which skill, and what does a great output of it contain?
About this skill
name: self-improving-skill-overnight-loop description: Run the original author's auto-research loop on a Claude skill: try a tweak, run binary assertions, keep or revert based on pass rate, and iterate all night.
Self-Improving Skill Overnight Loop
What This Does
The creator adapts the original author's three-file loop to skills by having Claude read the skill.md, propose a change, run five test prompts, and score outputs against 25 true or false assertions. If all pass, git commit; if any fail, get reset and try again, looping without asking for permission until interrupted or a perfect score is hit.
What You Get
- evals.json with true or false binary assertions
- auto-modify skill.md loop
- git commit or git reset per iteration
- per-iteration pass rate log
- overnight autonomous run
Customize
- assertions: array of binary tests per test prompt
- test_prompts: sample inputs to grade
- max_iterations: cap to prevent infinite loops
- commit_on_pass: whether to auto-commit each improvement
- stop_when: perfect score, manual, or time limit
Sample Output
First run scores 23 of 24 assertions, Claude adds one clarifying rule to the skill.md, second run hits a perfect score and commits.
Target Audience
Skill authors iterating on production skills
SKILL.md preview
---
name: self-improving-skill-overnight-loop
description: Use this skill when a skill author wants an automated overnight loop that tweaks a skill, scores it against test assertions, and keeps or reverts the change.
version: 1.0.0
category: Agent Orchestration
author: AgentVolt
license: proprietary
tags:
- agent-orchestration
- standard
---
# Self-Improving Skill Overnight Loop
Runs an unattended improvement loop on a skill definition: propose a tweak, run it against sample prompts, score the outputs, and keep the change only if it improves the pass rate.
## When to use
… (sign up to view the full skill)More ai & agents skills
View all AI & Agents skills →Ralph Loop Autonomous Overnight Runner
Wire the Claude Code CLI mode (-p flag) into the Ralph loop plugin so a while-loop keeps feeding the same prompt with updated files until the task is done.
Recursive Skill Failure-Fix Loop
Every time your skill fails on a real task, feed the failure back and have Claude update the skill.md so it never breaks that way again.
Critique Sub-Agent Reviewer
Every time the main agent produces copy, research, or code, hand it to a critique sub-agent that returns fix notes before the output ships.
AI Starter Toolkit
A free, curated starting point — the essential prompts and habits to get real value from AI on day one.