Statistical Analyst
Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
Before & after
Same prompt, same agent — the only difference is the skill.
An 8% lift sounds promising, but whether it's real depends on a few things like sample size, how long the test ran, and your baseline numbers. Larger sample sizes and longer test durations generally give you more confidence in the result. Without seeing your actual data, I can't say for certain if the lift is statistically significant. If you share the conversion rates and sample sizes for both variations, I could look at the numbers more carefully.
HYPOTHESIS TEST RESULTS TEST SETUP Control conversion rate: 12.4% (n=8452 users) Treatment conversion rate: 13.4% (n=8396 users) Observed lift: 8.1% (relative) Absolute difference: +1.0pp STATISTICAL ANALYSIS Two-proportion Z-test Z-statistic: 2.14 P-value: 0.032 (p < 0.05) Conclusion: STATISTICALLY SIGNIFICANT at 95% confidence level CONFIDENCE INTERVAL 95% CI on lift: 0.1-1.9pp Interpretation: We're 95% confident true lift is between 0.1 and 1.9 percentage points Note: Lower bound (0.1pp) is practically small EFFECT SIZE Cohen's h: 0.04 (small effect) Practical significance: Modest improvement, not transformational FALSE POSITIVE RISK P-value 0.032 vs typical threshold 0.05 Risk of false positive: 3.2% (acceptable) But: Running multiple tests or peeking at data increases false positive risk SAMPLE SIZE ADEQUACY Powered for: 80% detection of 1.2pp lift Actual detected: 1.0pp lift Duration: 14 days at current traffic (sufficient) RECOMMENDATION Deploy treatment. The lift is real and statistically significant. Caution: Effect is small; monitor post-launch for sustainability. Next test: Run longer (28 days) to confirm lift persists over time. IMPORTANT: This analysis is statistical illustration, not investment or trading advice
About this skill
name: statistical-analyst description: Use when Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes.
Statistical Analyst
Run hypothesis tests, analyze A/B experiment results, calculate sample sizes, and interpret statistical significance with effect sizes. Use when you need to validate whether observed differences are real, size an experiment correctly before launch, or interpret test results with confidence.
What you get
- Public GitHub repo (alirezarezvani/claude-skills)
- the statistical-analyst skill folder with SKILL.md. Part of a 337-skill / 30-agent / 70-command install.
Customize your output
- Fork the repo and adapt the skill's instructions and references to your workflow.
Example output
Activates automatically when your request matches Statistical Analyst; chains with the other skills, agents, and commands in the collection.
Best for
Creators, builders, and teams using Claude Code.
SKILL.md preview
---
name: statistical-analyst
description: Use this skill when a user needs to run hypothesis tests, analyze A/B experiment results, calculate required sample sizes, or interpret statistical significance and effect size before trusting a result.
version: 1.0.0
category: Development / Engineering
author: AgentVolt
license: proprietary
tags:
- development
- engineering
---
# Statistical Analyst
Runs the actual statistics behind an experiment — hypothesis tests, sample-size calculations, significance with effect size — so decisions aren't made on a raw percentage difference.
## When to use
… (sign up to view the full skill)More development skills
View all Development skills →Generate
Scaffold code, tests, and boilerplate from a short spec.
Performance Profiler
Systematic performance profiling for Node.js, Python, and Go applications.
Threat Detection
Use when hunting for threats in an environment, analyzing IOCs, or detecting behavioral anomalies in telemetry.
Agent Harness
Turns a domain folder of skills into a bounded agentic loop: compile a goal into a task plan, execute with the domain's tools, verify every task, retry within a budget.