Fine Tuning Expert
Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks.
Before & after
Same prompt, same agent — the only difference is the skill.
Fine-tuning a language model is becoming more common, but it's not trivial. You'll need training data — good examples of inputs and outputs that show the model what you want it to do. The quality of your data matters a lot; garbage in, garbage out. You could use PEFT or LoRA adapters to keep training efficient. You'll need to think about hyperparameters like learning rate and batch size, though defaults often work okay. Quantization is an option if you want to run the model locally. Hugging Face has some tools for this, and OpenAI has a fine-tuning API. You'd want to evaluate your model on a test set to make sure it's actually better than the baseline. It's worth experimenting with your data and settings to see what works. Let me know if you want to dive deeper into any particular approach.
FINE-TUNING WORKFLOW FOR CUSTOMER SUPPORT LLM
DATA PREPARATION
→ Collect 800-1000 real support conversations (request → response pairs)
→ Format as JSONL: {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
→ Quality check: Remove profanity, hallucinations, off-topic answers; keep only high-confidence examples
→ Split: 80% train (640 examples), 20% validation (160 examples)
HYPERPARAMETER CONFIG (LoRA adapter, using PEFT library)
→ Base model: Claude 3.5 Sonnet (or open-source Llama 3.1 70B if self-hosting)
→ LoRA rank: 16, alpha: 32 (conservative; 4GB VRAM overhead)
→ Learning rate: 1e-4, batch size: 4, epochs: 3 (prevents overfitting on small dataset)
→ Warmup steps: 10% of total training steps
TRAINING STRATEGY
→ Run on single A100 GPU (3 hours total)
→ Evaluate on holdout validation set every 100 steps; early stop if val loss plateaus
EVALUATION
→ A/B test adapter vs baseline on 50 new support tickets
→ Measure: Response relevance (BLEU score vs reference answers), customer satisfaction (binary good/bad), hallucination rate
→ Success threshold: 15% improvement in customer satisfaction
DEPLOYMENT
→ Save adapter weights (45MB vs 70GB full model)
→ Load at inference time: 200ms cold start, 50ms subsequent calls
→ Monitor for drift (new support patterns that adapter hasn't seen)
RISK: Small dataset may overfit to your specific jargon; validate on competitor scenarios.About this skill
name: fine-tuning-expert description: Use when Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks.
Fine Tuning Expert
Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.
What you get
- Public GitHub repo
- the skills/fine-tuning-expert folder with SKILL.md and references.
Customize your output
- Fork the repo and extend the skill's reference files for your own stack conventions.
Example output
Activates on a matching request (e.g. building or reviewing Fine Tuning Expert code) and can chain with other skills in the pack.
Best for
Full-stack developers and engineering teams using Claude Code.
SKILL.md preview
---
name: fine-tuning-expert
description: Use this skill when fine-tuning LLMs or adapting foundation models for a specific task, including LoRA/QLoRA setup, dataset preparation, and hyperparameter selection.
version: 1.0.0
category: Development / Data & ML
author: AgentVolt
license: proprietary
tags:
- development
- data-ml
---
# Fine Tuning Expert
Covers the practical steps of fine-tuning language models: adapter configuration, dataset preparation, hyperparameter selection, and deployment of the resulting model.
## When to use
- Configuring LoRA or QLoRA adapters for a custom task
… (sign up to view the full skill)More development skills
View all Development skills →Pandas Pro
Performs pandas DataFrame operations for data analysis, manipulation, and transformation.
Spark Engineer
Use when writing Spark jobs, debugging performance issues, or configuring cluster settings for Apache Spark applications, distributed data processing pipelines, or big data workloads.
RAG Architect
Designs and implements production-grade RAG systems by chunking documents, generating embeddings, configuring vector stores, building hybrid search pipelines.
Database Optimizer
Optimizes database queries and improves performance across PostgreSQL and MySQL systems.