LLM Fine-Tuning Plan: LoRA, QLoRA, SFT, DPO, Evals
Decide if fine-tuning beats prompting, then get a dataset spec, LoRA/QLoRA config, eval plan, and fixes for failing training runs.
Before & after
Same prompt, same agent — the only difference is the skill.
Fine-tuning a language model is becoming more common, but it's not trivial. You'll need training data — good examples of inputs and outputs that show the model what you want it to do. The quality of your data matters a lot; garbage in, garbage out. You could use PEFT or LoRA adapters to keep training efficient. You'll need to think about hyperparameters like learning rate and batch size, though defaults often work okay. Quantization is an option if you want to run the model locally. Hugging Face has some tools for this, and OpenAI has a fine-tuning API. You'd want to evaluate your model on a test set to make sure it's actually better than the baseline. It's worth experimenting with your data and settings to see what works. Let me know if you want to dive deeper into any particular approach.
FINE-TUNING WORKFLOW FOR CUSTOMER SUPPORT LLM
DATA PREPARATION
→ Collect 800-1000 real support conversations (request → response pairs)
→ Format as JSONL: {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
→ Quality check: Remove profanity, hallucinations, off-topic answers; keep only high-confidence examples
→ Split: 80% train (640 examples), 20% validation (160 examples)
HYPERPARAMETER CONFIG (LoRA adapter, using PEFT library)
→ Base model: Claude 3.5 Sonnet (or open-source Llama 3.1 70B if self-hosting)
→ LoRA rank: 16, alpha: 32 (conservative; 4GB VRAM overhead)
→ Learning rate: 1e-4, batch size: 4, epochs: 3 (prevents overfitting on small dataset)
→ Warmup steps: 10% of total training steps
TRAINING STRATEGY
→ Run on single A100 GPU (3 hours total)
→ Evaluate on holdout validation set every 100 steps; early stop if val loss plateaus
EVALUATION
→ A/B test adapter vs baseline on 50 new support tickets
→ Measure: Response relevance (BLEU score vs reference answers), customer satisfaction (binary good/bad), hallucination rate
→ Success threshold: 15% improvement in customer satisfaction
DEPLOYMENT
→ Save adapter weights (45MB vs 70GB full model)
→ Load at inference time: 200ms cold start, 50ms subsequent calls
→ Monitor for drift (new support patterns that adapter hasn't seen)
RISK: Small dataset may overfit to your specific jargon; validate on competitor scenarios.About this skill
Your fine-tuned model never stops generating. Or the loss hit zero while the outputs got worse. Or you spent a week training on facts that a retrieval step would have handled in an afternoon. This skill handles the decisions that make or break a fine-tune: whether to tune at all, what the data and eval set must look like, which settings to start from, and how to read a run that's going wrong.
What it does
- Tells you whether your goal needs fine-tuning, a better prompt, or RAG, and what baseline you must beat first
- Specs the dataset: chat-template format, loss masking, dedup, leakage-safe splits, and realistic example counts for your task type
- Gives starting LoRA/QLoRA/full-SFT/DPO settings (rank, alpha, target modules, learning rate, epochs, sequence length) with the reasoning behind each
- Diagnoses failing runs from symptoms: endless generation, flat or collapsing loss, NaNs, overfitting, DPO degradation, forgetting
- Plans merge, quantization, and serving so the deployed model matches what you evaluated
- Writes the training or data-prep code inline when you ask (PEFT/TRL or hosted fine-tuning JSONL)
What you get
A fine-tuning plan sized to your request: a go/no-go call, a data and eval spec, a config with stated assumptions, and a short list of failure checks. If you paste logs or outputs, you get a ranked diagnosis instead.
Best for
ML engineers and app developers adapting open-weight or hosted LLMs to a fixed output format, a tone, a classification task, or a cheaper distilled model. It is also for anyone debugging a run that "trained fine" but behaves badly.
SKILL.md preview
---
name: llm-fine-tuning-plan-lora-qlora-sft-dpo
… (sign up to view the full skill)More development skills
View all Development skills →Pandas Data Wrangling Without Silent Errors
Correct, fast pandas code for merging, cleaning, groupby, reshaping, and time series, with checks that catch silent row loss.
Apache Spark Job Tuning and Debugging
Finds why a Spark job is slow or failing from the plan and UI metrics, then gives the code and config fix plus how to verify it.
Reliable Data Pipelines and Analytics Models
Build and repair ETL/ELT pipelines with replay-safe writes, explicit data contracts, and actionable quality checks.
Statistical Tests, Power & A/B Readouts
Get hypothesis tests, effect sizes, confidence intervals, and sample-size calculations with experiment validity checks.