Senior Ml Engineer
ML engineering skill for productionizing models, building MLOps pipelines, and integrating LLMs.
Before & after
Same prompt, same agent — the only difference is the skill.
Model drift monitoring is important for maintaining model performance over time. You'll want to set up some kind of tracking for how your model's predictions compare to actual outcomes, and alert if there's significant degradation. You might use tools like MLflow or Evidently AI to help with this. You should probably define what drift means for your use case—whether it's prediction shift, feature shift, or performance degradation. You'd want to collect baseline metrics from when the model was trained and compare current performance to those baselines. It might help to set up automated alerts when performance drops below a threshold. You could also periodically retrain your model on fresh data. Would you like guidance on which monitoring tool to use, or help defining drift thresholds for your specific problem?
MODEL DRIFT MONITORING SETUP DEFINE DRIFT TYPES → Data drift: Feature distributions changed (e.g., user demographics shift) → Prediction drift: Model output distribution drifts (e.g., predictions skewed to one class) → Performance drift: Accuracy drops vs. production baseline (e.g., precision drops below 85%) BASELINE METRICS (from training) → Accuracy: 0.89 (Recorded: 2026-06-01) → Precision (class A): 0.91 → Recall (class A): 0.87 → Feature means + std (all 47 features logged) MONITORING PIPELINE → Tool: Evidently AI (open source) or Arize (managed) → Cadence: Daily checks at 2am UTC → Sample: Latest 1,000 predictions + actuals → Metrics computed: KS test for distribution shift, current accuracy vs. baseline ALERT THRESHOLDS → CRITICAL (page on-call): Accuracy drops >5% (below 0.84) → WARNING: Any feature drifts >3 std from baseline → INFO: Prediction distribution shift >10% (logged, not alerted) RESPONSE PLAYBOOK CRITICAL ALERT → Investigate root cause (feature drift? label corruption? concept drift?) → If fixable (bad feature): Hot-patch feature pipeline → If model issue: Trigger retraining on last 30 days data → If data issue: Quarantine predictions, rollback to previous model version IMPLEMENTATION → Dashboard: Shows current metrics vs. baseline, drift score over time → Logging: All predictions logged to data warehouse for audit → Cost: ~200/month for monitoring service + retraining compute
About this skill
name: senior-ml-engineer description: Use when ML engineering skill for productionizing models, building MLOps pipelines, and integrating LLMs.
Senior Ml Engineer
ML engineering skill for productionizing models, building MLOps pipelines, and integrating LLMs. Covers model deployment, feature stores, drift monitoring, RAG systems, and cost optimization. Use when the user asks about deploying ML models to production, setting up MLOps infrastructure (MLflow, Kubeflow, Kubernetes, Docker), monitoring model performance or drift, building RAG pipelines, or integrating LLM APIs with retry logic and cost controls. Focused on production and operational concerns rather than model research or initial training.
What you get
- Public GitHub repo (alirezarezvani/claude-skills)
- the senior-ml-engineer skill folder with SKILL.md. Part of a 337-skill / 30-agent / 70-command install.
Customize your output
- Fork the repo and adapt the skill's instructions and references to your workflow.
Example output
Activates automatically when your request matches Senior Ml Engineer; chains with the other skills, agents, and commands in the collection.
Best for
Creators, builders, and teams using Claude Code.
SKILL.md preview
---
name: senior-ml-engineer
description: Use this skill when deploying ML models to production, setting up MLOps infrastructure, monitoring model drift, building RAG pipelines, or integrating LLM APIs with cost controls.
version: 1.0.0
category: Development / Engineering
author: AgentVolt
license: proprietary
tags:
- development
- engineering
---
# Senior Ml Engineer
Covers productionizing machine learning models: deployment, feature stores, drift monitoring, RAG pipeline construction, and LLM integration with retry logic and cost controls, focused on operational concerns rather than research or training.
## When to use
… (sign up to view the full skill)More development skills
View all Development skills →Generate
Scaffold code, tests, and boilerplate from a short spec.
Performance Profiler
Systematic performance profiling for Node.js, Python, and Go applications.
Threat Detection
Use when hunting for threats in an environment, analyzing IOCs, or detecting behavioral anomalies in telemetry.
Agent Harness
Turns a domain folder of skills into a bounded agentic loop: compile a goal into a task plan, execute with the domain's tools, verify every task, retry within a budget.