ivyxivyx
← All skills
IVYXSTUDIO · SKILL
P

Pipeline Run Triage

pipeline-run-triage · v1.0.0

ivyx

Use when an MLOps pipeline run (boru hattı) failed or stopped and someone asks why, which data or test is the problem, and what to do; reads the record instead of running it again.

Who is asked

Risk low: it loads without asking anyone. Each tool it then calls still meets your policy.

Tools it may call

While this skill is active, a call to any other tool is refused. These still meet every policy rule; the list narrows, it approves nothing.

mlops.metric.historymlops.pipeline.describemlops.gate.evaluatemlops.dvc.statustrace.readduckdb.queryduckdb.inspectfs.readFile

What it tells the agent

The published SKILL.md, as written.

---
name: pipeline-run-triage
description: Use when an MLOps pipeline run (boru hattı) failed or stopped and someone asks why, which data or test is the problem, and what to do; reads the record instead of running it again.
allowed-tools: mlops.metric.history mlops.pipeline.describe mlops.gate.evaluate mlops.dvc.status trace.read duckdb.query duckdb.inspect fs.readFile
metadata:
  risk: low
---

Why a pipeline run failed

The answer is already on record. Do not run the pipeline again to find out: a run is a write the project asks a person about, and it adds a second failure to the history the person is reading. You change nothing in this procedure; you find the cause and say what would fix it.

1. Find the run and the stage

  1. mlops.metric.history: every gate's result per run. Take the newest run. A stage whose status is failed is where it stopped; give its metric, the measured value and the bound it broke, and when the run happened (atMs).
  2. mlops.pipeline.describe: the stages in order, so you can name the stages that were skipped after the one that stopped.
  3. When no gate failed, the pipeline step itself did: trace.read and find the newest run holding a failed DVC repro or pipeline run call; quote its error. mlops.dvc.status says which stages are out of date and why.
  4. If two or more recent runs failed at the same stage with the same value, say so: nothing changed between them.

2. Find what broke the gate

Always do this before you answer: the history gives a count, not a cause. Call mlops.gate.evaluate with the failed stage's id. It runs that one gate on its own and writes nothing, and its answer names what failed:

Gate capability What it names Then
mlops.pandera.validate (data contract) the column, the check and how many values failed duckdb.query on the stage's data file, passed as file: select * from {{file}} where <the failing condition> limit 5, and the count
test.pytest.run (tests) the failed tests and their assertion messages fs.readFile the test file to say what it expects
mlops.evidently.report (drift) the drifted columns and the share against the threshold duckdb.inspect the current and reference files
mlops.model.evaluate (model quality) the bar that failed, with both numbers nothing more

fs.readFile the contract or test file when you need the rule's exact wording. If the gate now passes while the run recorded a failure, something changed since the run: say both.

If you could not measure the cause, say so. Never write probably about it.

3. Report

  • Why it stopped: the stage, its title, the value against the bound.
  • What is wrong: the column, check, test or metric, with the counts you measured and up to five examples.
  • What would fix it, as choices for the person: fix the data, or change the rule when the new value is legitimate. Never edit the data, the contract or .punica/mlops.yaml yourself, and never suggest loosening a check without saying what it would then let through.
  • Offer to run the pipeline again once the person has made the fix.