MLOps

Author a gated training pipeline, run it locally, and see exactly which gate stopped it.

The hub owns the process — stage order, the stop rule, the record every run leaves behind — and carries no engine. DVC, pandera, Evidently and pytest arrive as separate provider extensions and are reached through the capability gateway, so this extension has no dependency on any of them and says so plainly when none are installed.

What you can do

• Lay out a training pipeline as ordered stages, and run the whole thing locally
• Put gates between the stages — a schema check, a drift check, a test suite — and have a red one stop the run instead of being a comment
• See exactly which gate stopped a run and why, rather than reading back through a log
• Add engines as you need them: DVC for reproduction, pandera for data contracts, Evidently for drift, pytest for tests
• Re-run the pipeline when your data moves, with drift able to trigger the retrain

Why it exists

Between "the notebook trained something" and "this model is in production" there is a gap nothing in the workspace covered: did the data schema break, did the tests pass, can this run be reproduced, and which of those questions failed. A gate that cannot stop the pipeline is a comment; this hub is what turns the checks into a run with a verdict.

Getting started

Open MLOps from the activity bar. On a workspace with no pipeline, the panel walks you the rest of the way — the full first-pipeline path is below.

Your first pipeline

You need a workspace folder, a Python environment the workspace can see, and one csv to train on. Everything below runs on this machine. Nothing is uploaded.

The panel walks the path with you. After the first click there is a Next step card above the stage list that reads the workspace and names the single thing that is missing. The steps below are what it will say, in order, so you can see where you are going. Do the one thing it asks and nothing else.

1. Open MLOps from the activity bar. On a workspace with no pipeline the panel offers Create mlops.yaml. That writes .punica/mlops.yaml with one stage bound to the dvc engine. The file is the pipeline. It stays in git, it is what a generated CI workflow is built from, and you can edit it by hand at any point.

2. Install the engine. The card will tell you it is missing by package name. Open Extensions and install extension-mlops-dvc.

3. Write the components before the pipeline that calls them. With the engine installed the card says the DAG is empty, and offers Scaffold train.py and evaluate.py. That writes src/train.py and src/evaluate.py already wired to experiment tracking, with the exit codes a gate reads. The model and the features stay your work, and the card prints what the scaffold could not decide for you, such as which column is the target.

4. Write the DAG. The same card offers Open DVC. That panel initializes the repository once (Initialize DVC here), then writes a commented dvc.yaml (Create dvc.yaml). Open the file and uncomment the train block, so the stage runs python src/train.py. A stage whose command does not exist fails at the furthest possible point from the file that declared it, which is why the components came first.

5. Add a gate. Now the card says the pipeline has nothing that can stop it. Install extension-mlops-pandera, open Data Contracts, and use Draft from a dataset… pointed at your csv. It reads the columns and their ranges and writes a contract you can edit. Back in MLOps the provider appears under PROVIDERS. Right click it and choose Add to pipeline. The gate is written into .punica/mlops.yaml above the training stage, because a contract that runs after training has protected nothing.

6. Run it with the play button in the MLOps header. Every stage is recorded, including the ones that did not get to run.

What a stopped run looks like

This is the part worth doing. Add a row to your csv that breaks the contract, a null in a column the draft marked as required or a value outside the range it measured, and run again.

The run stops at the contract stage. The stage card names the column and the check that failed, the stages after it are recorded as skipped rather than quietly omitted, and the notification says which gate ended the run. That record survives a restart, so a run that stopped last night is still answerable this morning.

A gate that cannot stop the pipeline is a comment. This is the difference.

The pipeline

Stages come from .punica/mlops.yaml and run in the order written there:

version: 1

pipeline:
  provider: dvc          # which engine runs the DAG

stages:
  - id: data-contract
    kind: gate
    capability: mlops.pandera.validate
    input:
      contract: contracts/train.py
    onFailure: stop

  - id: train
    kind: pipeline       # dispatched to the bound provider
    onFailure: stop

  - id: model-quality
    kind: gate
    capability: mlops.quality.check
    onFailure: continue

• kind: gate calls a capability and reads its verdict. The contract is { pass: boolean, reason?: string }; { ok } and { exitCode } are tolerated so a stage can name a capability from outside this family. An answer that carries none of those is a failure, never a pass — a gate that cannot be read must not wave a model through.
• kind: pipeline dispatches to mlops.<provider>.repro.
• onFailure: stop ends the run. Stages after it are recorded as skipped, so the record shows what did not get to run.
• A file that binds a provider but declares no stages still runs the DAG as a single implicit stage, and the panel says the stage was synthesized.
• Anything the parser has to ignore (a gate with no capability, a duplicate id, an unknown kind) is reported in the panel, not silently dropped.

Running it again when the data moves

A pipeline that only runs when someone types a command is not continuous training. triggers says what an outside event does:

triggers:
  onDrift: retrain       # ignore (default) | retrain
  stages: [train]        # optional: what a fired trigger runs

With retrain, a drift provider reporting that the data has moved runs the pipeline. The hub does not decide what drift is: a provider publishes mlops.drift.detected because it is the component that measured it, and this hub reads the file to decide what to do about it. Swapping Evidently for another drift engine changes nothing here.

ignore is the default on purpose. Installing a drift provider must not silently give a workspace a pipeline that starts itself, and the sidebar says so in words when it can.

Drift found by a gate inside a running pipeline does not start a second run. That drift belongs to the run that found it, and retriggering would run the pipeline whose own gate would report again.

mlops.trigger.fire is the same entry point for everything else: a schedule, a commit hook, or an MCP client. It records who asked and why, durably, because that record is the only trace an unattended run leaves behind.

Capabilities

Capability	Policy	What it does
mlops.pipeline.run	medium / none	Runs the stages in order; stops at the first failing stop-gate
mlops.trigger.fire	medium / none	The same run, from outside: records the source and reason first
mlops.gate.evaluate	low / none	One gate, headless — by stage id or an inline capability
mlops.pipeline.describe	low / none	The configured stages, the triggers, and which providers answer
mlops.open	low / none	Reveal the sidebar

Three events are published as durable audit records (kind: 'audit'): mlops.pipeline.finished, mlops.gate.failed and mlops.retrain.requested. A run that stopped, and a run that started itself, are both answerable after a restart, which is the point of recording them at all.

Both run capabilities declare a write side effect, and that is load-bearing rather than paperwork. The stages inside a run are dispatched as this extension, so the substrate's agent-mode policy sees only the outer capability. Declaring the write is what makes an AI caller — an MCP client on the local port, for instance — get asked, while the in-app drift chain stays automatic.

Model promotion lives in Experiments (mlops.model.promote, risk high with a step approval) because the registry lives in the MLflow server that extension already manages.

Limits worth knowing

• Provider capabilities run through the gateway, so a missing provider is reported by name (extension-mlops-dvc) instead of failing obscurely.
• Only one run at a time: two reproductions over one workspace race on the same cache, so the second caller is refused rather than queued.
• Engines are Electron-only (they spawn processes), so a browser host reports the stage as unavailable rather than pretending it passed.

Verification

npm run check:pipeline drives the shipped stage engine against a fake gateway: stage order, the stop rule, skipped stages, provider-missing, a throwing gate, verdict normalization, the durability declaration, the concurrent-run refusal, and the drift chain end to end (drift fires a configured retrain, does nothing when unconfigured, and does not re-enter a run in flight).