← All extensions
Compute banner

Compute

ivyx✓

Run a pipeline step on another machine and bring back what it wrote, with the run recorded here

Compute

Run a pipeline step on another machine and bring back what it wrote, with the run recorded here.

Training that needs a GPU you do not have, or more hours than your laptop can spare, runs on a machine you name. The data goes with the job, the model comes back into your workspace, and the run lands in your local MLflow beside the ones you trained here, so the quality gate, lineage and the evidence package read it the same way.

What you can do

  • Add the machines you may train on from a form in the Compute panel. It writes .punica/compute.yaml for you, keeps any comment you wrote there, and puts a Jupyter Server's token in the Secret Manager, never in the file.
  • Add a training step to your pipeline — Train on a Remote Machine is a step in the MLOps designer's palette, with the command, the files to send and the files to bring back as its settings.
  • Run it again without starting it again — the same work attaches to the job already running, so a long job outlives one run of the pipeline.
  • Watch and stop jobs — the panel lists every job with its state, shows its log and cancels a running one.
  • Keep the record here — each finished job is one MLflow run, tagged with the machine and the job.

What it does not do

It does not hold your keys. An SSH machine signs in through your own ssh and its agent, and a token is read from the secret manager by name. A value that looks like a secret is refused in a job's settings.

It does not call a job trained before its files are back. A job that is still running, failed, or wrote none of the outputs it was asked for fails its step with the reason.

It does not send data a policy keeps here. Sending a job counts as data leaving the machine, so a deny egress: remote rule in .punica/policy.yaml stops it.

Requirements

A compute engine for the kind of machine you name: Compute over SSH for a machine you reach with ssh, Compute on Jupyter for a Jupyter Server. The Secret Manager extension keeps a Jupyter Server's token. The MLOps extension 1.29.0 or newer draws the step on its canvas.

Getting started

  1. Install this extension and a compute engine.
  2. Open Compute and press Add a machine.
  3. Fill in its name, kind and address, then press Save and check. The result arrives as a notification.
  4. In the MLOps designer, drag Train on a Remote Machine into the pipeline, fill in the command and the files, and run.

Status

0.2.0 adds the form, and reads every secret through the Secret Manager. The SSH and Jupyter engines are live.