Test

Run the test specs for your capabilities, group them, and read pass and fail reports in a panel at the bottom of the editor. Your project's own test suite can run in the same panel once you install a test engine.

What you can do

• Group your test specs by the capability they check, with per-group stats and a run button for each group
• Run everything at once, one group, or a single spec
• Reset results when you want a clean slate
• Read a summary across all specs: total specs, total cases, passed, failed, skipped, and pass rate
• Expand any spec to see its last report, or a preview of the cases that have not run yet
• Score outputs with built-in checks: equality, contains, pattern match, numeric ranges, type, property, and array length
• Run your project's own test suite next to them, with the failing tests listed and the counts folded into the header

Getting started

Open the Test tab in the bottom panel, or run Open Test from the command palette. The test cases declared on your IVY Nodes and IVY Agents are picked up on their own and listed by capability. Press Run All Tests in the header, or right-click a spec and choose Run.

To run your project's own suite, install a test engine such as Python Tests. It appears under ENGINES as its own row; right-click it to run it or to open its panel.

Good to know

• The panel carries no test engine of its own. Each kind of test suite needs a separate engine extension.
• An engine's run can also be a gate in an MLOps pipeline, so a red suite stops the pipeline and both places report it the same way.
• Run All Tests runs only the specs, never an engine. An engine runs from its own row, so no test process starts without you asking for it.
• No workspace or kernel is needed. Results live in memory and reset when you reload.