Robotics

Author a robot arm's or a drone's behavior with an agent, prove it in a simulator, and move it only under a policy.

IVYX Robotics is the workspace where a behavior for a robot arm is designed, proven in simulation and carried to the arm with a person's yes. A behavior is an ivy-agent whose steps are this extension's capabilities (find an object, move the tool, close the jaws); it is proven by running it through randomized trials in a MuJoCo simulator that runs inside IVYX Studio, and the result is written onto the behavior as its evidence. Every move goes through the capability gateway as device.actuate, so a laboratory's .punica/policy.yaml reads each one.

This version drives the simulator. A real SO-101 is the next step; a target that is not the simulator is refused with a sentence that says so.

What you can do

• Set up a workspace: Robotics: Set up this workspace writes the SO-101 arm and its bench as MuJoCo files, .punica/robotics.yaml, an example pick-and-place behavior and a pipeline step that proves it.
• Watch the arm: the Scene tab draws the simulator's own state and follows every move.
• Author a behavior: in the agent editor, the palette's Robot group offers the steps; the robot-behavior-authoring skill writes one from a sentence and runs it once.
• Prove it: Prove in simulation on a behavior runs it through the scenario's trials and writes the count onto the behavior; the Agent Explorer shows it under Evidence, and the MLOps pipeline runs the same step as an Evaluate gate.
• Govern it: the setup can write a laboratory envelope into .punica/policy.yaml: the simulator is free, a real move needs one yes per behavior and must stay inside the envelope, anything outside is refused with its reason.
• Bring your own arm: the Robotics panel's robot row switches to a MuJoCo Menagerie arm (SO-101, Koch, Panda, UR5e, xArm 7, WidowX 250 S), fetched with its meshes and measured; robotics.robot.import converts a URDF of your own. The bench is laid out from the arm's reach.
• Fly a drone: switch to the Skydio X2 or the Crazyflie from the robot menu; the drone is flown before it is written down, lands on a pad that moves every trial, and the gate and pole courses teach going through and going around.
• Move around things: every move is planned around the table, the tray and the objects before it is made, and a pose nothing free reaches is refused in a sentence; robotics.robot.reach draws how far the tool reaches.
• Take the missions: five tasks in order (put the cube in the tray, two cubes, over the wall, the same job on another arm, ask for the real arm), each proven by a scenario, in the Scene drawer's Missions tab.
• Read a run back: the Timeline tab draws a trial's trace, its scrubber puts the arm where it was, a second run lays over the first, and Export as MCAP writes the record for Foxglove.
• Program it in Python: sdk/python (ivyx-robotics) sends every call through ivyx serve --stdio, the same door a model uses, so the envelope and the audit are the same.

What it does not do

• It does not move a real arm yet; the driver for the SO-101's servos is the next step.
• It does not measure force: the simulator and the SO-101's servos have no force sensor, and a force argument is recorded as asked.
• The grasp in the simulator holds an object with a weld once the jaws close on it, not with friction; what the simulation proves is where the behavior approaches and where it lets go.
• It does not read a camera: robotics.scene.locate reads the simulator's state.

Requirements

• IVYX Studio 1.0.69 or later: @punica/editor 1.76.0 (the device.actuate action term) and a window policy that lets the extension frame its own page, where the simulator's Worker runs.
• extension-ivy-agent 1.19.0 or later for the palette's Robot group; extension-ivy-agent-explorer 0.5.0 or later to read a behavior's evidence.

Getting started

1. Open a workspace and run Robotics: Set up this workspace from the palette, or press Set up this workspace in the Robotics panel.
2. Open the Scene (Robotics: Open the Scene).
3. In the Robotics panel, on Pick the red cube and place it in the tray, choose Prove in simulation. Fifty trials run; the row shows the count when they are done.
4. Read the behavior in the Agent Explorer: what it needs from a robot, and the evidence.

Status

0.7.0: a drone is a robot too; the Skydio X2 and the Crazyflie fly under one controller, three courses, three missions and a drone envelope.

0.4.0: the Robotics workbench, one tab for a mission, its behavior as a story with editable heights, the Scene and the proof.

0.5.0: the steps over the trial's clock under the proof, played on the wall clock, with a move's speed set by dragging its bar.

0.6.0: the Robotics icon opens the workbench and the sidebar is gone; the arm's menu, the joints and a searchable behavior list are in the workbench.

0.3.0: a robot from Menagerie or a URDF with its meshes, planned moves and a reach map, missions, the Scene beside the agent editor, run traces with a timeline and MCAP export, and a Python SDK; three behavior kinds. See docs/CLAUDE.md for what each was measured against.

0.2.0: a behavior is run once with robotics.behavior.run, which the authoring skill uses; the skills were measured with a local model (see docs/CLAUDE.md).

0.1.0: the simulator, the SO-101 model, the example behavior, the scenario step, four skills. The real arm is Dilim 2.