ROBOTICS
Prove a robot's behavior before anything moves
Write a behavior for an arm or a drone, run it through randomized trials in a simulator, and let a policy decide every move.
A behavior is an ivy-agent whose steps are robot moves: find an object, move the tool, close the jaws, take off, land. A MuJoCo simulator runs inside IVYX Studio, proves the behavior over many trials and writes the count onto it. Every move goes through the capability gateway, so the laboratory's policy reads each one.
From a sentence to a proven behavior
Each stage is a capability, so each one goes through the same gateway and lands in the same record.
- 1
Write
robotics.behavior.write
Ask for a pick-and-place, a stack or a flight. The file is written in the example's shape, checked against what is on the bench.
- 2
Run once
robotics.behavior.run
One run in the simulator, drawn in the Scene, with the result read from the simulator rather than from the agent.
- 3
Prove
robotics.sim.scenario
Trials with the object placed somewhere new each time. The count is written onto the behavior.
- 4
Publish
Ivy Marketplace
The behavior goes out with its steps, its test case and its proof, and installs beside the extension it needs.
- 5
A real robot
not yet
This version drives the simulator only. A move meant for anything else is refused, and the refusal says so.
A behavior is an agent
Steps are moves
Find an object in the scene, offset a pose, move the arm, open or close the gripper, take a drone off, fly it, land it. The agent editor offers them as steps under its Robot group.
Written from a sentence
Ask the AI Assistant for a pick-and-place, a stack or a flight, and the robot-behavior-authoring skill writes the behavior in the shape of the example and runs it once.
It says what it needs
The behavior's file names the robot it was written for, an arm with a gripper or a drone, and the extension its steps belong to. A behavior for another robot asks you to import that robot first.
Published like any agent
A proven behavior is published to the Ivy Marketplace with its steps, its tests and its proof. Adding one asks for extension-robotics, which comes from the extension marketplace.
Red object → tray · SO-101, step by step
ivy.agent.pick-place-so101
- 01Find the red objectscene.locate
- 02A pose above the cubepose.offset
- 03Move above the cubearm.moveTo
- 04The grasp posepose.offset
- 05Come down on the cubearm.moveTo
- 06Close the jawsgripper.set
- 07A pose to lift topose.offset
- 08Lift the cubearm.moveTo
- 09Find the trayscene.locate
- 10A pose over the traypose.offset
- 11Carry it over the trayarm.moveTo
- 12A pose to let go frompose.offset
- 13Lower into the trayarm.moveTo
- 14Open the jawsgripper.set
20 of 20 trials in the simulator · MuJoCo 3.17.0
Red object → tray · SO-101, as the marketplace lists it today. See the whole behavior
Proven in the simulator
MuJoCo runs inside the window, in a Worker of its own. A scenario places the object somewhere new on every trial, runs the behavior through the gateway as a real run, and reads the result from the simulator: was the object inside the tray, did the drone land on the pad, how many unintended contacts were counted.
The count is written onto the behavior as its evidence, and the scenario the setup writes passes at nine trials in ten with no contact. Change a height or a speed and the old proof no longer counts. The same step runs as an Evaluate gate in an MLOps pipeline.
What the published behaviors were proven with
- 9
- behaviors on the marketplace
- 178 / 180
- trials succeeded
- 0
- unintended contacts
- MuJoCo 3.17.0
- in the simulator
Summed from the proof each behavior was published with. The publisher ran these trials; the registry keeps the counts and runs nothing again.
The gate reads every move
One action word for moving
Every move declares device.actuate, so a single policy rule can select all of them, on an arm or a drone.
A move names its target
A move must say whether it is meant for the simulator or a real robot. A rule can let the simulator run freely and hold everything else.
A yes covers a behavior
Approval binds to the behavior, not to a pose that changes on every call. One yes for a behavior covers its moves.
An envelope, written for you
Setting up a workspace can write a laboratory envelope: the arm at most 0.3 m/s and 2 cm above the table, the drone inside a 4 m cage under 1.5 m. A move outside it is refused with its reason.
- name: so101-lab-envelope
description: The student's arm moves on the table top,
slowly, after one yes per behavior.
rules:
- kind: capability
action: [device.actuate]
effect: deny
when:
- { path: target, op: neq, value: sim }
- { path: speed, op: gt, value: 0.3 }
reason: The move is faster than the laboratory
envelope allows (at most 0.3 m/s).
- kind: capability
action: [device.actuate]
effect: require
approval: plan
scope: workspace
when:
- { path: target, op: neq, value: sim }
- kind: capability
action: [device.actuate]
effect: allow
when:
- { path: target, op: eq, value: sim }Bring the robot you teach with
Six arms and two drones from MuJoCo Menagerie. Each is fetched with its meshes and measured on your machine: where an arm's jaws close, how far it reaches, and a bench laid out from that reach. A URDF of your own imports too.
Arms
- SO-101
- Koch
- Franka Panda
- UR5e
- xArm 7
- WidowX 250 S
Drones
- Skydio X2
- Crazyflie 2
A behavior is on the marketplace
Moves are planned around the table, the tray and the objects before they are made. A pose nothing free can reach is refused in a sentence instead of being tried.
Work with it
The workbench and the Scene
The mission, the behavior as a story with editable heights, the simulator drawn as it runs, and the proof, on one tab. Drag a cube, the tray or a gate across the table and run again.
Missions
Tasks in order, from putting the cube in the tray to going over a wall, each proven by its scenario. A student starts here.
Read a run back
The Timeline draws a trial's trace and puts the arm where it was. Export a run as MCAP and open it in Foxglove.
Program it in Python
The ivyx-robotics client sends every call through ivyx serve, the same door a model uses, so the envelope and the record are the same.
Behaviors on the marketplace
9 behaviors are published today, each with the simulator proof it was published with. Add one from the Robotics tab of the Ivy Marketplace inside IVYX Studio.
What it does not do
It does not move a real robot yet
This version drives the simulator. A move for any other target is refused, and the refusal says so.
It does not measure force
Neither the simulator nor the SO-101's servos has a force sensor. A force asked for is recorded as asked.
A grasp is held, not gripped
Once the jaws close on an object the simulator holds it to them, without friction. What a proof shows is where the behavior approaches and where it lets go.
It does not see
Finding an object reads the simulator's state. No camera is read.
A proof is the publisher's
The registry keeps the count a behavior was published with and runs nothing again. Prove it on your own bench before you rely on it.