Vision

Watch a camera, a simulator or a video for what changes and start an agent on the event.

A watch reads a source's pictures a few times a second, here on your machine, with no model and no cost, and writes an event when something changes: a part appeared where none was, a part went, a part of the wrong colour came by, a corner of the picture moved. An event can start an agent, once, so the expensive look (a model reading the picture) happens only when something happened, never on every frame. Nothing is kept unless you ask, and then only for as long as you say.

What you can do

• List the sources this workspace can watch: the simulator's cameras when IVYX Robotics is installed (sim:overhead, sim:wrist_cam), any video file in the workspace (video:examples/line-qc/line.webm).
• Start a watch on a source with the rules that count as an event, how often to look, and how long the same thing stays one event.
• Read the events a watch made, with the picture behind each one for a few seconds, or longer when the watch keeps frames.
• Start an agent on an event: name the agent, the key that makes two events one, and the cooldown; the agent runs through the gateway under your policy, and the watch records what it started and what it skipped.
• Ask a model about an event: name the question and the core, and every part that appears is shown to the model as a picture; its one-line verdict is recorded, and a defect is raised for a person as an approval step.
• Stop a watch, or every watch, from the palette.

What it does not do

It tells colours and movement apart; it does not recognise, identify or follow a person. A frame is sensitive data: under a policy that denies remote egress no frame reaches a remote model, and no frame is written to disk unless a watch says so.

Requirements

IVYX Studio 1.0.73 or newer, which plays a workspace video in the window. IVYX Robotics 0.32.0 or newer for the simulator's cameras. A model for the questions an event asks: a local one, or a remote one you have chosen.

Getting started

1. Install this extension, and IVYX Robotics if you want to watch the simulator.
2. Ask the assistant to list the sources it can watch.
3. Ask it to watch one, for example the bench camera for a red part that appears, and to start your pick-and-place behavior when one does.
4. Move the part, or play a video of your line, and read the events and what they started.

Status

0.1.0 is the first release: watches, events, an agent or a model started by an event, and a person asked about a defect. There is no panel yet; events and verdicts are read through the assistant. A real camera is not supported yet.

Contract

punica-editor/docs/vision-sources.md is the contract between this hub and the extensions that serve frames; docs/CLAUDE.md is what was measured.