AI agents in the scene: how they became participants

Most AI tools put the model in a side panel. You type, it answers, and the work happens somewhere you don't see. In ARClip an agent works the way a person does: it is in the scene, everyone can see what it is doing, and you can talk to it while it builds.

This works for Claude or Cursor connected over MCP, for a team of models and for our built-in assistant.

Visible, like a person

When an agent changes the scene, an avatar appears over the object it is working on — the same kind of avatar people have, with hands, a name and its own color. The label says who it is and on whose behalf: "Claude · Anna". It also gets a row in the participant list, next to the people in the room.

With several agents at once, each has its own color, so on a busy scene you can tell at a glance who is building the dock and who is building the boat.

It reads the room

An agent can read the room's journal: everything typed in the chat, lines people said aloud to it, and a short log of what people did in the scene — "moved Table 3", "removed four blocks". Its own messages go to the same chat, and it can pin a note to an object: "the door opens this way".

So you don't switch to a separate window to direct it. Write in the room chat — "make the path wider, kids keep falling off" — and the agent working on the course sees it.

It hears and sees — only if you allow it

Two senses stay off until people in the room turn them on with the Agents button in the collaboration panel:

• Hearing. While your microphone is live in a call, your own browser turns your speech into text and adds it to the room journal. It only hears your microphone, and only in Chrome for now.
• Sight. While you share your screen, the agent can look at its latest frame — for example, to see the reference you are showing. Frames are kept for a minute.

Both permissions end with the call or the screen share. Next time, you decide again.

Undo is per person

When people and agents edit the same scene, Cmd+Z has to know whose step it undoes. Steps of agents connected over MCP and of team agents sit in their own history: open the participant list and undo an agent's last step from its row, without touching anything you did. Your Cmd+Z never takes them back. The built-in assistant works on your behalf, so its steps are yours to undo with Cmd+Z.

The conversation stays

The room chat keeps its history by session for 90 days, for signed-in members of the project. Open it from the chat or from the navigation bar and read back who was in the room — people and agents — and what was said, typed or aloud.

How to try it

• In the editor, ask the built-in assistant to build something and watch its avatar work.
• Or connect Claude, Claude Code or Cursor to ARClip over MCP and give it a task in your project: how to connect.
• To let agents hear and see, press Agents in the collaboration panel and turn on what you want.

More on the feature page: AI agents in your scene.