Skip to main content
An agent works in a loop: observe the screen, act, then read the new screen that the action returns. This page lists every action. The same actions are available as MCP tools and through POST /v1/phones/invoke. The parameter names on this page are the REST names. MCP names differ in a few places and are listed on the MCP page.

Summary

Actions on one phone run one at a time, in the order they arrive.

Common parameters

Device ids

A device id is the Mac’s name, a slash, and the phone’s id on that Mac:
Always copy ids from List phones.

Targets

tap, long_press and swipe take a target. A target is one of: Element numbers are valid only for the latest observation. After any action the numbers expire. Using an old number fails with The screen changed since the last observation. Observe again before targeting elements by id.

List phones

devices returns the phones that are online across all your Macs.

Observe

observe takes a screenshot and finds the elements on screen.
When no elements are found on a screen, elements is empty and the agent taps by x and y.

Tap

A tap on an element whose label contains a word like Post, Send or Buy waits for your approval. The request stays open until you decide. See Approvals and safety. Result with observe: true (default): an observation, same shape as Observe, taken after the screen has settled. Result with observe: false:
All actions below that accept observe return results in the same two shapes.

Long press

A long press is checked for approval the same way as a tap.

Swipe

Drags from one target to another. For plain scrolling, use Scroll.

Scroll


Type

Types into the text field that has focus. Tap the field first. A line break in the text presses Enter.
On Android, text with accents, non-Latin letters or emoji needs a helper keyboard app on the phone. See Connect phones.

Press a key

A key the phone does not support fails with a message that lists the supported keys.

Open an app

On Android, an app that is not installed fails with Could not open <appId>. Is it installed? Use list_apps to check.

List apps

On Android the list holds the apps you installed, not system apps, and name is the same as id. On iPhone name is the app’s display name.

Push media

Copies an image or video from a URL, or from a file on the Mac, into the phone’s gallery. Apps can then pick it, for example when you create a post.
Pass url or path.
  • Works on Android. On iPhone it fails, because iPhones don’t accept pushed media.
  • A download larger than 512 MB is rejected.
  • The file keeps its name. Use a link that ends in a real file name, such as clip.mp4.
Links to generated media are often short-lived. Fetch a fresh link right before you push it.