--- title: "Send input" method: POST path: "/computers/{id}/input" tags: ["Control"] --- # Send input `POST /computers/{id}/input` ## Request - Base URL: `https://app.mandala.computer/api/v1` - URL: `https://app.mandala.computer/api/v1/computers/{id}/input` - Auth: HTTP bearer Moves the pointer, clicks, drags, scrolls, types or presses keys. Three request shapes are accepted. The flat one below is this API’s own. The second is an Anthropic computer-use `tool_use.input` block posted verbatim — `coordinate: [x, y]`, `start_coordinate`, `scroll_direction`, `scroll_amount`, `duration`, and `text` doing double duty as either the string to type or the modifier keys to hold — so a model’s tool call can be forwarded without translation. The third is an OpenAI action with `type` instead of `action`: `click`, `double_click`, `drag`, `move`, `scroll`, `keypress`, `type`, `wait` or `screenshot`. Send one object or an array of 1–32 OpenAI action objects. Do not mix the two discriminators or dialect-specific fields. OpenAI pointer actions require both x and y, including zero. Click buttons are left, right, wheel (middle), back and forward. A drag path has 2–128 `{x,y}` points and holds one button through every waypoint. Coordinates are truncated to pixels, bounded to ±10,000,000, then clamped to the screen. OpenAI scroll_x/scroll_y are approximated as wheel notches: round each absolute distance divided by 100, with a minimum of one for a nonzero delta and zero for zero. Each axis is bounded to ±5000 pixels and their combined converted count to 50 notches; vertical then horizontal. Actual movement depends on the application. Optional keys (at most 16) are held around the entire OpenAI pointer gesture, including positioning, and released on failure. Arrow names, META/COMMAND and OPTION map to desktop keys. OpenAI wait lasts one second. OpenAI screenshot acknowledges a marker with ok:true; retrieve image bytes using GET /screenshot after the action(s). Duplicate or unused OpenAI fields, even zero-valued ones, are refused before auto-resume. The following alias/default rules apply to the two legacy action formats. A batch is fully validated before auto-resume or input. Combined limits are 400 typed Unicode characters, 50 wheel notches, 128 drag points and 10 one-second waits. The whole batch shares one input lane, one QMP connection when needed, and at most 75 seconds of work including cleanup, plus two five-second queues. QMP remains held during waits in a batch containing pointer or keyboard input; wait-only batches do not hold it. VNC and native guest applications are not serialized by this API lock. Execution stops at the first error. completed and results describe finished actions; failed_index is zero-based. The failing action may have partially applied. Inspect a fresh screenshot before retrying; never automatically replay the batch. Static validation/admission failures use the normal error response. A screenshot-only batch acknowledges markers without waking the computer. Optional context is read once after a successful batch. The proxy allows 93 seconds, including context and transport. Coordinates are absolute, in the screen space `resolution` reports. Keys can be held down for the length of a click, a scroll or a drag — ctrl-click, shift-click, shift-drag to extend a selection. Give them as `keys` (`["ctrl", "shift"]`), or as a chord string in `key` or `text` (`"ctrl+shift"`), once. On a click or a drag they are pressed before the pointer moves and released after the button. On a scroll with a coordinate the pointer moves FIRST, with no keys held, and the keys are then pressed for the wheel notches and released after them. Either way they are released when the action fails part way. A body is read strictly. A field this endpoint does not have, a field the action does not read (`keys` on a `move`, a `coordinate` on a `type`, `button` on a click), the held keys given in more than one field, and an unknown `action` each answer 400 naming what was wrong, before anything is sent to the computer and before a suspended one is resumed. A computer tool’s `screenshot` and `zoom` are not input: posted here, with or without `region`, they answer 400 with `code: not_an_input_action` and say where each is served. Take a screenshot from `GET /computers/{id}/screenshot`, and serve a zoom as a crop of the screenshot the model is looking at: `?capture=` set to that screenshot’s `X-GC-Capture` header and `?region=x0,y0,x1-x0,y1-y0` from the tool’s `[x0, y0, x1, y1]`, in that screenshot’s pixels. `computer_toolset_20260801` turns zoom on unless its `configs` turn it off, and a model may still call it, so a harness forwarding tool calls verbatim should route `zoom` itself. It should also spend a `key` call’s `repeat` as one input per press (`repeat` is not a field here), and split a `wait` longer than 30 seconds. A field at its zero value — `""`, `[]`, `0` — counts as not sent, except `x` and `y`, where `0` is a position: `{"action":"key","key":"ctrl+c","x":0,"y":0}` is a 400 (`key does not take x`). Leave them out, or send `null`. `amount` (or `scroll_amount`) is 1 to 50 notches, 3 when left out or not above 0; more than 50 is a 400. `duration` is at most 30 seconds, 1 second when left out or not above 0; longer is a 400. Those two, a scroll direction other than `up`, `down`, `left` or `right`, and a key name this endpoint does not know — in a chord or held during a pointer action — are checked with the rest of the body: a 400 before a suspended computer is resumed, and the same 400 from a stopped one. `type` requires non-empty valid Unicode text, at most 400 characters. Plain ASCII is typed as physical key events, about 12 ms a character (each key held 8 ms, then 4 ms before the next). Text containing any other character is typed WHOLE, in request order, by one guest helper: its ASCII at the same physical key positions and shift levels, and every other character by GTK Unicode composition. This works in supported Chromium GTK3 and Xfce Terminal/VTE configurations on Linux X11, including accents, CJK, emoji and combining sequences, without changing the clipboard or keyboard mapping. Tab and LF are keys; CRLF sends one Return. Bare CR and other C0/C1/DEL controls are refused. Malformed or unsupported text is checked before auto-resume or any keys. Unicode typing requires XTEST, XRes process identification, Python Xlib, GTK3, timeout, a neutral supported US keyboard mapping and an effective gtk-im-context-simple input context. A no-key preflight verifies the supported application/configuration before even a mixed request’s ASCII prefix. Firefox, other applications, GTK4, custom input modules, disabled GTK IME, Windows and native Wayland Unicode typing are unsupported. Guest configuration is not changed. The response reports `mechanism: physical | unicode | mixed`. Use exclusive desktop control and verify the resulting text: delivery does not prove application acceptance. Runtime failure or cancellation can leave partial text or composition state. Cancellation waits for the bounded in-flight helper; cleanup never sends Escape into uncertain focus. Unicode requests share 75 seconds for all work including cleanup, with a 95-second proxy allowance for queues and transport; inspect before retrying and never automatically replay. For fast text insertion use `{"action":"paste","text":"Café — 東京 😀"}`. This writes the Linux desktop clipboard and sends Ctrl+V. Use `key: "ctrl+shift+v"` for terminals that require it, Shift+Insert is refused because it can read the separate primary selection. Text must be non-empty UTF-8 without NUL, at most 8192 bytes. Windows and guests without working clipboard support are refused. Paste replaces the clipboard and leaves it in place. API input/clipboard writes are serialized through the shortcut, but native guest applications and VNC clients can still change it. Use exclusive desktop control and wait for the target to consume the paste before changing the clipboard again. A 200 means the clipboard write and shortcut were delivered, not that the application accepted or inserted the text. Check the target afterward; paste-disabled fields may ignore it. An interrupted request can have side effects: inspect before retrying, and do not automatically fall back to typing or replay it. `count` repeats a `left_click`, `right_click` or `middle_click` up to 10 times, for the applications that read a click count past three. Add `?context=1` to have the answer carry the desktop’s windows and the focused one as they stand just after the action — on any action, `wait` included. A query parameter other than `context`, one given twice or empty, and a `context` other than `1`, `true`, `0` or `false` each answer 400 before anything is sent to the computer. Requires the `member` role or higher. ## Path parameters - `id` string, required ## Query parameters - `context` '1' | 'true' | '0' | 'false' ## Request body - union - object - `action` string, required: One of `move`/`mouse_move`, `left_click`, `right_click`, `middle_click`, `double_click`, `triple_click`, `left_mouse_down`, `left_mouse_up`, `left_click_drag`, `scroll`, `type`, `paste`, `key`, `hold_key`, `cursor_position`, `wait`. - `x` integer: Absolute. Omit to act where the pointer already is, except on `move`/`mouse_move` and `left_click_drag` (whose destination it is): those need a coordinate, and one sent without it is refused with a 400 before a suspended computer is woken. `0` is a position, not an omission — except on `scroll`, where a flat `x: 0, y: 0` scrolls where the pointer is, and `coordinate: [0, 0]` is needed to scroll at the top-left corner. - `y` integer - `coordinate` number[]: `[x, y]`, as an alternative to x/y. - `start_coordinate` number[]: Where a `left_click_drag` starts. - `text` string: The string for `type` (non-empty, at most 400 Unicode characters) or `paste` (1–8192 UTF-8 bytes, no NUL). For `key` and `hold_key`, the chord. On a click, a scroll or a drag, the keys to hold down during it. - `key` string: A chord such as `ctrl+c`, for `key` and `hold_key`. On a click, a scroll or a drag, the keys to hold down during it. For `paste`: ctrl+v (default) or ctrl+shift+v. - `keys` string[]: The chord as separate keys, for `key`, `hold_key` and `paste`. On a click, a scroll or a drag, the keys to hold down during it. - `button` string: Scroll direction: `up`, `down`, `left`, `right`. - `scroll_direction` string: The computer-use spelling of `button`. - `amount` integer: Scroll notches, 1 to 50; 3 when left out. More than 50 is a 400. - `scroll_amount` integer: The computer-use spelling of `amount`. - `duration` number: Seconds, for `wait` and `hold_key`: at most 30, and 1 when left out. Longer is a 400. - `count` integer: How many times `left_click`, `right_click` or `middle_click` presses its button, 1 to 10; 1 when left out. The presses are paced as `double_click`’s are, so the guest sees one multi-click. More than 10, or below 1, is a 400, and so is `count` on `double_click` or `triple_click`, which are two and three clicks by name. - OpenAIInput - `type` 'click' | 'double_click' | 'drag' | 'move' | 'scroll' | 'keypress' | 'type' | 'wait' | 'screenshot', required: OpenAI action discriminator; mutually exclusive with action. - `x` number: Required for pointer actions except drag. Absolute pixels in the screen space `resolution` reports; rescale when a screenshot’s `X-GC-Capture-Size` differs. - `y` number: Required for pointer actions except drag. Absolute pixels in the screen space `resolution` reports; rescale when a screenshot’s `X-GC-Capture-Size` differs. - `button` 'left' | 'right' | 'wheel' | 'back' | 'forward': Required for click only. - `keys` string[], nullable: A keypress chord, or keys held during a pointer gesture. - `text` string: Required for type; existing Unicode typing limits apply. - `path` object[]: Required for drag: ordered {x,y} waypoints. - `x` number, required - `y` number, required - `scroll_x` number: Required for scroll: horizontal pixel delta, ±5000. - `scroll_y` number: Required for scroll: vertical pixel delta, ±5000. - OpenAIInput[] - `type` 'click' | 'double_click' | 'drag' | 'move' | 'scroll' | 'keypress' | 'type' | 'wait' | 'screenshot', required: OpenAI action discriminator; mutually exclusive with action. - `x` number: Required for pointer actions except drag. Absolute pixels in the screen space `resolution` reports; rescale when a screenshot’s `X-GC-Capture-Size` differs. - `y` number: Required for pointer actions except drag. Absolute pixels in the screen space `resolution` reports; rescale when a screenshot’s `X-GC-Capture-Size` differs. - `button` 'left' | 'right' | 'wheel' | 'back' | 'forward': Required for click only. - `keys` string[], nullable: A keypress chord, or keys held during a pointer gesture. - `text` string: Required for type; existing Unicode typing limits apply. - `path` object[]: Required for drag: ordered {x,y} waypoints. - `x` number, required - `y` number, required - `scroll_x` number: Required for scroll: horizontal pixel delta, ±5000. - `scroll_y` number: Required for scroll: vertical pixel delta, ±5000. ## Response `200` Delivered. `type` reports its mechanism; `cursor_position` answers where the pointer is and whether that is known. Verify application acceptance. - union - InputResult - `ok` boolean, required: The action was carried out. - `mechanism` 'physical' | 'unicode' | 'mixed': Present on successful `type`: `physical` for plain ASCII typed as US-layout key events; `unicode` for text with no ASCII, typed by GTK Unicode composition; `mixed` for both, typed in request order by the same guest helper, which pressed the ASCII key positions itself. Confirms dispatch; verify that the target accepted the exact text. - `x` integer: Where the pointer is, for `cursor_position`. Absent on every other action. - `y` integer: The same. - `known` boolean: Read this BEFORE `x` and `y`. QEMU’s tablet is an absolute pointing device: it accepts coordinates and reports none back, so the only position this platform can tell you about is one it put the pointer at itself. `false` means nothing has — a guest that has just booted, or been restored, has its pointer wherever it left it — and `x` and `y` are then `0`, which is a corner the pointer is probably not in rather than a reading. Absent on every action but `cursor_position`. - `context` InputContext - `windows` Window[], required: The windows `GET /computers/{id}/windows` lists by default — applications, without the desktop’s own panels and wallpaper — read once, straight after the action. There is no settle wait, so a window still opening when the read ran is not listed yet. `[]` for an empty desktop. - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `focused` Window, required - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `dom` PageContext - `url` string, required: The page’s address, at most 2048 characters. - `title` string, required: The page’s title, at most 300 characters. - `elements` PageElement[], required: The interactive elements visible in the viewport — links, buttons, form fields, `summary`, elements with a button, link, tab, menu item, checkbox, radio or textbox `role`, and `contenteditable` or `onclick` elements — that are not hidden or covered at their centre, in document order. At most 150. The top document only: nothing inside an iframe or a shadow root. `[]` when none is visible. - `tag` string, required: The element’s lower-case tag name: `a`, `button`, `input`. - `role` string, required: Its `role` attribute, or `""`. - `name` string, required: The first of its `aria-label`, `title`, `placeholder`, `alt` and `name`, or `""`. At most 100 characters. - `text` string, required: What it says: its text, or a field’s value. A password field’s value is never read, and a checkbox or radio has none. At most 100 characters. - `href` string: A link’s resolved target, at most 2048 characters. Present on links only. - `x` integer, required: The left edge of the element’s visible part, in screen pixels — the coordinates input actions take. Click at `x + width/2`, `y + height/2`. Placed from the Chromium window’s position, with the page filling the window below its toolbar. A page narrowed by a side panel or DevTools docked beside it is refused (no `dom`, and `context_error` says why); DevTools docked at the bottom is not detected and moves the page, not these. - `y` integer, required: The top edge of the visible part, in screen pixels. - `width` integer, required: The visible part’s width, in screen pixels; at least 1. - `height` integer, required: The visible part’s height, in screen pixels; at least 1. - `truncated` boolean, required: The page had more interactive elements than `elements` holds. - `context_error` string: What `?context=1` could not read. With no `context`, it says why the windows could not be listed: the sentence `GET /computers/{id}/windows` would have answered (a Windows guest, no desktop session, a guest agent not answering), or that the guest did not list its windows within 3 seconds. Beside a `context`, it says why that context has no `dom`: no window has focus, the focused window is not Chromium, Chromium is not listening on its debugging port, the focused Chromium is not the desktop’s own profile or no tab of it can be tied to the focused window, its page does not fill its window (a side panel or DevTools docked beside it), the computer’s image predates page elements, or the page was not described within the same 3 seconds. Either way the action itself was carried out — `ok` is still `true` — so do not send it again; list the windows or take a screenshot instead. - InputBatchResult - `ok` boolean, required: Whether every action completed. - `completed` integer, required: Number of completed actions, in order. Not a rollback guarantee. - `results` InputResult[], required: One result for each completed action. - `ok` boolean, required: The action was carried out. - `mechanism` 'physical' | 'unicode' | 'mixed': Present on successful `type`: `physical` for plain ASCII typed as US-layout key events; `unicode` for text with no ASCII, typed by GTK Unicode composition; `mixed` for both, typed in request order by the same guest helper, which pressed the ASCII key positions itself. Confirms dispatch; verify that the target accepted the exact text. - `x` integer: Where the pointer is, for `cursor_position`. Absent on every other action. - `y` integer: The same. - `known` boolean: Read this BEFORE `x` and `y`. QEMU’s tablet is an absolute pointing device: it accepts coordinates and reports none back, so the only position this platform can tell you about is one it put the pointer at itself. `false` means nothing has — a guest that has just booted, or been restored, has its pointer wherever it left it — and `x` and `y` are then `0`, which is a corner the pointer is probably not in rather than a reading. Absent on every action but `cursor_position`. - `context` InputContext - `windows` Window[], required: The windows `GET /computers/{id}/windows` lists by default — applications, without the desktop’s own panels and wallpaper — read once, straight after the action. There is no settle wait, so a window still opening when the read ran is not listed yet. `[]` for an empty desktop. - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `focused` Window, required - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `dom` PageContext - `url` string, required: The page’s address, at most 2048 characters. - `title` string, required: The page’s title, at most 300 characters. - `elements` PageElement[], required: The interactive elements visible in the viewport — links, buttons, form fields, `summary`, elements with a button, link, tab, menu item, checkbox, radio or textbox `role`, and `contenteditable` or `onclick` elements — that are not hidden or covered at their centre, in document order. At most 150. The top document only: nothing inside an iframe or a shadow root. `[]` when none is visible. - `tag` string, required: The element’s lower-case tag name: `a`, `button`, `input`. - `role` string, required: Its `role` attribute, or `""`. - `name` string, required: The first of its `aria-label`, `title`, `placeholder`, `alt` and `name`, or `""`. At most 100 characters. - `text` string, required: What it says: its text, or a field’s value. A password field’s value is never read, and a checkbox or radio has none. At most 100 characters. - `href` string: A link’s resolved target, at most 2048 characters. Present on links only. - `x` integer, required: The left edge of the element’s visible part, in screen pixels — the coordinates input actions take. Click at `x + width/2`, `y + height/2`. Placed from the Chromium window’s position, with the page filling the window below its toolbar. A page narrowed by a side panel or DevTools docked beside it is refused (no `dom`, and `context_error` says why); DevTools docked at the bottom is not detected and moves the page, not these. - `y` integer, required: The top edge of the visible part, in screen pixels. - `width` integer, required: The visible part’s width, in screen pixels; at least 1. - `height` integer, required: The visible part’s height, in screen pixels; at least 1. - `truncated` boolean, required: The page had more interactive elements than `elements` holds. - `context_error` string: What `?context=1` could not read. With no `context`, it says why the windows could not be listed: the sentence `GET /computers/{id}/windows` would have answered (a Windows guest, no desktop session, a guest agent not answering), or that the guest did not list its windows within 3 seconds. Beside a `context`, it says why that context has no `dom`: no window has focus, the focused window is not Chromium, Chromium is not listening on its debugging port, the focused Chromium is not the desktop’s own profile or no tab of it can be tied to the focused window, its page does not fill its window (a side panel or DevTools docked beside it), the computer’s image predates page elements, or the page was not described within the same 3 seconds. Either way the action itself was carried out — `ok` is still `true` — so do not send it again; list the windows or take a screenshot instead. - `failed_index` integer: Zero-based index of the failed action, only on execution failure. That action may have partially applied. - `error` string: Execution failure; inspect the desktop before retrying. - `context` InputContext - `windows` Window[], required: The windows `GET /computers/{id}/windows` lists by default — applications, without the desktop’s own panels and wallpaper — read once, straight after the action. There is no settle wait, so a window still opening when the read ran is not listed yet. `[]` for an empty desktop. - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `focused` Window, required - `id` string, required: The window id, for `POST /computers/{id}/windows/{window}`. - `title` string, required - `class` string, required - `type` string, required: The window type. On an X11 desktop this is the EWMH `_NET_WM_WINDOW_TYPE`, lower-cased with the `_NET_WM_WINDOW_TYPE_` prefix dropped: `normal`, `dialog`, `dock`, `desktop`. A window that declares only a VENDOR type is reported as that atom, lower-cased and prefix intact, so do not assume the value comes from that list. It is EMPTY when the window declares no type at all, which is legal — EWMH treats an untyped managed window as normal, and this reports what the window said rather than filling it in. The listing has one exception to that, below. On a WAYLAND desktop it is `normal` for every window, and nothing below applies. An EWMH window type is an X11 protocol; a Wayland toplevel does not declare one, so there is nothing to read and nothing is inferred from anything else. `type` is also the field where `GET /computers/{id}/windows` and the event stream can report a window differently. Two ways, and the same shape both times: the LISTING interprets what it reads, the STREAM reports what the window literally declared. Both are X11 only: 1. The listing reports `dialog` for an untyped window that names another as its parent (`WM_TRANSIENT_FOR`) — a toolkit too old to set a type still marks its modals that way, and “which of these is the dialog in front” is one of the questions the listing exists to answer. The guest watcher does not read that property, so the same legacy modal is `dialog` in the listing and empty on the stream. 2. A window may declare SEVERAL types — a vendor one first, a standard one after it as the fallback. The listing walks that list and reports the standard atom; the watcher forwards only the first. So a window typed `_COMPIZ_WINDOW_TYPE, _NET_WM_WINDOW_TYPE_NORMAL` is `normal` in the listing and `_compiz_window_type` on the stream. This applies to everything the watcher describes: `window.opened`, `window.focused`, and the `windows` that `hello` carries, which are one picture. The other fields of this object mean the same thing on both paths, so a difference in one of THOSE is only ever a difference in when each was read. If the type matters to you, ask the listing — it is the authority on the present here as it is elsewhere. - `pid` integer: The process inside the guest that owns this window, where the window says so. ABSENT rather than `0` when it does not: a guest is free to advertise `_NET_WM_PID` 0, and reporting that as “no pid” would be inventing an answer. It does not identify the window. An application that keeps one process for several windows — `xfce4-terminal` is one — reports the same pid on all of them, so killing this pid can take windows you never asked about. - `x` integer, required - `y` integer, required - `width` integer, required - `height` integer, required - `focused` boolean, required - `visible` boolean, required: False for a minimised window, and this is the only way to tell one from a window on the screen. A minimised window stays on the list and keeps the coordinates it had, so an agent that clicks at them is clicking at whatever is actually there. Nothing else in the row says so — the title, the class, the geometry and the `focused` flag all read as ordinary. - `dom` PageContext - `url` string, required: The page’s address, at most 2048 characters. - `title` string, required: The page’s title, at most 300 characters. - `elements` PageElement[], required: The interactive elements visible in the viewport — links, buttons, form fields, `summary`, elements with a button, link, tab, menu item, checkbox, radio or textbox `role`, and `contenteditable` or `onclick` elements — that are not hidden or covered at their centre, in document order. At most 150. The top document only: nothing inside an iframe or a shadow root. `[]` when none is visible. - `tag` string, required: The element’s lower-case tag name: `a`, `button`, `input`. - `role` string, required: Its `role` attribute, or `""`. - `name` string, required: The first of its `aria-label`, `title`, `placeholder`, `alt` and `name`, or `""`. At most 100 characters. - `text` string, required: What it says: its text, or a field’s value. A password field’s value is never read, and a checkbox or radio has none. At most 100 characters. - `href` string: A link’s resolved target, at most 2048 characters. Present on links only. - `x` integer, required: The left edge of the element’s visible part, in screen pixels — the coordinates input actions take. Click at `x + width/2`, `y + height/2`. Placed from the Chromium window’s position, with the page filling the window below its toolbar. A page narrowed by a side panel or DevTools docked beside it is refused (no `dom`, and `context_error` says why); DevTools docked at the bottom is not detected and moves the page, not these. - `y` integer, required: The top edge of the visible part, in screen pixels. - `width` integer, required: The visible part’s width, in screen pixels; at least 1. - `height` integer, required: The visible part’s height, in screen pixels; at least 1. - `truncated` boolean, required: The page had more interactive elements than `elements` holds. - `context_error` string: Optional context-read failure after an otherwise successful batch. ## Other responses - `400`: The request was malformed — a bad resolution string, a missing `command`, a `{pid}` or `{window}` that is not one. It is also what a well-formed request gets when the thing it names cannot be acted on and never will be — a restore onto a computer that is gone, for instance. Either way it is final: the message says what is wrong, and sending it again unchanged answers the same. That is the whole difference from a `503`, which means ask again shortly. - `401`: Platform authentication required or refused: reason is missing, invalid, or revoked, with WWW-Authenticate: Bearer (invalid_token for invalid/revoked). - `402`: Your plan refused this: a quota, or a payment that has not gone through. On an agent run it can instead be `Insufficient AI credits` with `top_up_url` (a run funded by the account’s AI credits), or the model provider’s own billing refusal of your `X-Model-Key`, relayed. - `403`: The credential is recognized but its holder lacks the required role or membership, or the holder/account is suspended, or the holder has not verified their email address. An ordinary denial for too low a role may carry no `reason`. The email one carries none either, and its message begins `Verify your email before`: it answers a create, a clone of a computer or a snapshot, a new API key and a new SSH key, and is cleared by verifying the address in the dashboard. `reason: "revoked"` marks authority that is gone: a suspended account or a lost membership (refused as the request arrives, or if it happens while the request is in flight), or a role that was lowered while the request was in flight. The message identifies the refusal; replacing a valid key alone does not restore authority, and sending the same call again does not help. - `404`: No such resource in your scope — a computer, snapshot, secret, webhook, SSH key, API key, workspace, operation and so on — or no such endpoint; a confirmed missing guest file is no such file in the guest. - `405`: This public path exists but does not accept that method. Allow lists only its public methods, with HEAD for GET and metadata OPTIONS. - `409`: Refused for the state something is in — resizing a computer that is running, running a command on one that is stopped, taking a snapshot while another capture of it is running. WHETHER RETRYING HELPS IS IN THE BODY, NOT THE STATUS, and this is the one thing to take from this entry. Some of these describe a state that is passing: a guest agent still inside its boot window, a guest agent busy with another call, a move already running on the account. Those clear on their own and the same request works a moment later. The rest describe a DECISION about the request you sent — the size does not fit, the computer is the wrong one for this, the saved session cannot travel — and no amount of retrying turns one of those into a yes. A client that treats 409 as uniformly transient loops forever on half of them. Where the platform can say which it is, it sends `reason` beside `error` — one word, meant to be switched on rather than read: `contention` and `starting` clear on their own, `unavailable` means the computer is not running and only starting it helps, `running` means the computer is running and only stopping it helps (a resize), `unsupported` means waiting will not change the answer — this computer cannot do it at all, or it needs another step first, which the message names (a stop, or removing its secret bindings, for example) — `exists` means a create-only upload found its path taken, `name_taken` means a secret of that name is already in that scope, `stale_revision` means a secret changed since the `revision_id` you sent was read (or a computer’s bound list since the `version` you sent), and `stale_capture` means the screenshot capture you named is no longer held: a newer one replaced it, or its host’s daemon restarted or the computer moved. `starting` covers the two minutes after a start or a restart, and after a reboot from inside the guest on a computer last cold-started recently enough to report one (a stop and a start gives an older one that); a guest agent that is still silent after that is a `502`. Match on that and never on the sentence, which is prose and is rewritten. **Absent means no classification was given** — treat it, and any word you do not recognise, as no answer, because not every refusal here has one yet. It is what the guest routes answer: the clipboard on a stopped computer used to be a 409 indistinguishable from a clipboard whose selection was claimed for an instant, and a blanket retry loop spun against the first until its deadline. Every 409 that clears on its own carries `contention` or `starting`: a host busy with another build or with no room for one, an operation already running on the same computer or snapshot (a capture, a copy, a delete, a restore or clone reading it, another agent run), a lock another request holds, a move or a delivery settling. So a 409 with no word, or with any other word, is not one that waiting fixes: read the sentence for the step it names. The ones worth knowing by name, because each has a next step that is not "wait": A resize past what the computer’s host can run carries `move`, an object rather than a message: `{"required":true,"possible":true}` means another host in this region could run it and the computer has to be moved there first — `POST /computers/{id}/move` with the same sizing fields is how you agree to that, and it is a separate call because relocating a computer is not something a resize should do to you quietly. `possible:false` means nowhere in the region can, and the size is the thing to change. The move itself then refuses for the things it re-decides at the moment it runs, and only one of those is worth waiting on: a computer that is running or suspended has to be stopped or resumed-and-stopped by you, a size that fits where it already is wants an ordinary resize instead, and a region with nowhere to put it wants a smaller size. `Another computer on this account is being moved right now` is the one that clears by itself, and it carries `reason: "contention"` — one move runs per account, and `GET /moves` says which. And the terminal socket refuses twice over: `reason: "unavailable"` with `resume_required: true` on a suspended computer, which clears when you start it and not before, and `reason: "unsupported"` on a running computer whose hardware carries no terminal channel — one last started before interactive terminals existed — which needs a stop and a start. Not a restart, which resets the same machine, and not a retry, which never comes good. - `413`: What you sent, or asked for, is past a size limit. A JSON request body is capped at 1 MiB unless the route states its own limit: `POST /computers/{id}/exec` takes up to 4 MiB, and some routes take less; guest file transfers, template documents and the clipboard have limits of their own. The message names the limit that applied. - `429`: You are asking faster than your plan allows. `Retry-After` says how many seconds to wait, and every metered response — anything past authentication and the re-check that your credential’s authority still holds — carries `RateLimit-Remaining`, so you do not have to be refused to find out. A `401`, a `403` from that re-check, an `OPTIONS` answer and an unexpected `500` carry none. The budget is spent in proportion to what a call costs rather than per request. What a call costs, in budget units, by kind (each operation’s exact figure is its `x-rate-cost` in the OpenAPI document): 1 to read one thing; 5 for a listing that spans the fleet, and 5 for a call into a running guest such as a screenshot or an exec; 10 for a power change and 10 to move file bytes; 25 for `GET /usage`, `GET /credits`, creating a public URL, and creating, rotating or testing a webhook; 50 to capture a snapshot; 100 to create, clone or restore a computer, and 100 to start an agent run, whose actions are then charged as they happen. The account-wide budget, as what an idle caller may spend at once and what refills each minute: No plan 1200 at once and 1200 a minute; Solo 4800 at once and 4800 a minute; Studio 12000 at once and 12000 a minute; Swarm 21600 at once and 21600 a minute. A single API key gets half of either. A `401` for a missing or unusable credential costs nothing: it is refused before anything is metered. A few refusals at the door cost one unit: a `403` because the role is too low or the route refuses an API key confined to a workspace, a `404` for a path that matches nothing, a `405`, and an `Idempotency-Key` answered from its stored result or still in progress. Everything else is charged in full when it is admitted, before it is carried out, so a refusal after that has spent the whole cost: a `400`, `402` or `409` from the route, a `403` its handler sends (on most routes, a suspended account included), and a model provider’s `401` on an agent run. The budget belongs to the account and is shared across everything on it, and any single API key may spend at most half of it. `RateLimit-Limit` is whichever of those two is binding on the request that carried it — usually the per-key half, and the account-wide figure when other keys have already spent more of it than this one has. All three `RateLimit-*` headers describe the same one, so `limit` minus `remaining` is what has been spent against it. A few `429`s are not about this budget, say so in their message, and can carry no `Retry-After`: a model provider’s rate limit relayed on an agent run. The cap on forks of one snapshot waiting for their secrets sends its own bounded expiry-based `Retry-After` with `reason: "contention"` (see `POST /snapshots/{id}/clone`). - `500`: The application could not finish this: an unexpected exception, a build that broke, or a storage fault. Application refusals carry the request correlation ID; unexpected internal details are not returned. - `502`: The agent inside the guest is not answering, or it rejected or malformed the private command protocol for this feature. The computer is up; the upstream agent interaction failed. This is not malformed caller input and carries no retry classification. - `503`: Something this depends on — a hypervisor, a store, or a host’s capacity — could not answer or take this right now. A read can be retried shortly. A change answered `503` may or may not have happened — a failure after the request was sent is answered the same way, with or without a warning in the message — so read the current state before retrying anything that is not safe to repeat, such as a create or a command. On the collection reads, `allow_partial=1` accepts a knowingly short answer instead. Also, on a suspended computer this request would resume, a sealed session the platform could not open or carry just then — the service that holds its key did not answer, or the host failed moving the saved session. Nothing ran in the guest. Read the computer’s status, then send it again; it usually clears in a moment. --- [API](https://skmtc.dev/mandala/apis/mandala-computer-api.md) · [All operations](https://skmtc.dev/mandala/apis/mandala-computer-api/llms.txt) · [OpenAPI document](https://skmtc.dev/mandala/apis/mandala-computer-api/revisions/2afbb87fa703?raw)