Agent Local Computer Control
Experimental local desktop control through a user-approved macOS companion with visible handoff and safety controls.
Tools
| Name | Description |
|---|---|
start_sessionStart a user-approved local computer control session. Returns a companion command for the user machine. | Start a user-approved local computer control session. Returns a companion command for the user machine. |
ensure_connectedPreflight local computer readiness and return recovery instructions if the companion is not connected. | Preflight local computer readiness and return recovery instructions if the companion is not connected. |
session_statusGet the current local computer session state, permissions, capabilities, and companion status. | Get the current local computer session state, permissions, capabilities, and companion status. |
snapshotCapture sanitized local target metadata. Use screenshot for visual observation; AX element refs are not exposed in v1. | Capture sanitized local target metadata. Use screenshot for visual observation; AX element refs are not exposed in v1. |
screenshotTake an agent-optimized screenshot of the selected local target. App/window sessions capture the selected window; display sessions require local confirmation. | Take an agent-optimized screenshot of the selected local target. App/window sessions capture the selected window; display sessions require local confirmation. |
move_mouseMove the cursor to bounded coordinates inside the selected local target without clicking. | Move the cursor to bounded coordinates inside the selected local target without clicking. |
clickClick bounded coordinates inside the selected local target. | Click bounded coordinates inside the selected local target. |
dragDrag the mouse along a bounded path of screenshot/display coordinates inside the selected local target. | Drag the mouse along a bounded path of screenshot/display coordinates inside the selected local target. |
typeType text into the focused field. Secure fields and terminal/admin targets are rejected by the local companion. | Type text into the focused field. Secure fields and terminal/admin targets are rejected by the local companion. |
press_keyPress a key or key chord in the selected local target. Supports Enter, Escape, Tab, Space, arrows, Home, End, PageUp, PageDown, Delete, function keys F1-F12, letters, digits, and common punctuation. | Press a key or key chord in the selected local target. Supports Enter, Escape, Tab, Space, arrows, Home, End, PageUp, PageDown, Delete, function keys F1-F12, letters, digits, and common punctuation. |
scrollScroll inside the selected local target. | Scroll inside the selected local target. |
waitWait for a short amount of time or for the local companion to settle after a UI change. | Wait for a short amount of time or for the local companion to settle after a UI change. |
run_stepsRun up to 50 deterministic desktop operations in one session round trip. Returns per-step ok/result/error entries and can include a final snapshot or screenshot. | Run up to 50 deterministic desktop operations in one session round trip. Returns per-step ok/result/error entries and can include a final snapshot or screenshot. |
request_humanHand control to the local user for login, MFA, sensitive prompts, or manual review. | Hand control to the local user for login, MFA, sensitive prompts, or manual review. |
resumeRequest agent control again. If the local user reclaimed control, local confirmation is required. | Request agent control again. If the local user reclaimed control, local confirmation is required. |
end_sessionEnd the local computer control session and disconnect the companion. | End the local computer control session and disconnect the companion. |
Quick Start
import { WeavzClient } from '@weavz-io/sdk'
const weavz = new WeavzClient({ apiKey: 'wvz_...' })
const result = await weavz.actions.execute('agent-local-computer-control', 'start_session', {
workspaceId: '550e8400-e29b-41d4-a716-446655440000',
integrationAlias: 'local_mac',
input: { /* ... */ },
})Related Integrations
Works well with
Frequently Asked Questions
How do I connect Agent Local Computer Control to my app?
Agent Local Computer Control does not require an external account connection. Install the SDK, add the integration to a workspace, then execute tools programmatically.
What tools does Agent Local Computer Control support?
Agent Local Computer Control supports 16 tools including Start Session, Ensure Connected, Session Status, Snapshot, Screenshot, and more.
Can I use Agent Local Computer Control with AI agents?
Yes. Add Agent Local Computer Control to an MCP server in Tool Mode or Code Mode. Code Mode provides 3 meta-tools with on-demand API discovery so agents can compose stateful workflows without loading every action schema up front.
Can Agent Local Computer Control be used in stateful agent workflows?
Yes. Agents can combine Agent Local Computer Control tools with Filesystem, State KV, Human Gates, input partials, and other workspace integrations while keeping execution scoped to the selected workspace or end user.
What authentication does Agent Local Computer Control use?
Agent Local Computer Control does not require external authentication.
How much does the Agent Local Computer Control integration cost?
The Agent Local Computer Control integration is included on all Weavz plans including the free tier with 20,000 actions per month. No credit card required.
Connect Agent Local Computer Control in minutes
Get 20,000 free action executions every month. No credit card required.