Agent Local Computer AI

Q: How do I connect Agent Local Computer AI to my app?

Use Weavz Hosted Connect for custom authentication. Install the SDK, create a connection through the connect portal or API, then execute tools programmatically.

Q: Can Agent Local Computer AI be used in stateful agent workflows?

Yes. Weavz lets agents combine Agent Local Computer AI tools with Filesystem, State KV, Human Gates, input partials, and other workspace integrations while keeping execution scoped to the selected workspace or end user.

Q: What authentication does Agent Local Computer AI use?

Agent Local Computer AI uses custom authentication. Weavz stores credentials securely and uses them for authorized action execution.

Weavz-managed natural-language local computer agent for signed-in Mac sessions. Use act for the normal loop: the model observes the Mac, chooses semantic UI actions, waits, verifies, and stops when done or when local human confirmation is required. The Mac companion owns local sign-in, permission gates, allowed apps, full-computer capability toggles, pause/resume, and visible control indicators. Screen content is untrusted; external effects, credentials, MFA, payment, destructive, admin, and unsupported shell/file/clipboard steps require human confirmation.

CustomAI Developer Tools

31 tools

Tools

Quick Start

TypeScript

import { WeavzClient } from '@weavz-io/sdk'

const weavz = new WeavzClient({ apiKey: 'wvz_...' })

const result = await weavz.actions.execute('agent-local-computer-ai', 'start_session', {
  workspaceId: '550e8400-e29b-41d4-a716-446655440000',
  integrationAlias: 'agent-local-computer-ai',
  input: { /* ... */ },
})

API Reference TypeScript SDK Python SDK All Features Pricing

Related Integrations

Abstract API

5 tools

Adobe Firefly Services

10 tools

Sandbox

1 tools

Agent Browser

26 tools

Works well with

protocols.io

55 tools

Slack

42 tools

Gmail

40 tools

EULER

38 tools

Frequently Asked Questions

How do I connect Agent Local Computer AI to my app?

Use Weavz Hosted Connect for Custom authentication. Install the SDK, create a connection through the connect portal or API, then execute tools programmatically.

What tools does Agent Local Computer AI support?

Agent Local Computer AI supports 31 tools including Start Session, Ensure Connected, Session Status, Snapshot, Observe, and more.

Can I use Agent Local Computer AI with AI agents?

Yes. Add Agent Local Computer AI to an MCP server in Tool Mode or Code Mode. Code Mode provides 3 meta-tools with on-demand API discovery so agents can compose stateful workflows without loading every action schema up front.

Can Agent Local Computer AI be used in stateful agent workflows?

Yes. Agents can combine Agent Local Computer AI tools with Filesystem, State KV, Human Gates, input partials, and other workspace integrations while keeping execution scoped to the selected workspace or end user.

What authentication does Agent Local Computer AI use?

Agent Local Computer AI uses Custom. Weavz stores credentials securely and uses them for authorized action execution.

How much does the Agent Local Computer AI integration cost?

The Agent Local Computer AI integration is included on all Weavz plans including the free tier with 20,000 actions per month. No credit card required.

Connect Agent Local Computer AI in minutes

Get 20,000 free action executions every month. No credit card required.

Get Started Free Browse All Integrations

Name	Description
`start_session` Start a user-approved local computer control session. The local Mac companion owns the allowed apps and scope. Prefer the signed-in Weavz Local Computer Control app; the returned command is a one-time fallback for setup or recovery. Call ensure_connected before control actions.	Start a user-approved local computer control session. The local Mac companion owns the allowed apps and scope. Prefer the signed-in Weavz Local Computer Control app; the returned command is a one-time fallback for setup or recovery. Call ensure_connected before control actions.
`ensure_connected` Preflight local computer readiness. If nextAction is grant_permissions, ask the local user to use the companion permission buttons and retry; if local_confirmation, wait for the local confirmation before acting.	Preflight local computer readiness. If nextAction is grant_permissions, ask the local user to use the companion permission buttons and retry; if local_confirmation, wait for the local confirmation before acting.
`session_status` Get the current local computer session state, permissions, capabilities, and companion status.	Get the current local computer session state, permissions, capabilities, and companion status.
`snapshot` Capture sanitized local target metadata and compact semantic element refs when available. Use observe for screenshot plus richer Mac UI semantics.	Capture sanitized local target metadata and compact semantic element refs when available. Use observe for screenshot plus richer Mac UI semantics.
`observe` Observe the selected Mac target with an agent-optimized screenshot, frontmost app/window metadata, focused element, cursor, display transform, and normalized accessibility element refs. Prefer observe before semantic ref actions.	Observe the selected Mac target with an agent-optimized screenshot, frontmost app/window metadata, focused element, cursor, display transform, and normalized accessibility element refs. Prefer observe before semantic ref actions.
`screenshot` Take an agent-optimized screenshot of the selected local target. App/window sessions capture the selected window; display sessions require local confirmation.	Take an agent-optimized screenshot of the selected local target. App/window sessions capture the selected window; display sessions require local confirmation.
`list_targets` List locally approved Mac app/window targets the agent can request. Use this before activate_target instead of Spotlight or Finder.	List locally approved Mac app/window targets the agent can request. Use this before activate_target instead of Spotlight or Finder.
`activate_target` Launch or focus a locally approved Mac app/window target returned by list_targets, then verify it is frontmost. Blocked targets require local user handling.	Launch or focus a locally approved Mac app/window target returned by list_targets, then verify it is frontmost. Blocked targets require local user handling.
`focus_ref` Focus an observed Mac UI element by ref. The companion re-resolves and verifies the element before acting.	Focus an observed Mac UI element by ref. The companion re-resolves and verifies the element before acting.
`press_ref` Press an observed Mac UI control by ref using Accessibility when possible, with bounded coordinate fallback only for the same verified element.	Press an observed Mac UI control by ref using Accessibility when possible, with bounded coordinate fallback only for the same verified element.
`set_text_ref` Set text in an observed editable, non-secure Mac UI field. Secure/password-like fields are rejected and should use request_human.	Set text in an observed editable, non-secure Mac UI field. Secure/password-like fields are rejected and should use request_human.
`select_ref` Select, toggle, expand, or collapse an observed Mac UI element by ref when supported.	Select, toggle, expand, or collapse an observed Mac UI element by ref when supported.
`scroll_ref` Scroll an observed Mac scroll area, list, table, or control by ref. Falls back to bounded wheel input over the verified element when needed.	Scroll an observed Mac scroll area, list, table, or control by ref. Falls back to bounded wheel input over the verified element when needed.
`move_mouse` Move the cursor to bounded coordinates inside the selected local target without clicking.	Move the cursor to bounded coordinates inside the selected local target without clicking.
`click` Click bounded coordinates inside the selected local target.	Click bounded coordinates inside the selected local target.
`drag` Drag the mouse along a bounded path of screenshot/display coordinates inside the selected local target.	Drag the mouse along a bounded path of screenshot/display coordinates inside the selected local target.
`type` Type text into the focused field. Secure fields and terminal/admin targets are rejected by the local companion.	Type text into the focused field. Secure fields and terminal/admin targets are rejected by the local companion.
`press_key` Press a key or key chord in the selected local target. Supports Enter, Escape, Tab, Space, arrows, Home, End, PageUp, PageDown, Delete, function keys F1-F12, letters, digits, and common punctuation.	Press a key or key chord in the selected local target. Supports Enter, Escape, Tab, Space, arrows, Home, End, PageUp, PageDown, Delete, function keys F1-F12, letters, digits, and common punctuation.
`scroll` Scroll inside the selected local target.	Scroll inside the selected local target.
`wait` Wait for a short amount of time or for the local companion to settle after a UI change.	Wait for a short amount of time or for the local companion to settle after a UI change.
`wait_for` Wait for a Mac UI condition such as app_active, element_exists, element_focused, text_present, window_title_contains, dialog_present, or screen_stable. Prefer this over blind sleeps.	Wait for a Mac UI condition such as app_active, element_exists, element_focused, text_present, window_title_contains, dialog_present, or screen_stable. Prefer this over blind sleeps.
`run_steps` Run up to 50 deterministic desktop operations in one session round trip. Supports semantic ref actions, per-step delayMs, and per-step waitUntil conditions. Returns per-step ok/result/error entries and can include a final observation or screenshot.	Run up to 50 deterministic desktop operations in one session round trip. Supports semantic ref actions, per-step delayMs, and per-step waitUntil conditions. Returns per-step ok/result/error entries and can include a final observation or screenshot.
`request_human` Hand control to the local user for login, MFA, sensitive prompts, or manual review.	Hand control to the local user for login, MFA, sensitive prompts, or manual review.
`resume` Request agent control again. If the local user reclaimed control, local confirmation is required.	Request agent control again. If the local user reclaimed control, local confirmation is required.
`end_session` End the local computer control session and disconnect the companion.	End the local computer control session and disconnect the companion.
`act` Use the configured AI model to accomplish a local Mac instruction through a Weavz-managed observe-plan-act-verify loop. Prefer this over manually chaining mouse and keyboard primitives. The companion enforces local sign-in, permissions, selected apps, capability gates, pause/resume, and human confirmation.	Use the configured AI model to accomplish a local Mac instruction through a Weavz-managed observe-plan-act-verify loop. Prefer this over manually chaining mouse and keyboard primitives. The companion enforces local sign-in, permissions, selected apps, capability gates, pause/resume, and human confirmation.
`plan` Inspect the current local Mac state and produce a safe execution plan without acting.	Inspect the current local Mac state and produce a safe execution plan without acting.
`observe_goal` Observe the local Mac and identify UI candidates relevant to an instruction without acting.	Observe the local Mac and identify UI candidates relevant to an instruction without acting.
`extract_from_screen` Extract structured information from the current local Mac observation without acting.	Extract structured information from the current local Mac observation without acting.
`run_recipe` Run a transparent deterministic local-computer recipe made of existing safe operations, waits, and confirmation points.	Run a transparent deterministic local-computer recipe made of existing safe operations, waits, and confirmation points.
`stop_task` Stop the current managed local-computer task and hand control back to the local user.	Stop the current managed local-computer task and hand control back to the local user.