computer-use

Name: computer-use
Author: stablyai

33kinstalls1.9kforks

How to install computer-use?

stablyai/orca/computer-use

$npx -y skills add stablyai/orca --skill computer-use

Installs into the current project.

Summary

Use Orca's computer-use CLI to inspect and operate local desktop app windows through accessibility trees, screenshots, and safe UI actions. Use for desktop app interaction: list apps/windows, get app state, read visible UI, click controls, type, press keys, scroll, drag, set valu

Files · 1

View on GitHub

SKILL.md

1	# Computer Use
2
3	Use this skill for desktop UI through `orca computer`. When the requested target is a website or web app, operate the desktop browser app/window that contains the page.
4
5	## Preconditions
6
7	- Choose the Orca executable once: use the `ORCA_CLI_COMMAND` environment value when set;
8	otherwise use `orca-dev` in a dev session exposing `ORCA_DEV_REPO_ROOT`, `orca-ide` on
9	Linux outside an Orca-managed terminal, and `orca` everywhere else. Never try bare
10	`orca` first on unmanaged Linux because it normally resolves to the GNOME screen reader.
11	- In every command example, `ORCA` is a documentation placeholder — including examples that
12	name a specific shell. Replace it with that chosen executable before running the command;
13	do not create a shell variable or run `ORCA` literally. Blocks that name no shell are
14	intentionally shell-neutral for POSIX shells, PowerShell, and cmd.exe.
15	- Prefer `--json`. Screenshot bytes are omitted from JSON and written to `screenshot.path`.
16	- Do not push, submit forms, send messages, buy items, delete data, change account settings, or expose secrets unless the user explicitly asked for that action.
17	- If an app contains sensitive content, read only what the user requested.
18
19	```text
20	ORCA status --json
21	ORCA computer capabilities --json
22	```
23
24	## Core Loop
25
26	```text
27	ORCA computer list-apps --json
28	ORCA computer get-app-state --app com.spotify.client --json
29	ORCA computer click --app com.spotify.client --element-index 42 --json
30	```
31
32	Use the fresh state returned by each action for the next element index. Element indexes are the numeric labels shown in the tree; they may be sparse when noisy sections are omitted, so never infer valid indexes from `elementCount` or "Visible elements." Element indexes are short-lived and go stale after delays, navigation, focus changes, scrolling, window changes, or app re-rendering.
33
34	In `--json` output, read the accessibility tree and action indexes from `result.snapshot.treeText`; `elementCount` is only a count and must not be used to infer indexes.
35
36	## App Selectors
37
38	Prefer bundle IDs from `list-apps`; names are acceptable when unambiguous. Use `pid:<number>` only when bundle ID or name matching is ambiguous.
39
40	```text
41	ORCA computer get-app-state --app com.microsoft.edgemac --json
42	ORCA computer get-app-state --app Spotify --json
43	ORCA computer get-app-state --app pid:12345 --json
44	```
45
46	For apps with multiple windows or ambiguous titles, run `list-windows` first. Prefer `--window-id <id>` when the listed id is not `none`; otherwise use `--window-index <n>`. Once you choose a window, pass the same selector to `get-app-state` and later actions until the target window changes.
47
48	## Commands
49
50	```text
51	ORCA computer permissions --json
52	ORCA computer capabilities --json
53	ORCA computer list-apps --json
54	ORCA computer list-windows --app <app> --json
55	ORCA computer get-app-state --app <app> --json
56	ORCA computer get-app-state --app <app> --restore-window --json
57	ORCA computer click --app <app> --element-index <index> --json
58	ORCA computer click --app <app> --x 100 --y 100 --json
59	ORCA computer perform-secondary-action --app <app> --element-index <index> --action <name> --json
60	ORCA computer set-value --app <app> --element-index <index> --value "text" --json
61	ORCA computer type-text --app <app> --text "text" --json
62	ORCA computer press-key --app <app> --key Return --json
63	ORCA computer hotkey --app <app> --key CmdOrCtrl+A --json
64	ORCA computer paste-text --app <app> --text "text" --json
65	ORCA computer scroll --app <app> (--element-index <index> \| --x <x> --y <y>) --direction down --json
66	ORCA computer drag --app <app> --from-element-index <index> --to-element-index <index> --json
67	ORCA computer drag --app <app> --from-x 100 --from-y 100 --to-x 300 --to-y 300 --json
68	```
69
70	Use `--no-screenshot` only when pixels are not needed. Use `--text-stdin` or `--value-stdin` for sensitive text so payloads do not land in shell history. On Linux and Windows, action payloads still pass through a short-lived local operation file, so avoid sending secrets unless the user explicitly asked for them:
71
72	POSIX-shell example (use the equivalent stdin mechanism without command-history exposure in
73	PowerShell or cmd.exe):
74
75	```bash
76	printf '%s' "$TEXT" \| ORCA computer set-value --app <app> --element-index <index> --value-stdin --json
77	```
78
79	## Action Rules
80
81	- Prefer semantic actions: `set-value` for editable fields, `click` for

People who liked this also viewed

6 picks

Type

computer-use

33kinstalls1.9kforks

View on GitHub

How to install computer-use?

stablyai/orca/computer-use

$npx -y skills add stablyai/orca --skill computer-use

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Computer Use
2
3	Use this skill for desktop UI through `orca computer`. When the requested target is a website or web app, operate the desktop browser app/window that contains the page.
4
5	## Preconditions
6
7	- Choose the Orca executable once: use the `ORCA_CLI_COMMAND` environment value when set;
8	otherwise use `orca-dev` in a dev session exposing `ORCA_DEV_REPO_ROOT`, `orca-ide` on
9	Linux outside an Orca-managed terminal, and `orca` everywhere else. Never try bare
10	`orca` first on unmanaged Linux because it normally resolves to the GNOME screen reader.
11	- In every command example, `ORCA` is a documentation placeholder — including examples that
12	name a specific shell. Replace it with that chosen executable before running the command;
13	do not create a shell variable or run `ORCA` literally. Blocks that name no shell are
14	intentionally shell-neutral for POSIX shells, PowerShell, and cmd.exe.
15	- Prefer `--json`. Screenshot bytes are omitted from JSON and written to `screenshot.path`.
16	- Do not push, submit forms, send messages, buy items, delete data, change account settings, or expose secrets unless the user explicitly asked for that action.
17	- If an app contains sensitive content, read only what the user requested.
18
19	```text
20	ORCA status --json
21	ORCA computer capabilities --json
22	```
23
24	## Core Loop
25
26	```text
27	ORCA computer list-apps --json
28	ORCA computer get-app-state --app com.spotify.client --json
29	ORCA computer click --app com.spotify.client --element-index 42 --json
30	```
31
32	Use the fresh state returned by each action for the next element index. Element indexes are the numeric labels shown in the tree; they may be sparse when noisy sections are omitted, so never infer valid indexes from `elementCount` or "Visible elements." Element indexes are short-lived and go stale after delays, navigation, focus changes, scrolling, window changes, or app re-rendering.
33
34	In `--json` output, read the accessibility tree and action indexes from `result.snapshot.treeText`; `elementCount` is only a count and must not be used to infer indexes.
35
36	## App Selectors
37
38	Prefer bundle IDs from `list-apps`; names are acceptable when unambiguous. Use `pid:<number>` only when bundle ID or name matching is ambiguous.
39
40	```text
41	ORCA computer get-app-state --app com.microsoft.edgemac --json
42	ORCA computer get-app-state --app Spotify --json
43	ORCA computer get-app-state --app pid:12345 --json
44	```
45
46	For apps with multiple windows or ambiguous titles, run `list-windows` first. Prefer `--window-id <id>` when the listed id is not `none`; otherwise use `--window-index <n>`. Once you choose a window, pass the same selector to `get-app-state` and later actions until the target window changes.
47
48	## Commands
49
50	```text
51	ORCA computer permissions --json
52	ORCA computer capabilities --json
53	ORCA computer list-apps --json
54	ORCA computer list-windows --app <app> --json
55	ORCA computer get-app-state --app <app> --json
56	ORCA computer get-app-state --app <app> --restore-window --json
57	ORCA computer click --app <app> --element-index <index> --json
58	ORCA computer click --app <app> --x 100 --y 100 --json
59	ORCA computer perform-secondary-action --app <app> --element-index <index> --action <name> --json
60	ORCA computer set-value --app <app> --element-index <index> --value "text" --json
61	ORCA computer type-text --app <app> --text "text" --json
62	ORCA computer press-key --app <app> --key Return --json
63	ORCA computer hotkey --app <app> --key CmdOrCtrl+A --json
64	ORCA computer paste-text --app <app> --text "text" --json
65	ORCA computer scroll --app <app> (--element-index <index> \| --x <x> --y <y>) --direction down --json
66	ORCA computer drag --app <app> --from-element-index <index> --to-element-index <index> --json
67	ORCA computer drag --app <app> --from-x 100 --from-y 100 --to-x 300 --to-y 300 --json
68	```
69
70	Use `--no-screenshot` only when pixels are not needed. Use `--text-stdin` or `--value-stdin` for sensitive text so payloads do not land in shell history. On Linux and Windows, action payloads still pass through a short-lived local operation file, so avoid sending secrets unless the user explicitly asked for them:
71
72	POSIX-shell example (use the equivalent stdin mechanism without command-history exposure in
73	PowerShell or cmd.exe):
74
75	```bash
76	printf '%s' "$TEXT" \| ORCA computer set-value --app <app> --element-index <index> --value-stdin --json
77	```
78
79	## Action Rules
80
81	- Prefer semantic actions: `set-value` for editable fields, `click` for

People who liked this also viewed

6 picks

Type