vlm
starsforks
How to install vlm?
graph-robots/open-robot-skills/vlm
$npx -y skills add graph-robots/open-robot-skills --skill vlmFree-form and yes/no visual question answering against a hosted
SKILL.md
| 1 | # vlm |
| 2 | |
| 3 | API-backed vision-language Q&A. Zero GPU: every provider is a remote |
| 4 | endpoint. Images are gap-native `uint8 [H, W, 3]` numpy arrays, PNG-encoded |
| 5 | on the wire. |
| 6 | |
| 7 | ## Providers |
| 8 | |
| 9 | Selected by `GAP_VLM_PROVIDER` (default `openrouter`); each tool also accepts |
| 10 | a per-call `provider=` override. |
| 11 | |
| 12 | | Provider | Backend | Config (env) | |
| 13 | |--------------|-----------------------------------------------------|-----------------------------------------------------------| |
| 14 | | `openrouter` | OpenRouter's OpenAI-compatible chat-completions API | `OPENROUTER_API_KEY` (or `GAP_VLM_API_KEY`); `GAP_VLM_MODEL` (default `gemini-3.1-flash-lite-preview`, see `DEFAULT_MODEL` in `tools.py`); set `GAP_VLM_BASE_URL` to point at another OpenAI-compatible server (e.g. a local vLLM) | |
| 15 | | `vertex` | Vertex AI via google-genai (Gemini models) | `GAP_VLM_MODEL`, `GAP_VLM_PROJECT_ID`, `GAP_VLM_REGION` | |
| 16 | |
| 17 | The `vertex` provider lazy-imports google-genai — install the engine's vertex |
| 18 | extra first: `pip install "graph-as-policy[vertex]"`. |
| 19 | |
| 20 | ## When to use |
| 21 | |
| 22 | - Semantic scene checks and checkpoint verification (`vlm.query_yes_no`). |
| 23 | - Free-form scene descriptions or attribute queries (`vlm.query`). |
| 24 | - Prefer `gemini-er.detect` when you need pixel-space bounding boxes, and |
| 25 | `molmo.point_prompt` when you need a single click point. |
| 26 | |
| 27 | ## Notes |
| 28 | |
| 29 | - `vlm.query_yes_no` coerces with the source-verbatim rule: answer is true |
| 30 | iff `"yes"` appears in the lowercased reply. |
| 31 | - Requests carry no system prompt and no temperature knob (mirrors the |
| 32 | original `vlm.v1` proto); both providers pin `temperature: 0.0`. |
People who liked this also viewed
6 picksType
- find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 2026
- grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026
- vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026
- improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026
- setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026
- triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026
