gemini-er
starsforks
How to install gemini-er?
graph-robots/open-robot-skills/gemini-er
$npx -y skills add graph-robots/open-robot-skills --skill gemini-erOpen-vocabulary 2D object detection via the Gemini Robotics-ER
SKILL.md
| 1 | # gemini-er |
| 2 | |
| 3 | API-backed 2D detection on Gemini Robotics-ER. Zero GPU. The canonical |
| 4 | perception recipe (from the dev tree's `perceive_gemini_er` workflow script) |
| 5 | is: `gemini-er.detect` → best box by `score` → `sam3.segment_box` on the full |
| 6 | frame at that box → depth projection → OBB fit. |
| 7 | |
| 8 | ## Install |
| 9 | |
| 10 | ```bash |
| 11 | uv sync --extra gemini-er # google-genai (pip: pip install -e ".[gemini-er]") |
| 12 | export GOOGLE_API_KEY=... # or GEMINI_API_KEY |
| 13 | ``` |
| 14 | |
| 15 | ## Config |
| 16 | |
| 17 | | Env | Meaning | Default | |
| 18 | |-----------------------|------------------------------------------|----------------------------------| |
| 19 | | `GAP_GEMINI_ER_MODEL` | Gemini model name | `gemini-robotics-er-1.5-preview` | |
| 20 | | `GOOGLE_API_KEY` / `GEMINI_API_KEY` | API key (SDK default resolution) | — | |
| 21 | |
| 22 | ## Contract |
| 23 | |
| 24 | `gemini-er.detect(image, query)` returns |
| 25 | `{"detections": [{"box": BoundingBox2D, "label": str, "score": float}]}`: |
| 26 | |
| 27 | - `box` is pixel-space `{x1, y1, x2, y2}` (top-left → bottom-right), clamped |
| 28 | to the image bounds. The model emits the Gemini `box_2d` convention |
| 29 | (`[ymin, xmin, ymax, xmax]` normalized 0–1000); conversion happens here. |
| 30 | - `score` defaults to 1.0 when the model reports none — callers select the |
| 31 | best detection with `max(..., key=score)`. |
| 32 | - No match (or unparseable model output) → empty `detections`, never an |
| 33 | error. Treat empty as "object not visible". |
| 34 | |
| 35 | ## When to use |
| 36 | |
| 37 | - Detection boxes for open-vocabulary prompts, no local weights. |
| 38 | - Prefer `molmo.point_prompt` when a single click point is enough, and |
| 39 | `vlm.query_yes_no` for semantic checks without localization. |
People who liked this also viewed
6 picksType
- find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 2026
- grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026
- vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026
- improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026
- setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026
- triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026
