$npx -y skills add graph-robots/open-robot-skills --skill tracking-objectsLong-running skill that drives the SAM3 tracker from the
| 1 | # tracking-objects |
| 2 | |
| 3 | Long-running tracker skill. Init on first frame, update per tick, close on |
| 4 | exit. Used as a parallel sibling to other long-running skills (e.g. a |
| 5 | policy) when continuous state estimation is needed. |
| 6 | |
| 7 | The skill is **class-based and stateful**: the tracker session id and the |
| 8 | last good mask/box live on the skill instance, so repeated visits to the |
| 9 | same state within one workflow execution can resume the session instead of |
| 10 | re-seeding it (pass `close_on_exit: false` to keep the session open across |
| 11 | visits; the final visit — or the instance teardown — closes it). |
| 12 | |
| 13 | It is also a **streaming** skill (`gap.streaming: true`): each update tick |
| 14 | publishes a tracker snapshot |
| 15 | `{mask, box, confidence, object_present, n_updates}` via `ctx.publish`, so |
| 16 | downstream `{"$ref": "<node>"}` consumers see the latest tracked state |
| 17 | while the loop is still running. |
| 18 | |
| 19 | ## Install |
| 20 | |
| 21 | Depends on the **sam3** tool bundle: |
| 22 | |
| 23 | ```bash |
| 24 | uv sync --extra sam3 # (pip: pip install -e "open-robot-skills[sam3]") |
| 25 | ``` |
| 26 | |
| 27 | ## When to use |
| 28 | |
| 29 | - A workflow that needs the live mask + box of an object across many |
| 30 | frames (e.g., a supervisor that monitors a target's location while a |
| 31 | policy manipulates it). |
| 32 | - Wrapped under a `parallel` state with a `join_policy` so the tracker is |
| 33 | cooperatively cancelled when the sibling branch finishes (the loop |
| 34 | checks `ctx.cancel_token` every tick). |
| 35 | |
| 36 | ## Output |
| 37 | |
| 38 | Returns the final mask, box, confidence, and a flag indicating whether |
| 39 | the object was visibly present at exit. Intermediate updates are |
| 40 | published as streaming snapshots; the return value exposes only the |
| 41 | final state. |
| 42 | |
| 43 | ## Tool form |
| 44 | |
| 45 | The bundle also exposes the loop as a flat tool — |
| 46 | `tracking-objects.track` — for callers that want to invoke it as a single |
| 47 | unit (one fresh tracker session per call) rather than as a workflow |
| 48 | state. |