mujoco-scene-authoring

Name: mujoco-scene-authoring
Author: Hebbian-Robotics

15stars0forks

How to install mujoco-scene-authoring?

hebbian-robotics/mujoco-workbench/mujoco-scene-authoring

$npx -y skills add Hebbian-Robotics/mujoco-workbench --skill mujoco-scene-authoring

Installs into the current project.

Summary

Use this when adding or editing mujoco-workbench scene modules, example robot loaders, layout dataclasses, assets, cameras, task plans, or the scene module contract.

Files · 1

View on GitHub

SKILL.md

1	# Mujoco Scene Authoring
2
3	Read `ARCHITECTURE.md` before changing scene boundaries or reusable package
4	modules.
5
6	Scenes are fully qualified Python modules. Bundled examples live in:
7
8	```text
9	examples/scenes/
10	```
11
12	Reusable workbench code lives in:
13
14	```text
15	mujoco_workbench/
16	```
17
18	A scene module must export:
19
20	```python
21	NAME: str
22	ARM_PREFIXES: tuple[ArmSide, ...]
23	N_CUBES: int
24	GRIPPABLES: tuple[str, ...]
25	CAMERAS: tuple[tuple[str, CameraRole], ...]
26	AUX_ACTUATOR_NAMES: tuple[str, ...]
27
28	def build_spec() -> tuple[mujoco.MjModel, mujoco.MjData]: ...
29	def apply_initial_state(model, data, arms, cube_body_ids) -> None: ...
30	def make_task_plan(model, data, arms, cube_body_ids) -> dict[ArmSide, list[Step]]: ...
31	```
32
33	Alternatively, a non-scripted scene may export `step_free_play(t, model, data)`.
34
35	Keep task-specific choreography inside the scene module or its layout module.
36	Keep generic stepping/rendering/contract behavior in `mujoco_workbench`.
37
38	Prefer layout dataclasses for durable world-frame geometry constants. Keep
39	robot/sensor loading details in `examples/robots/` unless they are truly generic.
40
41	## Placement Pattern
42
43	For cameras and contact/touch poses, calculate first and teleop second. The
44	indicator-check scene is the reference pattern:
45
46	- Express target geometry in the layout module: object centers, front faces,
47	alert/contact points, row normals, chassis click poses, and clearance margins.
48	- Use `mwb debug surface-point` to pick a point just outside an object AABB.
49	The normal points outward from the object toward the robot/camera.
50	- Use `mwb debug standoff` to back a camera, TCP, or wrist mount away from
51	that point by a chosen depth.
52	- Use `mwb debug camera-look-at` to compute MuJoCo `xyaxes`; MuJoCo cameras
53	look along local `-z`, so do not guess quaternions by eye.
54	- Bake the resulting values into named layout constants or scene tunables, then
55	verify with rendered snapshots/grids and only use teleop for final joint poses.
56
57	Examples:
58
59	```bash
60	uv run mwb debug surface-point \
61	--scene examples.scenes.mobile_aloha_piper_indicator_check \
62	--geom light_left_r3_s10 \
63	--normal 0,-1,0 \
64	--clearance 0.02
65
66	uv run mwb debug standoff \
67	--target 5.10,1.04,1.00 \
68	--direction 0,1,0 \
69	--distance 0.25
70
71	uv run mwb debug camera-look-at \
72	--position 5.10,0.79,1.08 \
73	--target 5.10,1.04,1.00 \
74	--up 0,0,1
75	```
76
77	For wrist cameras, define the tool axis in the link-local frame, place the mesh
78	and camera outside the wrist housing with a small standoff, and aim the camera
79	at the TCP/contact point. The camera should frame the interaction target, not
80	the robot's own wrist geometry. Declare `CAMERA_INVARIANTS` so parent-body or
81	mode regressions fail during `--inspect`.
82
83	For server/rack contact, derive the click pose from the rack face plus the
84	robot's nose/swept-corner extents. Keep yaw-in-place and final approach as
85	separate base-only steps when rotation would otherwise clip into nearby static
86	geometry.
87
88	## Clearance Pattern
89
90	When a scene clips, prefer moving geometry and waypoints farther apart before
91	micro-tuning joints. Give the robot base enough stand-off from racks/carts,
92	increase object lift/standoff distances, and keep helper arms parked away from
93	the load-bearing arm unless they are actively needed.
94
95	Run clearance after placement or task-plan edits:
96
97	```bash
98	uv run mwb debug clearance \
99	--scene examples.scenes.mobile_aloha_piper_indicator_check \
100	--sample-dt 0.50 \
101	--max-distance 0.005 \
102	--exact-geom \
103	--top 12
104	```
105
106	Read the reported `active_steps` first. Fix the largest real penetrations by
107	phase: widen object spacing, split long carries into approach/at/lift/clear
108	steps, pin or release objects before unsafe base/lift transitions, and park the
109	non-load-bearing arm. Re-run `clearance` and `contracts`; the indicator-check
110	scene is allowed to report intentional contact at the light/server face.
111
112	Use fully qualified scene modules in commands:
113
114	```bash
115	uv run mwb run examples.scenes.mobile_aloha_piper_indicator_check --inspect
116	```
117
118	After scene edits, run:
119
120	```bash
121	uv run ruff check --fix
122	uv run ruff format
123	uv run ty check
124	uv run pytest
125	uv run mwb run examples.scenes.mobile_aloha_piper_indicator_check --inspect
126	```
127
128	If the edit touches task timing, phase labels, geometry, attachments, or IK
129	targets, also run contract checks.

People who liked this also viewed

6 picks

Type

mujoco-scene-authoring

15stars0forks

View on GitHub

How to install mujoco-scene-authoring?

hebbian-robotics/mujoco-workbench/mujoco-scene-authoring

$npx -y skills add Hebbian-Robotics/mujoco-workbench --skill mujoco-scene-authoring

Installs into the current project.

Summary

Use this when adding or editing mujoco-workbench scene modules, example robot loaders, layout dataclasses, assets, cameras, task plans, or the scene module contract.

Files · 1

View on GitHub

SKILL.md

1	# Mujoco Scene Authoring
2
3	Read `ARCHITECTURE.md` before changing scene boundaries or reusable package
4	modules.
5
6	Scenes are fully qualified Python modules. Bundled examples live in:
7
8	```text
9	examples/scenes/
10	```
11
12	Reusable workbench code lives in:
13
14	```text
15	mujoco_workbench/
16	```
17
18	A scene module must export:
19
20	```python
21	NAME: str
22	ARM_PREFIXES: tuple[ArmSide, ...]
23	N_CUBES: int
24	GRIPPABLES: tuple[str, ...]
25	CAMERAS: tuple[tuple[str, CameraRole], ...]
26	AUX_ACTUATOR_NAMES: tuple[str, ...]
27
28	def build_spec() -> tuple[mujoco.MjModel, mujoco.MjData]: ...
29	def apply_initial_state(model, data, arms, cube_body_ids) -> None: ...
30	def make_task_plan(model, data, arms, cube_body_ids) -> dict[ArmSide, list[Step]]: ...
31	```
32
33	Alternatively, a non-scripted scene may export `step_free_play(t, model, data)`.
34
35	Keep task-specific choreography inside the scene module or its layout module.
36	Keep generic stepping/rendering/contract behavior in `mujoco_workbench`.
37
38	Prefer layout dataclasses for durable world-frame geometry constants. Keep
39	robot/sensor loading details in `examples/robots/` unless they are truly generic.
40
41	## Placement Pattern
42
43	For cameras and contact/touch poses, calculate first and teleop second. The
44	indicator-check scene is the reference pattern:
45
46	- Express target geometry in the layout module: object centers, front faces,
47	alert/contact points, row normals, chassis click poses, and clearance margins.
48	- Use `mwb debug surface-point` to pick a point just outside an object AABB.
49	The normal points outward from the object toward the robot/camera.
50	- Use `mwb debug standoff` to back a camera, TCP, or wrist mount away from
51	that point by a chosen depth.
52	- Use `mwb debug camera-look-at` to compute MuJoCo `xyaxes`; MuJoCo cameras
53	look along local `-z`, so do not guess quaternions by eye.
54	- Bake the resulting values into named layout constants or scene tunables, then
55	verify with rendered snapshots/grids and only use teleop for final joint poses.
56
57	Examples:
58
59	```bash
60	uv run mwb debug surface-point \
61	--scene examples.scenes.mobile_aloha_piper_indicator_check \
62	--geom light_left_r3_s10 \
63	--normal 0,-1,0 \
64	--clearance 0.02
65
66	uv run mwb debug standoff \
67	--target 5.10,1.04,1.00 \
68	--direction 0,1,0 \
69	--distance 0.25
70
71	uv run mwb debug camera-look-at \
72	--position 5.10,0.79,1.08 \
73	--target 5.10,1.04,1.00 \
74	--up 0,0,1
75	```
76
77	For wrist cameras, define the tool axis in the link-local frame, place the mesh
78	and camera outside the wrist housing with a small standoff, and aim the camera
79	at the TCP/contact point. The camera should frame the interaction target, not
80	the robot's own wrist geometry. Declare `CAMERA_INVARIANTS` so parent-body or
81	mode regressions fail during `--inspect`.
82
83	For server/rack contact, derive the click pose from the rack face plus the
84	robot's nose/swept-corner extents. Keep yaw-in-place and final approach as
85	separate base-only steps when rotation would otherwise clip into nearby static
86	geometry.
87
88	## Clearance Pattern
89
90	When a scene clips, prefer moving geometry and waypoints farther apart before
91	micro-tuning joints. Give the robot base enough stand-off from racks/carts,
92	increase object lift/standoff distances, and keep helper arms parked away from
93	the load-bearing arm unless they are actively needed.
94
95	Run clearance after placement or task-plan edits:
96
97	```bash
98	uv run mwb debug clearance \
99	--scene examples.scenes.mobile_aloha_piper_indicator_check \
100	--sample-dt 0.50 \
101	--max-distance 0.005 \
102	--exact-geom \
103	--top 12
104	```
105
106	Read the reported `active_steps` first. Fix the largest real penetrations by
107	phase: widen object spacing, split long carries into approach/at/lift/clear
108	steps, pin or release objects before unsafe base/lift transitions, and park the
109	non-load-bearing arm. Re-run `clearance` and `contracts`; the indicator-check
110	scene is allowed to report intentional contact at the light/server face.
111
112	Use fully qualified scene modules in commands:
113
114	```bash
115	uv run mwb run examples.scenes.mobile_aloha_piper_indicator_check --inspect
116	```
117
118	After scene edits, run:
119
120	```bash
121	uv run ruff check --fix
122	uv run ruff format
123	uv run ty check
124	uv run pytest
125	uv run mwb run examples.scenes.mobile_aloha_piper_indicator_check --inspect
126	```
127
128	If the edit touches task timing, phase labels, geometry, attachments, or IK
129	targets, also run contract checks.

People who liked this also viewed

6 picks

Type