$npx -y skills add wshobson/agents --skill saga-orchestrationImplement saga patterns for distributed transactions and cross-aggregate workflows. Use this skill when implementing distributed transactions across microservices where 2PC is unavailable, designing compensating actions for failed order workflows that span inventory, payment, and
| 1 | # Saga Orchestration |
| 2 | |
| 3 | Patterns for managing distributed transactions and long-running business processes without two-phase commit. |
| 4 | |
| 5 | ## Inputs and Outputs |
| 6 | |
| 7 | **What you provide:** |
| 8 | - Service boundaries and ownership (which service owns which step) |
| 9 | - Transaction requirements (which steps must be atomic, which can be eventual) |
| 10 | - Failure modes for each step (transient vs. permanent, retry policy) |
| 11 | - SLA requirements per step (informs timeout configuration) |
| 12 | - Existing event/messaging infrastructure (Kafka, RabbitMQ, SQS, etc.) |
| 13 | |
| 14 | **What this skill produces:** |
| 15 | - Saga definition with ordered steps, action commands, and compensation commands |
| 16 | - Orchestrator or choreography implementation for your chosen pattern |
| 17 | - Compensation logic for each participant service (idempotent, always-succeeds) |
| 18 | - Step timeout configuration with per-step deadlines |
| 19 | - Monitoring setup: state machine metrics, stuck saga detection, DLQ recovery |
| 20 | |
| 21 | --- |
| 22 | |
| 23 | ## When to Use This Skill |
| 24 | |
| 25 | - Coordinating multi-service transactions without distributed locks |
| 26 | - Implementing compensating transactions for partial failures |
| 27 | - Managing long-running business workflows (minutes to hours) |
| 28 | - Handling failures in distributed systems where atomicity is required |
| 29 | - Building order fulfillment, approval, or booking processes |
| 30 | - Replacing fragile two-phase commit with async compensation |
| 31 | |
| 32 | --- |
| 33 | |
| 34 | ## Detailed section: Core Concepts |
| 35 | |
| 36 | Moved to `references/details.md`. |
| 37 | |
| 38 | ## Detailed section: Templates |
| 39 | |
| 40 | Moved to `references/details.md`. |
| 41 | |
| 42 | ## Best Practices |
| 43 | |
| 44 | ### Do's |
| 45 | |
| 46 | - **Make every step idempotent** — Commands may be replayed on broker reconnect |
| 47 | - **Design compensations carefully** — They are the most critical code path |
| 48 | - **Use correlation IDs** — The `saga_id` must flow through every event and log |
| 49 | - **Implement per-step timeouts** — Never wait indefinitely for a participant reply |
| 50 | - **Log state transitions** — `saga_id`, `step_name`, `old_state → new_state` on every change |
| 51 | - **Test compensation paths explicitly** — Inject failures at each step index in integration tests |
| 52 | |
| 53 | ### Don'ts |
| 54 | |
| 55 | - **Don't assume instant completion** — Sagas are async and may take minutes |
| 56 | - **Don't skip compensation testing** — The rollback path is the hardest to get right |
| 57 | - **Don't couple services directly** — Use async messaging, never synchronous calls inside a saga step |
| 58 | - **Don't ignore partial failures** — A step that partially executed still needs compensation |
| 59 | - **Don't use a global timeout** — Each step has different latency characteristics |
| 60 | |
| 61 | --- |
| 62 | |
| 63 | ## Troubleshooting |
| 64 | |
| 65 | ### Saga stuck in COMPENSATING state |
| 66 | |
| 67 | A saga enters compensation but never reaches FAILED. This means a compensation handler is throwing an unhandled exception and never publishing `SagaCompensationCompleted`. Add dead-letter queue (DLQ) handling to compensation consumers and ensure every compensation action publishes a result event even when the underlying operation was already rolled back. |
| 68 | |
| 69 | ```python |
| 70 | async def handle_release_reservation(self, command: Dict): |
| 71 | try: |
| 72 | await self.release_reservation(command["original_result"]["reservation_id"]) |
| 73 | except ReservationNotFoundError: |
| 74 | pass # Already released — treat as success |
| 75 | # Always publish completion, regardless of outcome |
| 76 | await self.event_publisher.publish("SagaCompensationCompleted", { |
| 77 | "saga_id": command["saga_id"], |
| 78 | "step_name": "reserve_inventory" |
| 79 | }) |
| 80 | ``` |
| 81 | |
| 82 | ### Duplicate saga executions on restart |
| 83 | |
| 84 | If your orchestrator service restarts mid-saga, it may replay events and re-execute already-completed steps. Guard every step action with an idempotency key — see **Template 3** above. |
| 85 | |
| 86 | ### Choreography saga losing events |
| 87 | |
| 88 | In a choreography-based saga, a downstream service may miss an event if it was offline when published. Use a durable message broker (Kafka with replication, RabbitMQ with persistence) and store the current saga state in a dedicated `saga_log` table so you can replay from the last known good step. |
| 89 | |
| 90 | ### Timeout firing before a slow-but-valid step completes |
| 91 | |
| 92 | A step like `create_shipment` might take up to 15 minutes during peak load but your global timeout is 5 minutes, causing spurious compensation. Make step timeouts configurable per step type — see `references/advanced-patterns.md` for the `TimeoutSagaOrchestrator` implementation and the `STEP_TIMEOUTS` dict pattern. |
| 93 | |
| 94 | ### Compensation order not matching execution order |
| 95 | |
| 96 | When two steps both complete before a failure is detected, comp |