sqs-queue-auditor

Name: sqs-queue-auditor
Author: anyshift-io

16stars0forks

How to install sqs-queue-auditor?

anyshift-io/sre-skills/sqs-queue-auditor

$npx -y skills add anyshift-io/sre-skills --skill sqs-queue-auditor

Installs into the current project.

Summary

Audit a single AWS SQS queue's configuration for the misconfigurations that silently drop or re-deliver messages while every attribute reads as fine. Parses the GetQueueAttributes output (and the referenced dead-letter queue), checks the redrive path (DLQ present, maxReceiveCount

Files · 1

View on GitHub

SKILL.md

1	# sqs-queue-auditor
2
3	Configuration-audit skill for a single AWS SQS queue. Takes the `GetQueueAttributes` output for one queue, applies the judgment a senior engineer applies to that one source (the thresholds, the known-bad combinations, the one arithmetic relationship that turns a correct-looking config into silent message loss), and returns a ranked list of findings with recommendations. Then it names exactly where a single queue's configuration stops being able to answer the question.
4
5	## When to invoke
6
7	- An agent is asked to review, harden, or sanity-check an SQS queue before or after it ships.
8	- Messages are going missing or being processed twice and nobody can see why from the console.
9	- A dead-letter queue is configured but empty during an incident, and the question is whether it is actually wired to catch what is failing.
10	- A queue is being added to a Terraform module or a CDK stack and the config should be checked against the known-bad combinations before apply.
11
12	## What this skill reads, and what it does not
13
14	It reads the static configuration of one queue, plus the attributes of the dead-letter queue that queue's own RedrivePolicy points at. Both are SQS control-plane reads (`GetQueueAttributes`). That is the entire input. The audit is correct and complete for what a queue's configuration can tell you, and it is explicit about the rest:
15
16	- It does not read CloudWatch. Live behaviour (redrive volume, age of the oldest message, in-flight count, empty-receive rate) is a time-series, not an attribute.
17	- It does not read the consumers. Whether the visibility timeout is actually long enough is a property of how long the consumer takes, which is not in the queue.
18	- It does not read account IAM. The effective set of principals that can act on the queue is the union of the resource policy (visible) and every identity policy in the account (not visible here).
19	- It does not read the producers. Whether the right services are writing to the queue, and whether anyone is draining the DLQ, needs the inventory on either side.
20
21	Every audit ends by naming these. The boundary is the same one every time: the join across resources, across sources, or across time.
22
23	## The methodology, in order
24
25	### 1. Parse the attributes
26
27	`GetQueueAttributes` returns every value as a string, and the compound attributes are JSON documents encoded inside those strings. Before any judgment:
28
29	- Parse `RedrivePolicy` (a JSON string) into `deadLetterTargetArn` and `maxReceiveCount`. A queue with no `RedrivePolicy` has no DLQ.
30	- Parse `MessageRetentionPeriod`, `VisibilityTimeout`, `DelaySeconds` as integer seconds (they arrive as strings).
31	- Parse `Policy` (a JSON string) into IAM statements, if present.
32	- Read `FifoQueue`, `ContentBasedDeduplication`, `SqsManagedSseEnabled`, `KmsMasterKeyId`.
33	- If a DLQ is referenced, load its attributes too. The retention-ordering check is impossible without them.
34
35	A naive read skips the embedded JSON entirely and never sees the redrive wiring. Parsing it is step zero of the judgment.
36
37	### 2. Audit the redrive path
38
39	The dead-letter path is where messages are supposed to go when processing fails. Three things break it:
40
41	- No DLQ on a processing queue (R1). Without a `RedrivePolicy`, a poison message is retried until `MessageRetentionPeriod` expires, then deleted with no signal. There is no quarantine.
42	- maxReceiveCount out of band (R2). Below 3, a transient downstream blip dead-letters good messages. Above 10, poison messages are retried many times before quarantine, delaying detection and feeding R4. The sane band is roughly 3 to 10.
43	- DLQ retention not longer than the source (R3). A message's age is measured from its original `SentTimestamp`, and SQS does not reset that timestamp when the message moves to the DLQ. If the DLQ's retention is less than or equal to the source's, a message that fails late in the source's window arrives in the DLQ already near its age limit and is deleted

People who liked this also viewed

6 picks

Type

sqs-queue-auditor

16stars0forks

View on GitHub

How to install sqs-queue-auditor?

anyshift-io/sre-skills/sqs-queue-auditor

$npx -y skills add anyshift-io/sre-skills --skill sqs-queue-auditor

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# sqs-queue-auditor
2
3	Configuration-audit skill for a single AWS SQS queue. Takes the `GetQueueAttributes` output for one queue, applies the judgment a senior engineer applies to that one source (the thresholds, the known-bad combinations, the one arithmetic relationship that turns a correct-looking config into silent message loss), and returns a ranked list of findings with recommendations. Then it names exactly where a single queue's configuration stops being able to answer the question.
4
5	## When to invoke
6
7	- An agent is asked to review, harden, or sanity-check an SQS queue before or after it ships.
8	- Messages are going missing or being processed twice and nobody can see why from the console.
9	- A dead-letter queue is configured but empty during an incident, and the question is whether it is actually wired to catch what is failing.
10	- A queue is being added to a Terraform module or a CDK stack and the config should be checked against the known-bad combinations before apply.
11
12	## What this skill reads, and what it does not
13
14	It reads the static configuration of one queue, plus the attributes of the dead-letter queue that queue's own RedrivePolicy points at. Both are SQS control-plane reads (`GetQueueAttributes`). That is the entire input. The audit is correct and complete for what a queue's configuration can tell you, and it is explicit about the rest:
15
16	- It does not read CloudWatch. Live behaviour (redrive volume, age of the oldest message, in-flight count, empty-receive rate) is a time-series, not an attribute.
17	- It does not read the consumers. Whether the visibility timeout is actually long enough is a property of how long the consumer takes, which is not in the queue.
18	- It does not read account IAM. The effective set of principals that can act on the queue is the union of the resource policy (visible) and every identity policy in the account (not visible here).
19	- It does not read the producers. Whether the right services are writing to the queue, and whether anyone is draining the DLQ, needs the inventory on either side.
20
21	Every audit ends by naming these. The boundary is the same one every time: the join across resources, across sources, or across time.
22
23	## The methodology, in order
24
25	### 1. Parse the attributes
26
27	`GetQueueAttributes` returns every value as a string, and the compound attributes are JSON documents encoded inside those strings. Before any judgment:
28
29	- Parse `RedrivePolicy` (a JSON string) into `deadLetterTargetArn` and `maxReceiveCount`. A queue with no `RedrivePolicy` has no DLQ.
30	- Parse `MessageRetentionPeriod`, `VisibilityTimeout`, `DelaySeconds` as integer seconds (they arrive as strings).
31	- Parse `Policy` (a JSON string) into IAM statements, if present.
32	- Read `FifoQueue`, `ContentBasedDeduplication`, `SqsManagedSseEnabled`, `KmsMasterKeyId`.
33	- If a DLQ is referenced, load its attributes too. The retention-ordering check is impossible without them.
34
35	A naive read skips the embedded JSON entirely and never sees the redrive wiring. Parsing it is step zero of the judgment.
36
37	### 2. Audit the redrive path
38
39	The dead-letter path is where messages are supposed to go when processing fails. Three things break it:
40
41	- No DLQ on a processing queue (R1). Without a `RedrivePolicy`, a poison message is retried until `MessageRetentionPeriod` expires, then deleted with no signal. There is no quarantine.
42	- maxReceiveCount out of band (R2). Below 3, a transient downstream blip dead-letters good messages. Above 10, poison messages are retried many times before quarantine, delaying detection and feeding R4. The sane band is roughly 3 to 10.
43	- DLQ retention not longer than the source (R3). A message's age is measured from its original `SentTimestamp`, and SQS does not reset that timestamp when the message moves to the DLQ. If the DLQ's retention is less than or equal to the source's, a message that fails late in the source's window arrives in the DLQ already near its age limit and is deleted

People who liked this also viewed

6 picks

Type