storing-and-querying-vectors

Name: storing-and-querying-vectors
Author: aws

2.4kinstalls190forks

How to install storing-and-querying-vectors?

aws/agent-toolkit-for-aws/storing-and-querying-vectors

$npx -y skills add aws/agent-toolkit-for-aws --skill storing-and-querying-vectors

Installs into the current project.

Summary

Store and query vector embeddings using Amazon S3 Vectors, a cost-effective long-term vector storage service with its own API namespace (s3vectors). Triggers on: create S3 vector bucket, vector index, store embeddings, semantic search, RAG vector storage, similarity search, vecto

Files · 1

View on GitHub

SKILL.md

1	# Store and Query Vectors with Amazon S3 Vectors
2
3	## Overview
4
5	Amazon S3 Vectors is a cost-effective AWS service for storing and querying vector embeddings at scale. Optimized for long-term storage with subsecond latency for cold queries, as low as 100ms for warm queries.
6
7	## Decision Guide
8
9	- Hundreds/thousands of sustained queries per second (QPS): Wrong tool. Recommend OpenSearch.
10	- Hybrid search, aggregations, faceted search: Recommend OpenSearch with S3 Vectors as storage engine. For OpenSearch integration, search AWS docs for `"Using S3 Vectors with OpenSearch Service"`.
11	- Tiered (bulk + hot): S3 Vectors for storage + OpenSearch Serverless for real-time. See `references/limits-and-patterns.md`.
12	- Cost-effective storage, infrequent queries, RAG: S3 Vectors is the right fit. Proceed.
13
14	For latest guidance, search AWS docs for `"S3 Vectors best practices"`.
15
16	## Common Tasks
17
18	Classify the request before starting:
19
20	- Simple query: Existing index, skip to Step 6
21	- Standard: You MUST list existing indexes first and suggest reusing if relevant. Else, new index + store vectors, follow Steps 2-6
22	- Migration or multi-tenant: Read `references/limits-and-patterns.md` first, then Steps 2-6
23
24	You MUST execute commands using AWS MCP server tools when connected. Fall back to AWS CLI only if AWS MCP is unavailable. You MUST explain each step to the user before executing.
25
26	### 1. Verify Dependencies
27
28	Constraints:
29
30	- You MUST check whether AWS MCP tools or AWS CLI is available and inform user if missing
31	- You MUST confirm target AWS region
32
33	### 2. Create a Vector Bucket
34
35	You MUST confirm bucket name with user. Names: 3-63 chars, lowercase letters, numbers, hyphens only. Encryption (SSE-S3 default or SSE-KMS for compliance) is immutable after creation.
36
37	```bash
38	aws s3vectors create-vector-bucket \
39	--vector-bucket-name <BUCKET_NAME>
40	```
41
42	Constraints:
43
44	- You MUST explain encryption cannot be changed after creation
45	- For SSE-KMS, KMS key policy MUST grant `kms:GenerateDataKey` and `kms:Decrypt` to the S3 Vectors service principal `indexing.s3vectors.amazonaws.com`. You MUST use full KMS key ARN (not alias). See `references/limits-and-patterns.md` for command example.
46
47	### 3. Create a Vector Index
48
49	Every parameter is immutable after creation.
50
51	Pre-flight checklist (confirm ALL with user):
52
53	1. Dimension (required, integer 1-4096) -- MUST match embedding model output
54	2. Distance metric (required) -- `cosine` or `euclidean`. Use embedding model's recommended metric;
55	3. Non-filterable metadata keys (optional, max 10, 1-63 chars) -- Declare at creation or lose forever. For Bedrock Knowledge Bases integration, search AWS docs for `"S3 Vectors Bedrock Knowledge Bases prerequisites"` to get the required key names.
56	4. Encryption (optional) -- Inherits from bucket. Override per-index if needed.
57
58	```bash
59	aws s3vectors create-index \
60	--vector-bucket-name <BUCKET_NAME> \
61	--index-name <INDEX_NAME> \
62	--dimension <DIM> \
63	--distance-metric <cosine\|euclidean> \
64	--data-type float32 \
65	--metadata-configuration '{"nonFilterableMetadataKeys":["<KEY1>","<KEY2>"]}'
66	```
67
68	Omit `--metadata-configuration` if no non-filterable keys are needed.
69
70	Index names: 3-63 chars, lowercase, numbers, hyphens, dots. Unique within bucket. Filterable metadata: 2 KB limit. Total metadata (filterable + non-filterable combined): 40 KB. See `references/metadata-filtering.md`.
71
72	### 4. Generate Embeddings (if needed)
73
74	Skip to Step 5 (store) or Step 6 (query) if user already has embeddings.
75
76	Constraints:
77
78	- You MUST ask which embedding model to use if not specified
79	- You MUST NOT assume a default model
80	- Dimension MUST match Step 3
81	- You MUST use the same model for both storing and querying
82
83	Generate embeddings with Bedrock invoke-model:
84
85	```bash
86	aws bedrock-runtime invoke-model \
87	--model-id <MODEL_ID> \
88	--content-type application/json \
89	--cli-binary-format raw-in-base64-out \
90	--body '{"inputText": "your text"}' \
91	invoke-model-output.json
92	```
93
94	You MUST use `--cli-binary-format raw-in-base64-out` for CLI v2. Output file is required for CLI. The response key is model-dependent (e.g., embedding for Titan, embeddings for Cohere). For Titan, parse with `json.load(open('invoke-model-output.json'))['embedding']`. Use `embedding` array as `float32` in put-vectors or query-vectors. For batch embedding generation, use AWS SDK or CLI.
95
96	### 5. Put Vectors
97
98	```bas

People who liked this also viewed

6 picks

Type

storing-and-querying-vectors

2.4kinstalls190forks

View on GitHub

How to install storing-and-querying-vectors?

aws/agent-toolkit-for-aws/storing-and-querying-vectors

$npx -y skills add aws/agent-toolkit-for-aws --skill storing-and-querying-vectors

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Store and Query Vectors with Amazon S3 Vectors
2
3	## Overview
4
5	Amazon S3 Vectors is a cost-effective AWS service for storing and querying vector embeddings at scale. Optimized for long-term storage with subsecond latency for cold queries, as low as 100ms for warm queries.
6
7	## Decision Guide
8
9	- Hundreds/thousands of sustained queries per second (QPS): Wrong tool. Recommend OpenSearch.
10	- Hybrid search, aggregations, faceted search: Recommend OpenSearch with S3 Vectors as storage engine. For OpenSearch integration, search AWS docs for `"Using S3 Vectors with OpenSearch Service"`.
11	- Tiered (bulk + hot): S3 Vectors for storage + OpenSearch Serverless for real-time. See `references/limits-and-patterns.md`.
12	- Cost-effective storage, infrequent queries, RAG: S3 Vectors is the right fit. Proceed.
13
14	For latest guidance, search AWS docs for `"S3 Vectors best practices"`.
15
16	## Common Tasks
17
18	Classify the request before starting:
19
20	- Simple query: Existing index, skip to Step 6
21	- Standard: You MUST list existing indexes first and suggest reusing if relevant. Else, new index + store vectors, follow Steps 2-6
22	- Migration or multi-tenant: Read `references/limits-and-patterns.md` first, then Steps 2-6
23
24	You MUST execute commands using AWS MCP server tools when connected. Fall back to AWS CLI only if AWS MCP is unavailable. You MUST explain each step to the user before executing.
25
26	### 1. Verify Dependencies
27
28	Constraints:
29
30	- You MUST check whether AWS MCP tools or AWS CLI is available and inform user if missing
31	- You MUST confirm target AWS region
32
33	### 2. Create a Vector Bucket
34
35	You MUST confirm bucket name with user. Names: 3-63 chars, lowercase letters, numbers, hyphens only. Encryption (SSE-S3 default or SSE-KMS for compliance) is immutable after creation.
36
37	```bash
38	aws s3vectors create-vector-bucket \
39	--vector-bucket-name <BUCKET_NAME>
40	```
41
42	Constraints:
43
44	- You MUST explain encryption cannot be changed after creation
45	- For SSE-KMS, KMS key policy MUST grant `kms:GenerateDataKey` and `kms:Decrypt` to the S3 Vectors service principal `indexing.s3vectors.amazonaws.com`. You MUST use full KMS key ARN (not alias). See `references/limits-and-patterns.md` for command example.
46
47	### 3. Create a Vector Index
48
49	Every parameter is immutable after creation.
50
51	Pre-flight checklist (confirm ALL with user):
52
53	1. Dimension (required, integer 1-4096) -- MUST match embedding model output
54	2. Distance metric (required) -- `cosine` or `euclidean`. Use embedding model's recommended metric;
55	3. Non-filterable metadata keys (optional, max 10, 1-63 chars) -- Declare at creation or lose forever. For Bedrock Knowledge Bases integration, search AWS docs for `"S3 Vectors Bedrock Knowledge Bases prerequisites"` to get the required key names.
56	4. Encryption (optional) -- Inherits from bucket. Override per-index if needed.
57
58	```bash
59	aws s3vectors create-index \
60	--vector-bucket-name <BUCKET_NAME> \
61	--index-name <INDEX_NAME> \
62	--dimension <DIM> \
63	--distance-metric <cosine\|euclidean> \
64	--data-type float32 \
65	--metadata-configuration '{"nonFilterableMetadataKeys":["<KEY1>","<KEY2>"]}'
66	```
67
68	Omit `--metadata-configuration` if no non-filterable keys are needed.
69
70	Index names: 3-63 chars, lowercase, numbers, hyphens, dots. Unique within bucket. Filterable metadata: 2 KB limit. Total metadata (filterable + non-filterable combined): 40 KB. See `references/metadata-filtering.md`.
71
72	### 4. Generate Embeddings (if needed)
73
74	Skip to Step 5 (store) or Step 6 (query) if user already has embeddings.
75
76	Constraints:
77
78	- You MUST ask which embedding model to use if not specified
79	- You MUST NOT assume a default model
80	- Dimension MUST match Step 3
81	- You MUST use the same model for both storing and querying
82
83	Generate embeddings with Bedrock invoke-model:
84
85	```bash
86	aws bedrock-runtime invoke-model \
87	--model-id <MODEL_ID> \
88	--content-type application/json \
89	--cli-binary-format raw-in-base64-out \
90	--body '{"inputText": "your text"}' \
91	invoke-model-output.json
92	```
93
94	You MUST use `--cli-binary-format raw-in-base64-out` for CLI v2. Output file is required for CLI. The response key is model-dependent (e.g., embedding for Titan, embeddings for Cohere). For Titan, parse with `json.load(open('invoke-model-output.json'))['embedding']`. Use `embedding` array as `float32` in put-vectors or query-vectors. For batch embedding generation, use AWS SDK or CLI.
95
96	### 5. Put Vectors
97
98	```bas

People who liked this also viewed

6 picks

Type