finding-data-lake-assets

Name: finding-data-lake-assets
Author: aws

2.4kinstalls190forks

How to install finding-data-lake-assets?

aws/agent-toolkit-for-aws/finding-data-lake-assets

$npx -y skills add aws/agent-toolkit-for-aws --skill finding-data-lake-assets

Installs into the current project.

Summary

Resolve data lake and lakehouse asset references across Glue Data Catalog, S3, S3 Tables, and Redshift. Triggers on: find the table, where is our data, which table has, locate dataset, find data for, search catalog, what tables match, Redshift table, lakehouse table, data lake ta

Files · 1

View on GitHub

SKILL.md

1	# Find Data Lake Assets
2
3	## Overview
4
5	Resolves data lake asset references to concrete catalog entries. Acts as a
6	resolver for other skills and direct user requests. Covers Glue,
7	S3, S3 Tables, and Redshift. Optimized for low token usage — return the
8	answer fast and get out of the way.
9
10	Constraints for parameter acquisition:
11
12	- You MUST accept a single argument: table name, keyword, column name, or S3 path
13	- You MUST accept the argument as direct input or a pointer to a file containing the spec
14	- You MUST ask for the target AWS region if not already set
15	- You MUST confirm ambiguous input before searching (e.g., "Did you mean table X or bucket Y?")
16	- You MUST respect the user's decision to abort at any step
17
18	## Common Tasks
19
20	You MUST execute commands using AWS MCP server tools when connected — they
21	provide validation, sandboxed execution, and audit logging. Fall back to
22	AWS CLI only if MCP is unavailable. You MUST explain each step before
23	executing.
24
25	### 1. Verify Dependencies
26
27	Check for required tools and AWS access before searching.
28
29	Constraints:
30
31	- You MUST verify AWS MCP server tools (`aws___call_aws`) are available; fall back to AWS CLI if not
32	- You MUST confirm credentials with `aws sts get-caller-identity`
33	- You MUST inform the user about any missing tools and ask whether to proceed
34
35	### 2. Consult Catalog Context (experimental — suggested first lookup)
36
37	The customer may publish context skill assets in the Glue Data Catalog that map
38	their business language to the real tables — canonical names and aliases, join keys,
39	metrics, usage notes, descriptions — that the raw schema does not carry. When present,
40	this catalog is often enough to answer the request on its own.
41
42	These are the Glue Discovery operations (`SearchAssets` / `GetAsset` /
43	`ListIterableForms` / `BatchGetIterableForms`) — a distinct metadata-search surface,
44	NOT the legacy `glue search-tables` used in Step 5. They are experimental — not
45	available in every CLI build. Gate the lookup on two checks first:
46
47	1. Availability. Confirm the `GetAsset` operation exists in the caller's Glue
48	CLI model (redirect output so the CLI pager cannot block a non-interactive agent):
49
50	```
51	aws glue get-asset help > /dev/null 2>&1
52	# exit 0 = available. exit 2 (with "Invalid choice" in stderr) = not in this CLI (skip).
53	# any other non-zero (network/credential error) = inconclusive; treat as unavailable.
54	```
55
56	If it is not available, skip this step and go to the normal search workflow (Steps 3-7).
57	2. User opt-in. If available, ask the user: "I can check the Glue Data Catalog
58	for customer-authored context using an experimental SearchAssets/GetAsset API.
59	Use it? (yes/no)". Proceed only on an explicit yes; otherwise skip to Steps 3-7.
60
61	How this model differs: Discovery indexes assets (not databases/tables). Every
62	asset has an `Id` that is an ARN, and every lookup after `SearchAssets` keys off that ARN
63	via the identifier — there is no `--database-name`/`--table-name`. CLI flags are kebab-case
64	(`--search-text`, `--max-results`, `--filter-clause`); top-level response fields are PascalCase
65	(`Id`, `AssetName`, `Forms`). NOTE: a `*.Content` value is itself a JSON STRING with its own
66	camelCase schema (e.g. `dataLocation`, `dataFormat`, `isPartitionKey`) — parse it as embedded JSON,
67	do not expect PascalCase inside. The operations you need:
68
69	\| Operation \| Input → Output \|
70	\|---\|---\|
71	\| `search-assets` \| `--search-text` (+ optional `--filter-clause`) → `Items[]` of `{Id, AssetName, Type, Namespace, AssetTypeId, UpdatedAt}` (NOTE: search items do NOT include a description — call `get-asset` for `Description`/`Forms`) \|
72	\| `get-asset` \| `--identifier <Id, an ARN>` → one asset's `{Description, Forms, IterableForms}`. `Forms."amazon::Table".Content` is JSON `{dataLocation, dataFormat, type}`; advertises column availability via `IterableForms: {"columns": {...}}` \|
73	\| `list-iterable-forms` \| `--asset-identifier <table ARN> --iterable-form-name columns` → that table's columns `Items[]` of `{ItemId, ItemName, Description}` (ItemId = `<table-ARN>#<columnName>`) \|
74	\| `batch-get-iterable-forms` \| `--asset-identifier <table ARN> --iterable-form-name columns --item-identifiers <id1> <id2> ...` (space-separated) → `Items[]` of `{ItemName, Forms}` where `Forms.Column.Content` is JSON `{"type": "...", "isPartitionKey": ...}` \|

People who liked this also viewed

6 picks

Type

finding-data-lake-assets

2.4kinstalls190forks

View on GitHub

How to install finding-data-lake-assets?

aws/agent-toolkit-for-aws/finding-data-lake-assets

$npx -y skills add aws/agent-toolkit-for-aws --skill finding-data-lake-assets

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Find Data Lake Assets
2
3	## Overview
4
5	Resolves data lake asset references to concrete catalog entries. Acts as a
6	resolver for other skills and direct user requests. Covers Glue,
7	S3, S3 Tables, and Redshift. Optimized for low token usage — return the
8	answer fast and get out of the way.
9
10	Constraints for parameter acquisition:
11
12	- You MUST accept a single argument: table name, keyword, column name, or S3 path
13	- You MUST accept the argument as direct input or a pointer to a file containing the spec
14	- You MUST ask for the target AWS region if not already set
15	- You MUST confirm ambiguous input before searching (e.g., "Did you mean table X or bucket Y?")
16	- You MUST respect the user's decision to abort at any step
17
18	## Common Tasks
19
20	You MUST execute commands using AWS MCP server tools when connected — they
21	provide validation, sandboxed execution, and audit logging. Fall back to
22	AWS CLI only if MCP is unavailable. You MUST explain each step before
23	executing.
24
25	### 1. Verify Dependencies
26
27	Check for required tools and AWS access before searching.
28
29	Constraints:
30
31	- You MUST verify AWS MCP server tools (`aws___call_aws`) are available; fall back to AWS CLI if not
32	- You MUST confirm credentials with `aws sts get-caller-identity`
33	- You MUST inform the user about any missing tools and ask whether to proceed
34
35	### 2. Consult Catalog Context (experimental — suggested first lookup)
36
37	The customer may publish context skill assets in the Glue Data Catalog that map
38	their business language to the real tables — canonical names and aliases, join keys,
39	metrics, usage notes, descriptions — that the raw schema does not carry. When present,
40	this catalog is often enough to answer the request on its own.
41
42	These are the Glue Discovery operations (`SearchAssets` / `GetAsset` /
43	`ListIterableForms` / `BatchGetIterableForms`) — a distinct metadata-search surface,
44	NOT the legacy `glue search-tables` used in Step 5. They are experimental — not
45	available in every CLI build. Gate the lookup on two checks first:
46
47	1. Availability. Confirm the `GetAsset` operation exists in the caller's Glue
48	CLI model (redirect output so the CLI pager cannot block a non-interactive agent):
49
50	```
51	aws glue get-asset help > /dev/null 2>&1
52	# exit 0 = available. exit 2 (with "Invalid choice" in stderr) = not in this CLI (skip).
53	# any other non-zero (network/credential error) = inconclusive; treat as unavailable.
54	```
55
56	If it is not available, skip this step and go to the normal search workflow (Steps 3-7).
57	2. User opt-in. If available, ask the user: "I can check the Glue Data Catalog
58	for customer-authored context using an experimental SearchAssets/GetAsset API.
59	Use it? (yes/no)". Proceed only on an explicit yes; otherwise skip to Steps 3-7.
60
61	How this model differs: Discovery indexes assets (not databases/tables). Every
62	asset has an `Id` that is an ARN, and every lookup after `SearchAssets` keys off that ARN
63	via the identifier — there is no `--database-name`/`--table-name`. CLI flags are kebab-case
64	(`--search-text`, `--max-results`, `--filter-clause`); top-level response fields are PascalCase
65	(`Id`, `AssetName`, `Forms`). NOTE: a `*.Content` value is itself a JSON STRING with its own
66	camelCase schema (e.g. `dataLocation`, `dataFormat`, `isPartitionKey`) — parse it as embedded JSON,
67	do not expect PascalCase inside. The operations you need:
68
69	\| Operation \| Input → Output \|
70	\|---\|---\|
71	\| `search-assets` \| `--search-text` (+ optional `--filter-clause`) → `Items[]` of `{Id, AssetName, Type, Namespace, AssetTypeId, UpdatedAt}` (NOTE: search items do NOT include a description — call `get-asset` for `Description`/`Forms`) \|
72	\| `get-asset` \| `--identifier <Id, an ARN>` → one asset's `{Description, Forms, IterableForms}`. `Forms."amazon::Table".Content` is JSON `{dataLocation, dataFormat, type}`; advertises column availability via `IterableForms: {"columns": {...}}` \|
73	\| `list-iterable-forms` \| `--asset-identifier <table ARN> --iterable-form-name columns` → that table's columns `Items[]` of `{ItemId, ItemName, Description}` (ItemId = `<table-ARN>#<columnName>`) \|
74	\| `batch-get-iterable-forms` \| `--asset-identifier <table ARN> --iterable-form-name columns --item-identifiers <id1> <id2> ...` (space-separated) → `Items[]` of `{ItemName, Forms}` where `Forms.Column.Content` is JSON `{"type": "...", "isPartitionKey": ...}` \|

People who liked this also viewed

6 picks

Type