shopify-admin-agentic-crawler-access

Name: shopify-admin-agentic-crawler-access
Author: 40RTY-ai

168stars17forks

How to install shopify-admin-agentic-crawler-access?

40rty-ai/shopify-admin-skills/shopify-admin-agentic-crawler-access

$npx -y skills add 40RTY-ai/shopify-admin-skills --skill shopify-admin-agentic-crawler-access

Installs into the current project.

Summary

Edit the theme's robots.txt.liquid to explicitly allow AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Amazonbot) so AI assistants are permitted to read the catalog.

Files · 1

View on GitHub

SKILL.md

1	## Purpose
2	If your `robots.txt` doesn't address the AI crawlers by name — or worse, disallows them — assistants like ChatGPT, Claude, and Perplexity may never read your store, so you simply don't exist to AI shoppers. Shopify generates `robots.txt` from a theme template (`templates/robots.txt.liquid`); this skill reads that template and appends explicit allow rules for the major AI user-agents (with a clear, idempotent managed block) so foundational models and AI search are permitted to crawl your pages. Fixes the agentiq.report finding `robots-ai-rules`.
3
4	## Prerequisites
5	- Authenticated Shopify CLI session (`shopify auth login --store <domain>`)
6	- Required API scopes: `read_themes`, `write_themes`
7
8	## Parameters
9	All skills accept these universal parameters:
10
11	\| Parameter \| Type \| Required \| Default \| Description \|
12	\|-----------\|--------\|----------\|---------\|-------------\|
13	\| store \| string \| yes \| — \| Store domain (e.g., mystore.myshopify.com) \|
14	\| format \| string \| no \| human \| Output format: `human` (default) or `json` \|
15	\| dry_run \| bool \| no \| true \| Preview the new robots.txt.liquid without writing (defaults ON — this edits the live theme) \|
16
17	Skill-specific parameters:
18
19	\| Parameter \| Type \| Required \| Default \| Description \|
20	\|-----------\|------\|----------\|---------\|-------------\|
21	\| theme_id \| string \| no \| — \| Theme GID to edit (defaults to the published MAIN theme) \|
22	\| agents \| string \| no \| GPTBot,OAI-SearchBot,ChatGPT-User,ClaudeBot,Claude-Web,PerplexityBot,Perplexity-User,Google-Extended,Amazonbot,Applebot-Extended \| Comma list of AI user-agents to allow \|
23	\| mode \| string \| no \| allow \| `allow` (add Allow rules) or `audit` (report current state only, no write) \|
24
25	## Safety
26
27	> ⚠️ Step 2 (`themeFilesUpsert`) modifies a file in the LIVE published theme. A malformed `robots.txt.liquid` can de-index the store from ALL search engines. This skill writes only inside a clearly-delimited `# BEGIN agentic-crawler-access … # END` managed block (re-running replaces just that block, never your other rules), defaults `dry_run: true`, and recommends duplicating the theme first. Review the full proposed file before setting `dry_run: false`.
28
29	## Workflow Steps
30
31	1. OPERATION: `themes` — query
32	Inputs: `roles: [MAIN]` (unless `theme_id` given), then `theme.files(filenames: ["templates/robots.txt.liquid"])`
33	Expected output: Current `robots.txt.liquid` content (or absence — Shopify uses a default if the template doesn't exist).
34
35	2. COMPUTE (no API): construct the new template. If no custom template exists, start from the Shopify default (`{{ robots.default_groups }}`) and append the managed block with `User-agent:`/`Allow: /` stanzas for each agent in `agents`. If a managed block already exists, replace only it. Emit the full proposed file diff.
36
37	3. OPERATION: `themeFilesUpsert` — mutation
38	Inputs: `themeId`, `files: [{ filename: "templates/robots.txt.liquid", body: { type: TEXT, value: <new content> } }]` (skipped when `dry_run` or `mode: audit`).
39	Expected output: Upserted theme file; collect `userErrors`.
40
41	## GraphQL Operations
42
43	```graphql
44	# themes:query — validated against api_version 2025-01
45	query CrawlerThemeFile {
46	themes(first: 1, roles: [MAIN]) {
47	nodes {
48	id
49	name
50	files(filenames: ["templates/robots.txt.liquid"]) {
51	nodes {
52	filename
53	body { ... on OnlineStoreThemeFileBodyText { content } }
54	}
55	}
56	}
57	}
58	}
59	```
60
61	```graphql
62	# themeFilesUpsert:mutation — validated against api_version 2025-01
63	mutation CrawlerAllowUpsert($themeId: ID!, $files: [OnlineStoreThemeFilesUpsertFileInput!]!) {
64	themeFilesUpsert(themeId: $themeId, files: $files) {
65	upsertedThemeFiles { filename }
66	userErrors { filename code message }
67	}
68	}
69	```
70
71	Managed block written into `templates/robots.txt.liquid` (after `{{ robots.default_groups }}`):
72
73	```liquid
74	# BEGIN agentic-crawler-access (managed by shopify-admin-agentic-crawler-access)
75	User-agent: GPTBot
76	Allow: /
77	User-agent: ClaudeBot
78	Allow: /
79	User-agent: PerplexityBot
80	Allow: /
81	User-agent: Google-Extended
82	Allow: /
83	# ... one stanza per agent in `agents`
84	# END agentic-crawler-access
85	```
86
87	## Session Tracking
88
89	Claude MUST emit the following output at each stage. This is mandatory.
90
91	On start, emit:
92	```
93	╔══════════════════════════════════════════════╗
94	║ SKILL: <skill name> ║
95	║ Store: <store domain>

People who liked this also viewed

6 picks

Type

shopify-admin-agentic-crawler-access

168stars17forks

View on GitHub

How to install shopify-admin-agentic-crawler-access?

40rty-ai/shopify-admin-skills/shopify-admin-agentic-crawler-access

$npx -y skills add 40RTY-ai/shopify-admin-skills --skill shopify-admin-agentic-crawler-access

Installs into the current project.

Summary

Edit the theme's robots.txt.liquid to explicitly allow AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Amazonbot) so AI assistants are permitted to read the catalog.

Files · 1

View on GitHub

SKILL.md

1	## Purpose
2	If your `robots.txt` doesn't address the AI crawlers by name — or worse, disallows them — assistants like ChatGPT, Claude, and Perplexity may never read your store, so you simply don't exist to AI shoppers. Shopify generates `robots.txt` from a theme template (`templates/robots.txt.liquid`); this skill reads that template and appends explicit allow rules for the major AI user-agents (with a clear, idempotent managed block) so foundational models and AI search are permitted to crawl your pages. Fixes the agentiq.report finding `robots-ai-rules`.
3
4	## Prerequisites
5	- Authenticated Shopify CLI session (`shopify auth login --store <domain>`)
6	- Required API scopes: `read_themes`, `write_themes`
7
8	## Parameters
9	All skills accept these universal parameters:
10
11	\| Parameter \| Type \| Required \| Default \| Description \|
12	\|-----------\|--------\|----------\|---------\|-------------\|
13	\| store \| string \| yes \| — \| Store domain (e.g., mystore.myshopify.com) \|
14	\| format \| string \| no \| human \| Output format: `human` (default) or `json` \|
15	\| dry_run \| bool \| no \| true \| Preview the new robots.txt.liquid without writing (defaults ON — this edits the live theme) \|
16
17	Skill-specific parameters:
18
19	\| Parameter \| Type \| Required \| Default \| Description \|
20	\|-----------\|------\|----------\|---------\|-------------\|
21	\| theme_id \| string \| no \| — \| Theme GID to edit (defaults to the published MAIN theme) \|
22	\| agents \| string \| no \| GPTBot,OAI-SearchBot,ChatGPT-User,ClaudeBot,Claude-Web,PerplexityBot,Perplexity-User,Google-Extended,Amazonbot,Applebot-Extended \| Comma list of AI user-agents to allow \|
23	\| mode \| string \| no \| allow \| `allow` (add Allow rules) or `audit` (report current state only, no write) \|
24
25	## Safety
26
27	> ⚠️ Step 2 (`themeFilesUpsert`) modifies a file in the LIVE published theme. A malformed `robots.txt.liquid` can de-index the store from ALL search engines. This skill writes only inside a clearly-delimited `# BEGIN agentic-crawler-access … # END` managed block (re-running replaces just that block, never your other rules), defaults `dry_run: true`, and recommends duplicating the theme first. Review the full proposed file before setting `dry_run: false`.
28
29	## Workflow Steps
30
31	1. OPERATION: `themes` — query
32	Inputs: `roles: [MAIN]` (unless `theme_id` given), then `theme.files(filenames: ["templates/robots.txt.liquid"])`
33	Expected output: Current `robots.txt.liquid` content (or absence — Shopify uses a default if the template doesn't exist).
34
35	2. COMPUTE (no API): construct the new template. If no custom template exists, start from the Shopify default (`{{ robots.default_groups }}`) and append the managed block with `User-agent:`/`Allow: /` stanzas for each agent in `agents`. If a managed block already exists, replace only it. Emit the full proposed file diff.
36
37	3. OPERATION: `themeFilesUpsert` — mutation
38	Inputs: `themeId`, `files: [{ filename: "templates/robots.txt.liquid", body: { type: TEXT, value: <new content> } }]` (skipped when `dry_run` or `mode: audit`).
39	Expected output: Upserted theme file; collect `userErrors`.
40
41	## GraphQL Operations
42
43	```graphql
44	# themes:query — validated against api_version 2025-01
45	query CrawlerThemeFile {
46	themes(first: 1, roles: [MAIN]) {
47	nodes {
48	id
49	name
50	files(filenames: ["templates/robots.txt.liquid"]) {
51	nodes {
52	filename
53	body { ... on OnlineStoreThemeFileBodyText { content } }
54	}
55	}
56	}
57	}
58	}
59	```
60
61	```graphql
62	# themeFilesUpsert:mutation — validated against api_version 2025-01
63	mutation CrawlerAllowUpsert($themeId: ID!, $files: [OnlineStoreThemeFilesUpsertFileInput!]!) {
64	themeFilesUpsert(themeId: $themeId, files: $files) {
65	upsertedThemeFiles { filename }
66	userErrors { filename code message }
67	}
68	}
69	```
70
71	Managed block written into `templates/robots.txt.liquid` (after `{{ robots.default_groups }}`):
72
73	```liquid
74	# BEGIN agentic-crawler-access (managed by shopify-admin-agentic-crawler-access)
75	User-agent: GPTBot
76	Allow: /
77	User-agent: ClaudeBot
78	Allow: /
79	User-agent: PerplexityBot
80	Allow: /
81	User-agent: Google-Extended
82	Allow: /
83	# ... one stanza per agent in `agents`
84	# END agentic-crawler-access
85	```
86
87	## Session Tracking
88
89	Claude MUST emit the following output at each stage. This is mandatory.
90
91	On start, emit:
92	```
93	╔══════════════════════════════════════════════╗
94	║ SKILL: <skill name> ║
95	║ Store: <store domain>

People who liked this also viewed

6 picks

Type