$npx -y skills add aws-samples/sample-strands-agents-agentskills --skill file-processingProcess and analyze CSV, JSON, and text files with data transformation, cleaning, analysis, and visualization capabilities
| 1 | # File Processing Skill |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | Process structured data files (CSV, JSON, text) with comprehensive capabilities for data cleaning, transformation, analysis, and export. This skill enables working with data files without requiring users to write code. |
| 6 | |
| 7 | ## When to Use This Skill |
| 8 | |
| 9 | Use this skill when you need to: |
| 10 | - Load and parse CSV or JSON files |
| 11 | - Clean and transform data |
| 12 | - Perform statistical analysis |
| 13 | - Filter, sort, or aggregate data |
| 14 | - Merge or join datasets |
| 15 | - Convert between formats (CSV ↔ JSON) |
| 16 | - Generate summary reports |
| 17 | |
| 18 | ## Capabilities |
| 19 | |
| 20 | ### 1. Data Loading |
| 21 | |
| 22 | Supported formats: |
| 23 | - **CSV files**: Any delimiter (comma, tab, semicolon, etc.) |
| 24 | - **JSON files**: Single objects or arrays of objects |
| 25 | - **Text files**: Custom delimited formats |
| 26 | |
| 27 | ### 2. Data Cleaning |
| 28 | |
| 29 | Available operations: |
| 30 | - Remove duplicate rows |
| 31 | - Handle missing values (drop, fill, interpolate) |
| 32 | - Normalize text (trim whitespace, standardize case) |
| 33 | - Convert data types |
| 34 | - Remove outliers |
| 35 | - Validate data against rules |
| 36 | |
| 37 | ### 3. Data Transformation |
| 38 | |
| 39 | Available operations: |
| 40 | - **Filter**: Select rows based on conditions |
| 41 | - **Select**: Choose specific columns |
| 42 | - **Sort**: Order by one or more columns |
| 43 | - **Group**: Aggregate data by categories |
| 44 | - **Pivot**: Reshape data (wide ↔ long format) |
| 45 | - **Merge**: Combine multiple datasets |
| 46 | - **Calculate**: Add derived columns |
| 47 | |
| 48 | ### 4. Data Analysis |
| 49 | |
| 50 | Available analyses: |
| 51 | - Descriptive statistics (mean, median, std, etc.) |
| 52 | - Frequency distributions |
| 53 | - Correlation analysis |
| 54 | - Trend detection |
| 55 | - Missing data analysis |
| 56 | - Data quality assessment |
| 57 | |
| 58 | ### 5. Export |
| 59 | |
| 60 | Output formats: |
| 61 | - CSV files |
| 62 | - JSON files (objects or arrays) |
| 63 | - Markdown tables |
| 64 | - Summary reports |
| 65 | |
| 66 | ## Instructions for Execution |
| 67 | |
| 68 | When this skill is activated, follow these steps: |
| 69 | |
| 70 | ### Step 1: Understand the Request |
| 71 | |
| 72 | Ask clarifying questions if needed: |
| 73 | - What file(s) need to be processed? |
| 74 | - What specific analysis or transformation is required? |
| 75 | - What output format is desired? |
| 76 | - Are there any specific requirements or constraints? |
| 77 | |
| 78 | ### Step 2: Load the Data |
| 79 | |
| 80 | Use `shell` to load and process data: |
| 81 | |
| 82 | ```python |
| 83 | # For CSV files |
| 84 | import csv |
| 85 | |
| 86 | # Read from file path |
| 87 | with open('data.csv', 'r') as f: |
| 88 | reader = csv.DictReader(f) |
| 89 | data = list(reader) |
| 90 | |
| 91 | # For JSON files |
| 92 | import json |
| 93 | with open('data.json', 'r') as f: |
| 94 | data = json.load(f) |
| 95 | ``` |
| 96 | |
| 97 | Alternatively, use the supporting scripts: |
| 98 | ```python |
| 99 | # Execute the helper script |
| 100 | ("scripts/process.py") |
| 101 | ``` |
| 102 | |
| 103 | ### Step 3: Perform Operations |
| 104 | |
| 105 | Apply the requested transformations or analyses: |
| 106 | |
| 107 | ```python |
| 108 | # Example: Filter and aggregate |
| 109 | filtered = [row for row in data if float(row['amount']) > 100] |
| 110 | |
| 111 | # Example: Calculate statistics |
| 112 | from statistics import mean, median |
| 113 | amounts = [float(row['amount']) for row in data] |
| 114 | avg = mean(amounts) |
| 115 | med = median(amounts) |
| 116 | ``` |
| 117 | |
| 118 | ### Step 4: Generate Output |
| 119 | |
| 120 | Format results according to user needs: |
| 121 | |
| 122 | ```python |
| 123 | # As markdown table |
| 124 | def to_markdown_table(data, columns=None): |
| 125 | if not data: |
| 126 | return "No data" |
| 127 | |
| 128 | if columns is None: |
| 129 | columns = list(data[0].keys()) |
| 130 | |
| 131 | # Header |
| 132 | header = "| " + " | ".join(columns) + " |" |
| 133 | separator = "| " + " | ".join(["---"] * len(columns)) + " |" |
| 134 | |
| 135 | # Rows |
| 136 | rows = [] |
| 137 | for row in data: |
| 138 | row_str = "| " + " | ".join(str(row.get(col, "")) for col in columns) + " |" |
| 139 | rows.append(row_str) |
| 140 | |
| 141 | return "\n".join([header, separator] + rows) |
| 142 | |
| 143 | print(to_markdown_table(filtered)) |
| 144 | ``` |
| 145 | |
| 146 | ## Common Use Cases |
| 147 | |
| 148 | ### Use Case 1: CSV Analysis |
| 149 | |
| 150 | ```python |
| 151 | # Example: Analyze sales data |
| 152 | import csv |
| 153 | from io import StringIO |
| 154 | from statistics import mean, sum as total |
| 155 | |
| 156 | # Load CSV |
| 157 | reader = csv.DictReader(StringIO(file_content)) |
| 158 | data = list(reader) |
| 159 | |
| 160 | # Calculate metrics |
| 161 | total_sales = sum(float(row['amount']) for row in data) |
| 162 | avg_sales = mean(float(row['amount']) for row in data) |
| 163 | unique_customers = len(set(row['customer_id'] for row in data)) |
| 164 | |
| 165 | print(f"Total Sales: ${total_sales:,.2f}") |
| 166 | print(f"Average Sale: ${avg_sales:,.2f}") |
| 167 | print(f"Unique Customers: {unique_customers}") |
| 168 | ``` |
| 169 | |
| 170 | ### Use Case 2: Data Filtering |
| 171 | |
| 172 | ```python |
| 173 | # Example: Filter records by criteria |
| 174 | filtered = [ |
| 175 | row for row in data |
| 176 | if row['status'] == 'active' and float(row['score']) >= 80 |
| 177 | ] |
| 178 | |
| 179 | print(f"Found {len(filtered)} matching records") |
| 180 | ``` |
| 181 | |
| 182 | ### Use Case 3: Data Grouping |
| 183 | |
| 184 | ```python |
| 185 | # Example: Group and aggregate |
| 186 | from collections import defaultdict |
| 187 | |
| 188 | grouped = defaultdict(list) |
| 189 | for row in data: |
| 190 | grouped[row['category']].append(float(row['value'])) |
| 191 | |
| 192 | summary = {} |
| 193 | for category, values in grouped.items(): |
| 194 | summary[category] = { |
| 195 | 'count': len(values), |
| 196 | 'total': sum(values), |
| 197 | 'average': sum(values) / len(values) |
| 198 | } |
| 199 | |
| 200 | for category, stats in summary.items(): |
| 201 | print(f"{category}: {stats['count']} items, avg = {stats['average']:.2f}") |
| 202 | ``` |
| 203 | |
| 204 | ### Use Case 4: Format Conversion |
| 205 | |
| 206 | ```python |
| 207 | # Example: CSV to JSON |
| 208 | impor |