bymicrosoft· 555 skills
Query and analyze data in Azure Data Explorer (Kusto/ADX) using KQL for log analytics, telemetry, and time series analysis. WHEN: KQL queries, Kusto database queries, Azure Data Explorer, ADX clusters, log analytics, time series data, IoT telemetry, anomaly detection.
$npx -y skills add microsoft/azure-skills --skill azure-kustoInstalls into the current project.
Run `npx skills use "https://github.com/microsoft/azure-skills" --skill "microsoft/azure-skills/azure-kusto"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.
Use the skills in "https://github.com/microsoft/azure-skills" that are relevant to the current task. Run `npx skills add "https://github.com/microsoft/azure-skills"` and select the relevant skills, then follow their instructions.
| 1 | # Azure Data Explorer (Kusto) Query & Analytics |
| 2 | |
| 3 | Execute KQL queries and manage Azure Data Explorer resources for fast, scalable big data analytics on log, telemetry, and time series data. |
| 4 | |
| 5 | ## Skill Activation Triggers |
| 6 | |
| 7 | **Use this skill immediately when the user asks to:** |
| 8 | - "Query my Kusto database for [data pattern]" |
| 9 | - "Show me events in the last hour from Azure Data Explorer" |
| 10 | - "Analyze logs in my ADX cluster" |
| 11 | - "Run a KQL query on [database]" |
| 12 | - "What tables are in my Kusto database?" |
| 13 | - "Show me the schema for [table]" |
| 14 | - "List my Azure Data Explorer clusters" |
| 15 | - "Aggregate telemetry data by [dimension]" |
| 16 | - "Create a time series chart from my logs" |
| 17 | |
| 18 | **Key Indicators:** |
| 19 | - Mentions "Kusto", "Azure Data Explorer", "ADX", or "KQL" |
| 20 | - Log analytics or telemetry analysis requests |
| 21 | - Time series data exploration |
| 22 | - IoT data analysis queries |
| 23 | - SIEM or security analytics tasks |
| 24 | - Requests for data aggregation on large datasets |
| 25 | - Performance monitoring or APM queries |
| 26 | |
| 27 | ## Overview |
| 28 | |
| 29 | This skill enables querying and managing Azure Data Explorer (Kusto), a fast and highly scalable data exploration service optimized for log and telemetry data. Azure Data Explorer provides sub-second query performance on billions of records using the Kusto Query Language (KQL). |
| 30 | |
| 31 | Key capabilities: |
| 32 | - **Query Execution**: Run KQL queries against massive datasets |
| 33 | - **Schema Exploration**: Discover tables, columns, and data types |
| 34 | - **Resource Management**: List clusters and databases |
| 35 | - **Analytics**: Aggregations, time series, anomaly detection, machine learning |
| 36 | |
| 37 | ## Core Workflow |
| 38 | |
| 39 | 1. **Discover Resources**: List available clusters and databases in subscription |
| 40 | 2. **Explore Schema**: Retrieve table structures to understand data model |
| 41 | 3. **Query Data**: Execute KQL queries for analysis, filtering, aggregation |
| 42 | 4. **Analyze Results**: Process query output for insights and reporting |
| 43 | |
| 44 | ## Query Patterns |
| 45 | |
| 46 | ### Pattern 1: Basic Data Retrieval |
| 47 | Fetch recent records from a table with simple filtering. |
| 48 | |
| 49 | **Example KQL**: |
| 50 | ```kql |
| 51 | Events |
| 52 | | where Timestamp > ago(1h) |
| 53 | | take 100 |
| 54 | ``` |
| 55 | |
| 56 | **Use for**: Quick data inspection, recent event retrieval |
| 57 | |
| 58 | ### Pattern 2: Aggregation Analysis |
| 59 | Summarize data by dimensions for insights and reporting. |
| 60 | |
| 61 | **Example KQL**: |
| 62 | ```kql |
| 63 | Events |
| 64 | | summarize count() by EventType, bin(Timestamp, 1h) |
| 65 | | order by count_ desc |
| 66 | ``` |
| 67 | |
| 68 | **Use for**: Event counting, distribution analysis, top-N queries |
| 69 | |
| 70 | ### Pattern 3: Time Series Analytics |
| 71 | Analyze data over time windows for trends and patterns. |
| 72 | |
| 73 | **Example KQL**: |
| 74 | ```kql |
| 75 | Telemetry |
| 76 | | where Timestamp > ago(24h) |
| 77 | | summarize avg(ResponseTime), percentiles(ResponseTime, 50, 95, 99) by bin(Timestamp, 5m) |
| 78 | | render timechart |
| 79 | ``` |
| 80 | |
| 81 | **Use for**: Performance monitoring, trend analysis, anomaly detection |
| 82 | |
| 83 | ### Pattern 4: Join and Correlation |
| 84 | Combine multiple tables for cross-dataset analysis. |
| 85 | |
| 86 | **Example KQL**: |
| 87 | ```kql |
| 88 | Events |
| 89 | | where EventType == "Error" |
| 90 | | join kind=inner ( |
| 91 | Logs |
| 92 | | where Severity == "Critical" |
| 93 | ) on CorrelationId |
| 94 | | project Timestamp, EventType, LogMessage, Severity |
| 95 | ``` |
| 96 | |
| 97 | **Use for**: Root cause analysis, correlated event tracking |
| 98 | |
| 99 | ### Pattern 5: Schema Discovery |
| 100 | Explore table structure before querying. |
| 101 | |
| 102 | **Tools**: `kusto_table_schema_get` |
| 103 | |
| 104 | **Use for**: Understanding data model, query planning |
| 105 | |
| 106 | ## Key Data Fields |
| 107 | |
| 108 | When executing queries, common field patterns: |
| 109 | - **Timestamp**: Time of event (datetime) - use `ago()`, `between()`, `bin()` for time filtering |
| 110 | - **EventType/Category**: Classification field for grouping |
| 111 | - **CorrelationId/SessionId**: For tracing related events |
| 112 | - **Severity/Level**: For filtering by importance |
| 113 | - **Dimensions**: Custom properties for grouping and filtering |
| 114 | |
| 115 | ## Result Format |
| 116 | |
| 117 | Query results include: |
| 118 | - **Columns**: Field names and data types |
| 119 | - **Rows**: Data records matching query |
| 120 | - **Statistics**: Row count, execution time, resource utilization |
| 121 | - **Visualization**: Chart rendering hints (timechart, barchart, etc.) |
| 122 | |
| 123 | ## KQL Best Practices |
| 124 | |
| 125 | **🟢 Performance Optimized:** |
| 126 | - Filter early: Use `where` before joins and aggregations |
| 127 | - Limit result size: Use `take` or `limit` to reduce data transfer |
| 128 | - Time filters: Always filter by time range for time series data |
| 129 | - Indexed columns: Filter on indexed columns first |
| 130 | |
| 131 | **🔵 Query Patterns:** |
| 132 | - Use `summarize` for aggregations instead of `count()` alone |
| 133 | - Use `bin()` for time bucketing in time series |
| 134 | - Use `project` to select only needed columns |
| 135 | - Use `extend` to add calculated fields |
| 136 | |
| 137 | **🟡 Common Functions:** |
| 138 | - `ago(timespan)`: Relative time (ago(1h), ago(7d)) |
| 139 | - `between(start .. end)`: Range filtering |
| 140 | - `startswith()`, `contains()`, `matches regex`: String filtering |
| 141 | - `parse`, `extract`: Extract values from strings |
| 142 | - `percentiles()`, `avg()`, `sum()`, `max()`, `min()`: Aggregations |
| 143 | |
| 144 | ## Best Practices |
| 145 | |
| 146 | - Always include time range filters to optimize query performance |
| 147 | - Use `take` or `limit` for exploratory queries to avoid large result sets |
| 148 | - Leverage `summarize` for aggregations instead of client-side processing |
| 149 | - Store frequently-used queries as functions in the database |
| 150 | - Use materialized views for repeated aggregations |
| 151 | - Monitor query performance and resource consumption |
| 152 | - Apply data retention policies to manage storage costs |
| 153 | - Use streaming ingestion for real-time analytics (< 1 second latency) |
| 154 | - Integrate with Azure Monitor for operational insights |
| 155 | |
| 156 | ## MCP Tools Used |
| 157 | |
| 158 | | Tool | Purpose | |
| 159 | |------|---------| |
| 160 | | `kusto_cluster_list` | List all Azure Data Explorer clusters in a subscription | |
| 161 | | `kusto_database_list` | List all databases in a specific Kusto cluster | |
| 162 | | `kusto_query` | Execute KQL queries against a Kusto database | |
| 163 | | `kusto_table_schema_get` | Retrieve schema information for a specific table | |
| 164 | |
| 165 | **Required Parameters**: |
| 166 | - `subscription`: Azure subscription ID or display name |
| 167 | - `cluster`: Kusto cluster name (e.g., "mycluster") |
| 168 | - `database`: Database name |
| 169 | - `query`: KQL query string (for query operations) |
| 170 | - `table`: Table name (for schema operations) |
| 171 | |
| 172 | **Optional Parameters**: |
| 173 | - `resource-group`: Resource group name (for listing operations) |
| 174 | - `tenant`: Azure AD tenant ID |
| 175 | |
| 176 | ## Fallback Strategy: Azure CLI Commands |
| 177 | |
| 178 | If Azure MCP Kusto tools fail, timeout, or are unavailable, use Azure CLI commands as fallback. |
| 179 | |
| 180 | ### CLI Command Reference |
| 181 | |
| 182 | | Operation | Azure CLI Command | |
| 183 | |-----------|-------------------| |
| 184 | | List clusters | `az kusto cluster list --resource-group <rg-name>` | |
| 185 | | List databases | `az kusto database list --cluster-name <cluster> --resource-group <rg-name>` | |
| 186 | | Show cluster | `az kusto cluster show --name <cluster> --resource-group <rg-name>` | |
| 187 | | Show database | `az kusto database show --cluster-name <cluster> --database-name <db> --resource-group <rg-name>` | |
| 188 | |
| 189 | ### KQL Query via Azure CLI |
| 190 | |
| 191 | For queries, use the Kusto REST API or direct cluster URL: |
| 192 | ```bash |
| 193 | az rest --method post \ |
| 194 | --url "https://<cluster>.<region>.kusto.windows.net/v1/rest/query" \ |
| 195 | --body "{ \"db\": \"<database>\", \"csl\": \"<kql-query>\" }" |
| 196 | ``` |
| 197 | |
| 198 | ### When to Fallback |
| 199 | |
| 200 | Switch to Azure CLI when: |
| 201 | - MCP tool returns timeout error (queries > 60 seconds) |
| 202 | - MCP tool returns "service unavailable" or connection errors |
| 203 | - Authentication failures with MCP tools |
| 204 | - Empty response when database is known to have data |
| 205 | |
| 206 | ## Common Issues |
| 207 | |
| 208 | - **Access Denied**: Verify database permissions (Viewer role minimum for queries) |
| 209 | - **Query Timeout**: Optimize query with time filters, reduce result set, or increase timeout |
| 210 | - **Syntax Error**: Validate KQL syntax - common issues: missing pipes, incorrect operators |
| 211 | - **Empty Results**: Check time range filters (may be too restrictive), verify table name |
| 212 | - **Cluster Not Found**: Check cluster name format (exclude ".kusto.windows.net" suffix) |
| 213 | - **High CPU Usage**: Query too broad - add filters, reduce time range, limit aggregations |
| 214 | - **Ingestion Lag**: Streaming data may have 1-30 second delay depending on ingestion method |
| 215 | |
| 216 | ## Use Cases |
| 217 | |
| 218 | - **Log Analytics**: Application logs, system logs, audit logs |
| 219 | - **IoT Analytics**: Sensor data, device telemetry, real-time monitoring |
| 220 | - **Security Analytics**: SIEM data, threat detection, security event correlation |
| 221 | - **APM**: Application performance metrics, user behavior, error tracking |
| 222 | - **Business Intelligence**: Clickstream analysis, user analytics, operational KPIs |