log-analysis
installsforks
How to install log-analysis?
chaterm/terminal-skills/log-analysis
$npx -y skills add chaterm/terminal-skills --skill log-analysis日志分析与处理
SKILL.md
| 1 | # 日志分析与处理 |
| 2 | |
| 3 | ## 概述 |
| 4 | 日志聚合、分析工具、告警配置等技能。 |
| 5 | |
| 6 | ## 日志查看 |
| 7 | |
| 8 | ### 基础命令 |
| 9 | ```bash |
| 10 | # 实时查看 |
| 11 | tail -f /var/log/syslog |
| 12 | tail -f /var/log/nginx/access.log |
| 13 | |
| 14 | # 多文件同时查看 |
| 15 | tail -f /var/log/nginx/*.log |
| 16 | multitail /var/log/nginx/access.log /var/log/nginx/error.log |
| 17 | |
| 18 | # 查看最后N行 |
| 19 | tail -n 100 /var/log/syslog |
| 20 | |
| 21 | # 查看开头 |
| 22 | head -n 100 /var/log/syslog |
| 23 | |
| 24 | # 分页查看 |
| 25 | less /var/log/syslog |
| 26 | less +F /var/log/syslog # 类似 tail -f |
| 27 | ``` |
| 28 | |
| 29 | ### 按时间过滤 |
| 30 | ```bash |
| 31 | # 使用 sed 按时间范围 |
| 32 | sed -n '/2024-01-15 10:00/,/2024-01-15 11:00/p' /var/log/app.log |
| 33 | |
| 34 | # 使用 awk |
| 35 | awk '/2024-01-15 10:/ && /2024-01-15 11:/' /var/log/app.log |
| 36 | |
| 37 | # 使用 journalctl |
| 38 | journalctl --since "2024-01-15 10:00" --until "2024-01-15 11:00" |
| 39 | journalctl --since "1 hour ago" |
| 40 | journalctl --since today |
| 41 | ``` |
| 42 | |
| 43 | ## 文本搜索 |
| 44 | |
| 45 | ### grep |
| 46 | ```bash |
| 47 | # 基础搜索 |
| 48 | grep "error" /var/log/syslog |
| 49 | grep -i "error" /var/log/syslog # 忽略大小写 |
| 50 | grep -r "error" /var/log/ # 递归搜索 |
| 51 | |
| 52 | # 正则表达式 |
| 53 | grep -E "error|warning" /var/log/syslog |
| 54 | grep -P "\d{4}-\d{2}-\d{2}" /var/log/syslog # Perl 正则 |
| 55 | |
| 56 | # 上下文 |
| 57 | grep -A 3 "error" /var/log/syslog # 后3行 |
| 58 | grep -B 3 "error" /var/log/syslog # 前3行 |
| 59 | grep -C 3 "error" /var/log/syslog # 前后3行 |
| 60 | |
| 61 | # 统计 |
| 62 | grep -c "error" /var/log/syslog # 计数 |
| 63 | grep -l "error" /var/log/*.log # 只显示文件名 |
| 64 | |
| 65 | # 排除 |
| 66 | grep -v "debug" /var/log/syslog # 排除包含 debug 的行 |
| 67 | ``` |
| 68 | |
| 69 | ### ripgrep (rg) |
| 70 | ```bash |
| 71 | # 更快的搜索 |
| 72 | rg "error" /var/log/ |
| 73 | rg -i "error" /var/log/ # 忽略大小写 |
| 74 | rg -C 3 "error" /var/log/ # 上下文 |
| 75 | rg --type log "error" # 按文件类型 |
| 76 | ``` |
| 77 | |
| 78 | ## 文本处理 |
| 79 | |
| 80 | ### awk |
| 81 | ```bash |
| 82 | # 打印特定列 |
| 83 | awk '{print $1, $4}' /var/log/nginx/access.log |
| 84 | |
| 85 | # 条件过滤 |
| 86 | awk '$9 == 500' /var/log/nginx/access.log |
| 87 | awk '$9 >= 400 && $9 < 500' /var/log/nginx/access.log |
| 88 | |
| 89 | # 统计 |
| 90 | awk '{sum += $10} END {print sum}' /var/log/nginx/access.log |
| 91 | awk '{count[$9]++} END {for (c in count) print c, count[c]}' /var/log/nginx/access.log |
| 92 | |
| 93 | # 自定义分隔符 |
| 94 | awk -F: '{print $1}' /etc/passwd |
| 95 | awk -F'[ :]' '{print $1, $2}' /var/log/syslog |
| 96 | ``` |
| 97 | |
| 98 | ### sed |
| 99 | ```bash |
| 100 | # 替换 |
| 101 | sed 's/old/new/g' file.log |
| 102 | sed -i 's/old/new/g' file.log # 原地修改 |
| 103 | |
| 104 | # 删除行 |
| 105 | sed '/pattern/d' file.log |
| 106 | sed '1,10d' file.log # 删除前10行 |
| 107 | |
| 108 | # 提取行 |
| 109 | sed -n '10,20p' file.log # 打印10-20行 |
| 110 | sed -n '/start/,/end/p' file.log # 打印匹配范围 |
| 111 | ``` |
| 112 | |
| 113 | ### sort & uniq |
| 114 | ```bash |
| 115 | # 排序 |
| 116 | sort file.log |
| 117 | sort -r file.log # 逆序 |
| 118 | sort -n file.log # 数字排序 |
| 119 | sort -k2 file.log # 按第2列排序 |
| 120 | |
| 121 | # 去重统计 |
| 122 | sort file.log | uniq |
| 123 | sort file.log | uniq -c # 计数 |
| 124 | sort file.log | uniq -c | sort -rn # 按频率排序 |
| 125 | ``` |
| 126 | |
| 127 | ## Nginx 日志分析 |
| 128 | |
| 129 | ### 常用分析 |
| 130 | ```bash |
| 131 | # 访问量统计 |
| 132 | wc -l /var/log/nginx/access.log |
| 133 | |
| 134 | # IP 访问排行 |
| 135 | awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20 |
| 136 | |
| 137 | # 状态码统计 |
| 138 | awk '{print $9}' access.log | sort | uniq -c | sort -rn |
| 139 | |
| 140 | # URL 访问排行 |
| 141 | awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20 |
| 142 | |
| 143 | # 每小时访问量 |
| 144 | awk '{print substr($4,14,2)}' access.log | sort | uniq -c |
| 145 | |
| 146 | # 慢请求（响应时间 > 1s） |
| 147 | awk '$NF > 1' access.log | head -20 |
| 148 | |
| 149 | # 404 错误 |
| 150 | awk '$9 == 404 {print $7}' access.log | sort | uniq -c | sort -rn |
| 151 | |
| 152 | # 带宽统计 |
| 153 | awk '{sum += $10} END {print sum/1024/1024 " MB"}' access.log |
| 154 | ``` |
| 155 | |
| 156 | ### GoAccess 实时分析 |
| 157 | ```bash |
| 158 | # 安装 |
| 159 | apt install goaccess |
| 160 | |
| 161 | # 实时分析 |
| 162 | goaccess /var/log/nginx/access.log -c |
| 163 | |
| 164 | # 生成 HTML 报告 |
| 165 | goaccess /var/log/nginx/access.log -o report.html --log-format=COMBINED |
| 166 | |
| 167 | # 实时 HTML |
| 168 | goaccess /var/log/nginx/access.log -o /var/www/html/report.html --real-time-html |
| 169 | ``` |
| 170 | |
| 171 | ## 日志轮转 |
| 172 | |
| 173 | ### logrotate 配置 |
| 174 | ```bash |
| 175 | # /etc/logrotate.d/myapp |
| 176 | /var/log/myapp/*.log { |
| 177 | daily # 每天轮转 |
| 178 | rotate 7 # 保留7个 |
| 179 | compress # 压缩 |
| 180 | delaycompress # 延迟压缩 |
| 181 | missingok # 文件不存在不报错 |
| 182 | notifempty # 空文件不轮转 |
| 183 | create 0640 www-data www-data # 创建新文件 |
| 184 | sharedscripts |
| 185 | postrotate |
| 186 | systemctl reload nginx > /dev/null 2>&1 || true |
| 187 | endscript |
| 188 | } |
| 189 | ``` |
| 190 | |
| 191 | ### 手动轮转 |
| 192 | ```bash |
| 193 | # 测试配置 |
| 194 | logrotate -d /etc/logrotate.d/myapp |
| 195 | |
| 196 | # 强制轮转 |
| 197 | logrotate -f /etc/logrotate.d/myapp |
| 198 | ``` |
| 199 | |
| 200 | ## 常见场景 |
| 201 | |
| 202 | ### 场景 1：错误日志分析 |
| 203 | ```bash |
| 204 | # 统计错误类型 |
| 205 | grep -E "ERROR|WARN|FATAL" /var/log/app.log | \ |
| 206 | awk '{print $3}' | sort | uniq -c | sort -rn |
| 207 | |
| 208 | # 最近1小时的错误 |
| 209 | awk -v date="$(date -d '1 hour ago' '+%Y-%m-%d %H')" \ |
| 210 | '$0 ~ date && /ERROR/' /var/log/app.log |
| 211 | ``` |
| 212 | |
| 213 | ### 场景 2：性能分析 |
| 214 | ```bash |
| 215 | # 响应时间分布 |
| 216 | awk '{print int($NF)}' access.log | sort -n | uniq -c | \ |
| 217 | awk '{printf "%ds: %d\n", $2, $1}' |
| 218 | |
| 219 | # 慢请求 Top 10 |
| 220 | awk '{print $NF, $7}' access.log | sort -rn | head -10 |
| 221 | ``` |
| 222 | |
| 223 | ### 场景 3：安全分析 |
| 224 | ```bash |
| 225 | # 失败登录尝试 |
| 226 | grep "Failed password" /var/log/auth.log | \ |
| 227 | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn |
| 228 | |
| 229 | # 可疑 IP |
| 230 | awk '$9 ~ /4[0-9][0-9]/ {print $1}' access.log | \ |
| 231 | sort | uniq -c | sort -rn | head -20 |
| 232 | ``` |
| 233 | |
| 234 | ## 故障排查 |
| 235 | |
| 236 | | 问题 | 排查方法 | |
| 237 | |------|----------| |
| 238 | | 日志太大 | 配置 logrotate、按时间过滤 | |
| 239 | | 搜索慢 | 使用 ripgrep、建立索引 | |
| 240 | | 格式不统一 | 使用 awk 自定义解析 | |
| 241 | | 实时监控 | tail -f、GoAccess | |
| 242 | | 历史分析 | ELK Stack、Loki | |
People who liked this also viewed
6 picksType
- microsoft-foundryDeploy, evaluate, fine-tune, and manage Foundry agents end-to-end with azd: hosted agent scaffold/run/deploy, prompt agent create, batch eval, continuous eval, prompt optimizer, Agent Optimizer scaffoSkillsJul 2026
- azure-deployExecute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application SkillsJul 2026
- azure-preparePrepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wSkillsJul 2026
- azure-validate--- name: azure-validate description: "Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity pSkillsJul 2026
- azure-aigatewayConfigure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbSkillsJul 2026
- azure-hosted-copilot-sdkOfficial agent plugin providing skills and MCP server configurations for Azure scenarios.SkillsJul 2026
