$npx -y skills add BagelHole/DevOps-Security-Agent-Skills --skill gcp-audit-logsConfigure GCP Cloud Audit Logs for compliance. Set up log routing and BigQuery analysis. Use when auditing GCP activity.
| 1 | # GCP Audit Logs |
| 2 | |
| 3 | Audit GCP activity with Cloud Audit Logs for compliance, security investigation, and operational monitoring. |
| 4 | |
| 5 | ## When to Use |
| 6 | |
| 7 | - Enabling organization-wide audit logging across GCP projects |
| 8 | - Meeting compliance requirements for SOC 2, HIPAA, PCI DSS, or FedRAMP |
| 9 | - Investigating unauthorized access or suspicious API activity |
| 10 | - Setting up alerting on administrative and data access events |
| 11 | - Exporting logs to BigQuery for long-term analysis and reporting |
| 12 | |
| 13 | ## Audit Log Types |
| 14 | |
| 15 | ```yaml |
| 16 | log_types: |
| 17 | admin_activity: |
| 18 | description: API calls that modify resource configuration or metadata |
| 19 | enabled: Always (cannot be disabled) |
| 20 | retention: 400 days (default) |
| 21 | cost: No charge |
| 22 | examples: |
| 23 | - Creating or deleting VM instances |
| 24 | - Changing IAM policies |
| 25 | - Modifying firewall rules |
| 26 | |
| 27 | data_access: |
| 28 | description: API calls that read resource configuration, metadata, or user data |
| 29 | enabled: Must be explicitly enabled (except BigQuery) |
| 30 | retention: 30 days (default) |
| 31 | cost: Can be significant at high volume |
| 32 | subtypes: |
| 33 | ADMIN_READ: Read resource configuration/metadata |
| 34 | DATA_READ: Read user-provided data |
| 35 | DATA_WRITE: Write user-provided data |
| 36 | |
| 37 | system_event: |
| 38 | description: Actions performed by GCP systems on behalf of resources |
| 39 | enabled: Always (cannot be disabled) |
| 40 | retention: 400 days (default) |
| 41 | cost: No charge |
| 42 | examples: |
| 43 | - Live migration of VM instances |
| 44 | - Automatic scaling events |
| 45 | |
| 46 | policy_denied: |
| 47 | description: Actions denied by VPC Service Controls or organization policies |
| 48 | enabled: Always (cannot be disabled) |
| 49 | retention: 400 days (default) |
| 50 | cost: No charge |
| 51 | ``` |
| 52 | |
| 53 | ## Enable Data Access Logs for an Organization |
| 54 | |
| 55 | ```bash |
| 56 | # Get current org IAM policy |
| 57 | gcloud organizations get-iam-policy ORG_ID --format=json > org-policy.json |
| 58 | |
| 59 | # Add audit config to org-policy.json: |
| 60 | # { |
| 61 | # "auditConfigs": [ |
| 62 | # { |
| 63 | # "service": "allServices", |
| 64 | # "auditLogConfigs": [ |
| 65 | # {"logType": "ADMIN_READ"}, |
| 66 | # {"logType": "DATA_READ"}, |
| 67 | # {"logType": "DATA_WRITE"} |
| 68 | # ] |
| 69 | # } |
| 70 | # ], |
| 71 | # ...existing bindings... |
| 72 | # } |
| 73 | |
| 74 | # Apply the updated policy |
| 75 | gcloud organizations set-iam-policy ORG_ID org-policy.json |
| 76 | |
| 77 | # Enable data access logs for specific services at project level |
| 78 | gcloud projects get-iam-policy PROJECT_ID --format=json > project-policy.json |
| 79 | |
| 80 | # Example: enable only for Cloud Storage and BigQuery |
| 81 | # { |
| 82 | # "auditConfigs": [ |
| 83 | # { |
| 84 | # "service": "storage.googleapis.com", |
| 85 | # "auditLogConfigs": [ |
| 86 | # {"logType": "DATA_READ"}, |
| 87 | # {"logType": "DATA_WRITE"} |
| 88 | # ] |
| 89 | # }, |
| 90 | # { |
| 91 | # "service": "bigquery.googleapis.com", |
| 92 | # "auditLogConfigs": [ |
| 93 | # {"logType": "DATA_READ"}, |
| 94 | # {"logType": "DATA_WRITE"} |
| 95 | # ] |
| 96 | # } |
| 97 | # ] |
| 98 | # } |
| 99 | |
| 100 | gcloud projects set-iam-policy PROJECT_ID project-policy.json |
| 101 | ``` |
| 102 | |
| 103 | ## Configure Log Sinks for Export |
| 104 | |
| 105 | ```bash |
| 106 | # Create BigQuery dataset for audit log export |
| 107 | bq mk --dataset \ |
| 108 | --description "Audit log export" \ |
| 109 | --default_table_expiration 0 \ |
| 110 | --location US \ |
| 111 | PROJECT_ID:audit_logs |
| 112 | |
| 113 | # Create organization-level log sink to BigQuery |
| 114 | gcloud logging sinks create org-audit-bigquery \ |
| 115 | bigquery.googleapis.com/projects/PROJECT_ID/datasets/audit_logs \ |
| 116 | --organization=ORG_ID \ |
| 117 | --include-children \ |
| 118 | --log-filter='logName:"cloudaudit.googleapis.com"' |
| 119 | |
| 120 | # Get the sink writer identity and grant BigQuery access |
| 121 | SINK_SA=$(gcloud logging sinks describe org-audit-bigquery \ |
| 122 | --organization=ORG_ID --format='value(writerIdentity)') |
| 123 | |
| 124 | bq add-iam-policy-binding \ |
| 125 | --member="$SINK_SA" \ |
| 126 | --role="roles/bigquery.dataEditor" \ |
| 127 | PROJECT_ID:audit_logs |
| 128 | |
| 129 | # Create Cloud Storage sink for long-term archive |
| 130 | gsutil mb -l US -b on gs://org-audit-logs-archive |
| 131 | gsutil retention set 7y gs://org-audit-logs-archive |
| 132 | |
| 133 | gcloud logging sinks create org-audit-storage \ |
| 134 | storage.googleapis.com/org-audit-logs-archive \ |
| 135 | --organization=ORG_ID \ |
| 136 | --include-children \ |
| 137 | --log-filter='logName:"cloudaudit.googleapis.com"' |
| 138 | |
| 139 | STORAGE_SA=$(gcloud logging sinks describe org-audit-storage \ |
| 140 | --organization=ORG_ID --format='value(writerIdentity)') |
| 141 | |
| 142 | gsutil iam ch "$STORAGE_SA:objectCreator" gs://org-audit-logs-archive |
| 143 | |
| 144 | # Create Pub/Sub sink for real-time streaming to SIEM |
| 145 | gcloud pubsub topics create audit-log-stream |
| 146 | |
| 147 | gcloud logging sinks create org-audit-pubsub \ |
| 148 | pubsub.googleapis.com/projects/PROJECT_ID/topics/audit-log-stream \ |
| 149 | --organization=ORG_ID \ |
| 150 | --include-children \ |
| 151 | --log-filter='logName:"cloudaudit.googleapis.com" AND (protoPayload.methodName:"delete" OR protoPayload.methodName:"setIamPolicy" OR severity>=WARNING)' |
| 152 | |
| 153 | PUBSUB_SA=$(gcloud logging sinks describe org-audit-pubsub \ |
| 154 | --organization=ORG_ID --format='value(writerIdentity)') |
| 155 | |
| 156 | gcloud pubsub topics add-i |