llmops-platform-engineering

Name: llmops-platform-engineering
Author: BagelHole

42stars4forks

How to install llmops-platform-engineering?

bagelhole/devops-security-agent-skills/llmops-platform-engineering

$npx -y skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering

Installs into the current project.

Summary

Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.

Files · 1

View on GitHub

SKILL.md

1	# LLMOps Platform Engineering
2
3	Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
4
5	## When to Use This Skill
6
7	- Building an internal platform for teams to deploy and manage LLM-powered features
8	- Designing CI/CD pipelines that include model evaluation gates
9	- Setting up A/B testing infrastructure for model versions
10	- Creating Kubernetes-based model serving infrastructure
11	- Establishing governance workflows for model promotion
12
13	## Prerequisites
14
15	- Kubernetes cluster with GPU node pools (or cloud inference API access)
16	- Container registry (Harbor, ECR, GCR, or ACR)
17	- CI/CD system (GitHub Actions, GitLab CI, or Argo Workflows)
18	- Observability stack (Prometheus + Grafana + OpenTelemetry)
19	- Model registry (MLflow or custom metadata store)
20
21	## Outcomes
22
23	- Standardized path from experiment to production
24	- Safe model rollout with quality and safety gates
25	- Repeatable infra modules for inference, vector DB, and observability
26	- Clear ownership model across platform, app, and security teams
27
28	## Reference Architecture
29
30	1. Control Plane: model registry, prompt/version catalog, policy checks, eval pipeline.
31	2. Data Plane: inference gateway, vector database, cache, feature store.
32	3. Ops Plane: telemetry, alerting, SLO dashboards, cost analytics.
33	4. Security Plane: IAM boundaries, secret rotation, content filters, audit logs.
34
35	## Model Promotion Pipeline
36
37	```yaml
38	# .github/workflows/model-promotion.yaml
39	name: Model Promotion Pipeline
40	on:
41	workflow_dispatch:
42	inputs:
43	model_name:
44	description: "Model identifier"
45	required: true
46	model_version:
47	description: "Model version to promote"
48	required: true
49	target_env:
50	description: "Target environment"
51	required: true
52	type: choice
53	options: [staging, production]
54
55	jobs:
56	evaluate:
57	runs-on: ubuntu-latest
58	steps:
59	- uses: actions/checkout@v4
60
61	- name: Run quality evaluation suite
62	run: \|
63	python -m evals.run \
64	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
65	--suite quality \
66	--output results/quality.json
67
68	- name: Run safety evaluation suite
69	run: \|
70	python -m evals.run \
71	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
72	--suite safety \
73	--output results/safety.json
74
75	- name: Run latency benchmark
76	run: \|
77	python -m evals.benchmark \
78	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
79	--concurrent-users 50 \
80	--duration 300 \
81	--output results/latency.json
82
83	- name: Gate check - quality
84	run: \|
85	python -m evals.gate_check \
86	--results results/quality.json \
87	--threshold-file thresholds/quality.yaml
88
89	- name: Gate check - safety
90	run: \|
91	python -m evals.gate_check \
92	--results results/safety.json \
93	--threshold-file thresholds/safety.yaml
94
95	- name: Gate check - latency
96	run: \|
97	python -m evals.gate_check \
98	--results results/latency.json \
99	--threshold-file thresholds/latency.yaml
100
101	- name: Upload eval evidence
102	uses: actions/upload-artifact@v4
103	with:
104	name: eval-results-${{ inputs.model_version }}
105	path: results/
106
107	approve:
108	needs: evaluate
109	runs-on: ubuntu-latest
110	environment: ${{ inputs.target_env }}
111	steps:
112	- name: Record approval
113	run: \|
114	echo "Approved by: ${{ github.actor }}"
115	echo "Model: ${{ inputs.model_name }}:${{ inputs.model_version }}"
116	echo "Target: ${{ inputs.target_env }}"
117	echo "Time: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
118
119	deploy:
120	needs: approve
121	runs-on: ubuntu-latest
122	steps:
123	- uses: actions/checkout@v4
124
125	- name: Deploy canary
126	run: \|
127	kubectl set image deployment/${{ inputs.model_name }}-canary \
128	model=${{ inputs.model_name }}:${{ inputs.model_version }} \
129	-n ai-${{ inputs.target_env }}
130
131	- name: Wait for canary validation (15 min)
132	run: \|
133	python -m canary.validate \
134	--deployment ${{ inputs.model_name }}-canary \
135	--namespace ai-${{ inputs.target_env }} \
136	--duration 900 \
137	--quality-threshold 0.85 \
138	--error-rate-threshold 0.02
139
140	- name: Promote to full rollout
141	run: \|
142	kubectl set image deployment/${{ inputs.model_name }} \
143	model=${{ inputs.model_name }}:${{ inputs.model_version }} \

People who liked this also viewed

6 picks

Type

llmops-platform-engineering

42stars4forks

View on GitHub

How to install llmops-platform-engineering?

bagelhole/devops-security-agent-skills/llmops-platform-engineering

$npx -y skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering

Installs into the current project.

Summary

Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.

Files · 1

View on GitHub

SKILL.md

1	# LLMOps Platform Engineering
2
3	Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
4
5	## When to Use This Skill
6
7	- Building an internal platform for teams to deploy and manage LLM-powered features
8	- Designing CI/CD pipelines that include model evaluation gates
9	- Setting up A/B testing infrastructure for model versions
10	- Creating Kubernetes-based model serving infrastructure
11	- Establishing governance workflows for model promotion
12
13	## Prerequisites
14
15	- Kubernetes cluster with GPU node pools (or cloud inference API access)
16	- Container registry (Harbor, ECR, GCR, or ACR)
17	- CI/CD system (GitHub Actions, GitLab CI, or Argo Workflows)
18	- Observability stack (Prometheus + Grafana + OpenTelemetry)
19	- Model registry (MLflow or custom metadata store)
20
21	## Outcomes
22
23	- Standardized path from experiment to production
24	- Safe model rollout with quality and safety gates
25	- Repeatable infra modules for inference, vector DB, and observability
26	- Clear ownership model across platform, app, and security teams
27
28	## Reference Architecture
29
30	1. Control Plane: model registry, prompt/version catalog, policy checks, eval pipeline.
31	2. Data Plane: inference gateway, vector database, cache, feature store.
32	3. Ops Plane: telemetry, alerting, SLO dashboards, cost analytics.
33	4. Security Plane: IAM boundaries, secret rotation, content filters, audit logs.
34
35	## Model Promotion Pipeline
36
37	```yaml
38	# .github/workflows/model-promotion.yaml
39	name: Model Promotion Pipeline
40	on:
41	workflow_dispatch:
42	inputs:
43	model_name:
44	description: "Model identifier"
45	required: true
46	model_version:
47	description: "Model version to promote"
48	required: true
49	target_env:
50	description: "Target environment"
51	required: true
52	type: choice
53	options: [staging, production]
54
55	jobs:
56	evaluate:
57	runs-on: ubuntu-latest
58	steps:
59	- uses: actions/checkout@v4
60
61	- name: Run quality evaluation suite
62	run: \|
63	python -m evals.run \
64	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
65	--suite quality \
66	--output results/quality.json
67
68	- name: Run safety evaluation suite
69	run: \|
70	python -m evals.run \
71	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
72	--suite safety \
73	--output results/safety.json
74
75	- name: Run latency benchmark
76	run: \|
77	python -m evals.benchmark \
78	--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
79	--concurrent-users 50 \
80	--duration 300 \
81	--output results/latency.json
82
83	- name: Gate check - quality
84	run: \|
85	python -m evals.gate_check \
86	--results results/quality.json \
87	--threshold-file thresholds/quality.yaml
88
89	- name: Gate check - safety
90	run: \|
91	python -m evals.gate_check \
92	--results results/safety.json \
93	--threshold-file thresholds/safety.yaml
94
95	- name: Gate check - latency
96	run: \|
97	python -m evals.gate_check \
98	--results results/latency.json \
99	--threshold-file thresholds/latency.yaml
100
101	- name: Upload eval evidence
102	uses: actions/upload-artifact@v4
103	with:
104	name: eval-results-${{ inputs.model_version }}
105	path: results/
106
107	approve:
108	needs: evaluate
109	runs-on: ubuntu-latest
110	environment: ${{ inputs.target_env }}
111	steps:
112	- name: Record approval
113	run: \|
114	echo "Approved by: ${{ github.actor }}"
115	echo "Model: ${{ inputs.model_name }}:${{ inputs.model_version }}"
116	echo "Target: ${{ inputs.target_env }}"
117	echo "Time: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
118
119	deploy:
120	needs: approve
121	runs-on: ubuntu-latest
122	steps:
123	- uses: actions/checkout@v4
124
125	- name: Deploy canary
126	run: \|
127	kubectl set image deployment/${{ inputs.model_name }}-canary \
128	model=${{ inputs.model_name }}:${{ inputs.model_version }} \
129	-n ai-${{ inputs.target_env }}
130
131	- name: Wait for canary validation (15 min)
132	run: \|
133	python -m canary.validate \
134	--deployment ${{ inputs.model_name }}-canary \
135	--namespace ai-${{ inputs.target_env }} \
136	--duration 900 \
137	--quality-threshold 0.85 \
138	--error-rate-threshold 0.02
139
140	- name: Promote to full rollout
141	run: \|
142	kubectl set image deployment/${{ inputs.model_name }} \
143	model=${{ inputs.model_name }}:${{ inputs.model_version }} \

People who liked this also viewed

6 picks

Type