ai-observability

Name: ai-observability
Author: rrezartprebreza

174stars33forks

How to install ai-observability?

rrezartprebreza/spring-boot-skills/ai-observability

$npx -y skills add rrezartprebreza/spring-boot-skills --skill ai-observability

Installs into the current project.

Summary

Use when adding monitoring, metrics, logging, or tracing to Spring AI or LLM integration code. Covers token tracking, latency measurement, cost estimation, and prompt/response logging. Use when user mentions AI monitoring, token costs, or LLM observability.

Files · 1

View on GitHub

SKILL.md

1	# AI Observability
2
3	## Dependencies
4
5	```xml
6	<dependency>
7	<groupId>org.springframework.boot</groupId>
8	<artifactId>spring-boot-starter-actuator</artifactId>
9	</dependency>
10	<dependency>
11	<groupId>io.micrometer</groupId>
12	<artifactId>micrometer-registry-prometheus</artifactId>
13	</dependency>
14	```
15
16	## Spring AI Built-in Observability
17
18	Spring AI 1.0+ includes built-in Micrometer instrumentation:
19
20	```yaml
21	spring:
22	ai:
23	chat:
24	observations:
25	log-prompt: true # GA renamed include-prompt → log-prompt. OFF in prod (PII).
26	log-completion: true # GA renamed include-completion → log-completion
27	management:
28	metrics:
29	tags:
30	application: order-service
31	endpoints:
32	web:
33	exposure:
34	include: health,prometheus,metrics
35	```
36
37	Auto-generated metrics (OpenTelemetry GenAI semantic conventions):
38	- `gen_ai.client.operation` — model call latency, tagged with provider and model
39	- `gen_ai.client.token.usage` — token counts (input/output/total)
40	- `spring.ai.chat.client` — ChatClient-level operation timer/span
41
42	## Custom AI Metrics
43
44	```java
45	@Component
46	@RequiredArgsConstructor
47	public class AiMetrics {
48
49	private final MeterRegistry meterRegistry;
50
51	private final Timer.Builder promptTimer = Timer.builder("ai.prompt.latency")
52	.description("LLM prompt latency");
53
54	private final Counter.Builder tokenCounter = Counter.builder("ai.tokens.used")
55	.description("Total tokens consumed");
56
57	public <T> T track(String operation, String model, Supplier<T> call) {
58	return Timer.builder("ai.prompt.latency")
59	.tag("operation", operation)
60	.tag("model", model)
61	.register(meterRegistry)
62	.recordCallable(() -> call.get());
63	}
64
65	public void recordTokens(String operation, String model, int inputTokens, int outputTokens) {
66	Counter.builder("ai.tokens.used")
67	.tag("operation", operation)
68	.tag("model", model)
69	.tag("type", "input")
70	.register(meterRegistry)
71	.increment(inputTokens);
72
73	Counter.builder("ai.tokens.used")
74	.tag("operation", operation)
75	.tag("model", model)
76	.tag("type", "output")
77	.register(meterRegistry)
78	.increment(outputTokens);
79	}
80	}
81	```
82
83	## Prompt/Response Logging Advisor
84
85	GA replaced the whole advisor API: `CallAroundAdvisor` → `CallAdvisor`, `AdvisedRequest` →
86	`ChatClientRequest`, `AdvisedResponse` → `ChatClientResponse`, and `Usage.getGenerationTokens()` →
87	`getCompletionTokens()`. Agents reliably generate the old one — it does not compile on 1.0.
88
89	```java
90	@Component
91	public class AiAuditAdvisor implements CallAdvisor {
92
93	private static final Logger log = LoggerFactory.getLogger(AiAuditAdvisor.class);
94
95	@Override
96	public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
97	String requestId = UUID.randomUUID().toString();
98	long start = System.currentTimeMillis();
99
100	log.info("[AI-AUDIT] requestId={} promptLength={}",
101	requestId, request.prompt().getUserMessage().getText().length());
102
103	try {
104	ChatClientResponse response = chain.nextCall(request);
105	long latency = System.currentTimeMillis() - start;
106
107	ChatResponse chatResponse = response.chatResponse();
108	if (chatResponse != null && chatResponse.getMetadata() != null) {
109	Usage usage = chatResponse.getMetadata().getUsage();
110	log.info("[AI-AUDIT] requestId={} latencyMs={} inputTokens={} outputTokens={}",
111	requestId, latency,
112	usage.getPromptTokens(), usage.getCompletionTokens()); // GA: not getGenerationTokens()
113	}
114	return response;
115	} catch (Exception e) {
116	log.error("[AI-AUDIT] requestId={} FAILED after {}ms", requestId,
117	System.currentTimeMillis() - start, e);
118	throw e;
119	}
120	}
121
122	@Override
123	public String getName() { return "AiAuditAdvisor"; }
124
125	@Override
126	public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
127	}
128	```
129
130	## Cost Estimation
131
132	```java
133	@Service
134	public class AiCostEstimator {
135
136	// Prices per million tokens — update when pricing changes
137	private static final Map<String, double[]> PRICING = Map.of(
138	"claude-sonnet-4-20250514", new double[]{3.0, 15.0}, // [input, output] per 1M tokens
139	"claude-haiku-4-5-20251001", new double[]{0.8, 4.0},
140	"gpt-4o", new double[]{5.0, 15.0},
141	"gpt-4o-mini", new double[]{0.15, 0.6}
142	);
143
144	public double estimateCost(String model, int inputTokens, int outputTokens) {

People who liked this also viewed

6 picks

Type

ai-observability

174stars33forks

View on GitHub

How to install ai-observability?

rrezartprebreza/spring-boot-skills/ai-observability

$npx -y skills add rrezartprebreza/spring-boot-skills --skill ai-observability

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# AI Observability
2
3	## Dependencies
4
5	```xml
6	<dependency>
7	<groupId>org.springframework.boot</groupId>
8	<artifactId>spring-boot-starter-actuator</artifactId>
9	</dependency>
10	<dependency>
11	<groupId>io.micrometer</groupId>
12	<artifactId>micrometer-registry-prometheus</artifactId>
13	</dependency>
14	```
15
16	## Spring AI Built-in Observability
17
18	Spring AI 1.0+ includes built-in Micrometer instrumentation:
19
20	```yaml
21	spring:
22	ai:
23	chat:
24	observations:
25	log-prompt: true # GA renamed include-prompt → log-prompt. OFF in prod (PII).
26	log-completion: true # GA renamed include-completion → log-completion
27	management:
28	metrics:
29	tags:
30	application: order-service
31	endpoints:
32	web:
33	exposure:
34	include: health,prometheus,metrics
35	```
36
37	Auto-generated metrics (OpenTelemetry GenAI semantic conventions):
38	- `gen_ai.client.operation` — model call latency, tagged with provider and model
39	- `gen_ai.client.token.usage` — token counts (input/output/total)
40	- `spring.ai.chat.client` — ChatClient-level operation timer/span
41
42	## Custom AI Metrics
43
44	```java
45	@Component
46	@RequiredArgsConstructor
47	public class AiMetrics {
48
49	private final MeterRegistry meterRegistry;
50
51	private final Timer.Builder promptTimer = Timer.builder("ai.prompt.latency")
52	.description("LLM prompt latency");
53
54	private final Counter.Builder tokenCounter = Counter.builder("ai.tokens.used")
55	.description("Total tokens consumed");
56
57	public <T> T track(String operation, String model, Supplier<T> call) {
58	return Timer.builder("ai.prompt.latency")
59	.tag("operation", operation)
60	.tag("model", model)
61	.register(meterRegistry)
62	.recordCallable(() -> call.get());
63	}
64
65	public void recordTokens(String operation, String model, int inputTokens, int outputTokens) {
66	Counter.builder("ai.tokens.used")
67	.tag("operation", operation)
68	.tag("model", model)
69	.tag("type", "input")
70	.register(meterRegistry)
71	.increment(inputTokens);
72
73	Counter.builder("ai.tokens.used")
74	.tag("operation", operation)
75	.tag("model", model)
76	.tag("type", "output")
77	.register(meterRegistry)
78	.increment(outputTokens);
79	}
80	}
81	```
82
83	## Prompt/Response Logging Advisor
84
85	GA replaced the whole advisor API: `CallAroundAdvisor` → `CallAdvisor`, `AdvisedRequest` →
86	`ChatClientRequest`, `AdvisedResponse` → `ChatClientResponse`, and `Usage.getGenerationTokens()` →
87	`getCompletionTokens()`. Agents reliably generate the old one — it does not compile on 1.0.
88
89	```java
90	@Component
91	public class AiAuditAdvisor implements CallAdvisor {
92
93	private static final Logger log = LoggerFactory.getLogger(AiAuditAdvisor.class);
94
95	@Override
96	public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
97	String requestId = UUID.randomUUID().toString();
98	long start = System.currentTimeMillis();
99
100	log.info("[AI-AUDIT] requestId={} promptLength={}",
101	requestId, request.prompt().getUserMessage().getText().length());
102
103	try {
104	ChatClientResponse response = chain.nextCall(request);
105	long latency = System.currentTimeMillis() - start;
106
107	ChatResponse chatResponse = response.chatResponse();
108	if (chatResponse != null && chatResponse.getMetadata() != null) {
109	Usage usage = chatResponse.getMetadata().getUsage();
110	log.info("[AI-AUDIT] requestId={} latencyMs={} inputTokens={} outputTokens={}",
111	requestId, latency,
112	usage.getPromptTokens(), usage.getCompletionTokens()); // GA: not getGenerationTokens()
113	}
114	return response;
115	} catch (Exception e) {
116	log.error("[AI-AUDIT] requestId={} FAILED after {}ms", requestId,
117	System.currentTimeMillis() - start, e);
118	throw e;
119	}
120	}
121
122	@Override
123	public String getName() { return "AiAuditAdvisor"; }
124
125	@Override
126	public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
127	}
128	```
129
130	## Cost Estimation
131
132	```java
133	@Service
134	public class AiCostEstimator {
135
136	// Prices per million tokens — update when pricing changes
137	private static final Map<String, double[]> PRICING = Map.of(
138	"claude-sonnet-4-20250514", new double[]{3.0, 15.0}, // [input, output] per 1M tokens
139	"claude-haiku-4-5-20251001", new double[]{0.8, 4.0},
140	"gpt-4o", new double[]{5.0, 15.0},
141	"gpt-4o-mini", new double[]{0.15, 0.6}
142	);
143
144	public double estimateCost(String model, int inputTokens, int outputTokens) {

People who liked this also viewed

6 picks

Type