avro-protobuf-json-schema-registry

Name: avro-protobuf-json-schema-registry
Author: vaquarkhan

19stars4forks

How to install avro-protobuf-json-schema-registry?

vaquarkhan/data-engineering-agent-skills/avro-protobuf-json-schema-registry

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill avro-protobuf-json-schema-registry

Installs into the current project.

Summary

Guides agents through schema-registry-backed event contracts. Use when managing Avro, Protobuf, or JSON Schema for event streams, compatibility policies, producer and consumer evolution, or contract enforcement in messaging systems.

Files · 1

View on GitHub

SKILL.md

1	# Avro Protobuf JSON Schema Registry
2
3	## Overview
4
5	Use this skill when schema management for events must be explicit and enforceable. It helps agents coordinate producer changes, consumer compatibility, registry policy, and versioned contracts across teams sharing event streams.
6
7	## When to Use
8
9	- adopting a schema registry for event-driven systems
10	- defining or modifying Avro, Protobuf, or JSON Schema definitions
11	- setting compatibility policies for topics or subjects
12	- coordinating producer schema changes across multiple consumers
13	- enforcing contract governance for multi-team event ecosystems
14
15	Do not use this when schemas are managed purely through application code with no shared registry or when the system has a single producer-consumer pair with no evolution concerns.
16
17	## Workflow
18
19	1. Define schema ownership and subject naming.
20	Include:
21	- who owns each schema subject (team, service, domain)
22	- subject naming strategy: `TopicNameStrategy`, `RecordNameStrategy`, or `TopicRecordNameStrategy`
23	- where schemas are stored: Confluent Schema Registry, AWS Glue Schema Registry, Apicurio, or equivalent
24	- schema source of truth: registry-first or code-first with CI sync
25
26	2. Choose the schema format intentionally.
27	- `Avro`: strong ecosystem support, compact binary, requires registry for deserialization
28	- `Protobuf`: excellent for multi-language teams, supports nested messages and services
29	- `JSON Schema`: human-readable, looser typing, easier adoption but weaker guarantees
30	- consider: serialization size, language support, tooling maturity, and team familiarity
31	- do not mix formats within a single domain unless isolation is absolute
32
33	3. Define and enforce compatibility policies.
34	- `BACKWARD`: new schema can read old data (safe for consumer-first evolution)
35	- `FORWARD`: old schema can read new data (safe for producer-first evolution)
36	- `FULL`: both backward and forward compatible (safest, most restrictive)
37	- `NONE`: no compatibility checking (use only for development or isolated topics)
38	- set policy per subject, not globally — different topics have different evolution needs
39	- test compatibility in CI before publishing schema changes
40
41	4. Plan producer and consumer evolution paths.
42	- additive changes (new optional fields): safe under backward compatibility
43	- field removal or rename: breaking under most policies — requires migration
44	- type changes: generally breaking — prefer new fields over type modifications
45	- document the upgrade order: which side deploys first (producer or consumer)?
46	- use default values explicitly to support backward reading
47
48	5. Integrate schema validation into the release workflow.
49	- validate schema compatibility in CI using registry API or CLI tools
50	- block releases that break compatibility policy
51	- version schemas alongside service code — not independently
52	- produce evidence of compatibility checks for release gates
53	- define rollback plan: can a previous schema version be re-registered safely?
54
55	6. Handle multi-team governance and schema discovery.
56	- publish schema documentation alongside event catalog or data catalog
57	- make consumer dependency visible — who reads which schema versions?
58	- define deprecation process for old schema versions
59	- plan for schema registry availability as a critical path dependency
60
61	## Common Rationalizations
62
63	\| Rationalization \| Reality \|
64	\| --- \| --- \|
65	\| "We can evolve schemas freely since it's just adding fields." \| Even additive changes can break consumers if default values are missing or field semantics change without notice. \|
66	\| "The registry is just for serialization, not governance." \| The registry is the enforcement point for cross-team contracts. Treating it as a utility ignores its governance value. \|
67	\| "JSON Schema is easier so we'll use it everywhere." \| JSON Schema provides weaker guarantees and no binary efficiency. Format choice should match the system's durability, performance, and typing needs. \|
68	\| "Compatibility can be set to NONE during development and tightened later." \| Teams rarely tighten compatibility retroactively. Setting policies early prevents breaking changes from accumulating. \|
69
70	## Red Flags
71
72	- no compatibility policy defined on production subjects
73	- schemas are modified directly in the registry without CI validation
74	- producer and consumer deploy simultaneously with no coordination order
75	- no documentation of which consumers depend on which schema versions
76	- schema registry is a single point of failure with no availability plan
77	- field semantics change without version bumps or consumer notification
78	- deprecated schema versions are never cleaned up or documented

People who liked this also viewed

6 picks

Type

avro-protobuf-json-schema-registry

19stars4forks

View on GitHub

How to install avro-protobuf-json-schema-registry?

vaquarkhan/data-engineering-agent-skills/avro-protobuf-json-schema-registry

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill avro-protobuf-json-schema-registry

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Avro Protobuf JSON Schema Registry
2
3	## Overview
4
5	Use this skill when schema management for events must be explicit and enforceable. It helps agents coordinate producer changes, consumer compatibility, registry policy, and versioned contracts across teams sharing event streams.
6
7	## When to Use
8
9	- adopting a schema registry for event-driven systems
10	- defining or modifying Avro, Protobuf, or JSON Schema definitions
11	- setting compatibility policies for topics or subjects
12	- coordinating producer schema changes across multiple consumers
13	- enforcing contract governance for multi-team event ecosystems
14
15	Do not use this when schemas are managed purely through application code with no shared registry or when the system has a single producer-consumer pair with no evolution concerns.
16
17	## Workflow
18
19	1. Define schema ownership and subject naming.
20	Include:
21	- who owns each schema subject (team, service, domain)
22	- subject naming strategy: `TopicNameStrategy`, `RecordNameStrategy`, or `TopicRecordNameStrategy`
23	- where schemas are stored: Confluent Schema Registry, AWS Glue Schema Registry, Apicurio, or equivalent
24	- schema source of truth: registry-first or code-first with CI sync
25
26	2. Choose the schema format intentionally.
27	- `Avro`: strong ecosystem support, compact binary, requires registry for deserialization
28	- `Protobuf`: excellent for multi-language teams, supports nested messages and services
29	- `JSON Schema`: human-readable, looser typing, easier adoption but weaker guarantees
30	- consider: serialization size, language support, tooling maturity, and team familiarity
31	- do not mix formats within a single domain unless isolation is absolute
32
33	3. Define and enforce compatibility policies.
34	- `BACKWARD`: new schema can read old data (safe for consumer-first evolution)
35	- `FORWARD`: old schema can read new data (safe for producer-first evolution)
36	- `FULL`: both backward and forward compatible (safest, most restrictive)
37	- `NONE`: no compatibility checking (use only for development or isolated topics)
38	- set policy per subject, not globally — different topics have different evolution needs
39	- test compatibility in CI before publishing schema changes
40
41	4. Plan producer and consumer evolution paths.
42	- additive changes (new optional fields): safe under backward compatibility
43	- field removal or rename: breaking under most policies — requires migration
44	- type changes: generally breaking — prefer new fields over type modifications
45	- document the upgrade order: which side deploys first (producer or consumer)?
46	- use default values explicitly to support backward reading
47
48	5. Integrate schema validation into the release workflow.
49	- validate schema compatibility in CI using registry API or CLI tools
50	- block releases that break compatibility policy
51	- version schemas alongside service code — not independently
52	- produce evidence of compatibility checks for release gates
53	- define rollback plan: can a previous schema version be re-registered safely?
54
55	6. Handle multi-team governance and schema discovery.
56	- publish schema documentation alongside event catalog or data catalog
57	- make consumer dependency visible — who reads which schema versions?
58	- define deprecation process for old schema versions
59	- plan for schema registry availability as a critical path dependency
60
61	## Common Rationalizations
62
63	\| Rationalization \| Reality \|
64	\| --- \| --- \|
65	\| "We can evolve schemas freely since it's just adding fields." \| Even additive changes can break consumers if default values are missing or field semantics change without notice. \|
66	\| "The registry is just for serialization, not governance." \| The registry is the enforcement point for cross-team contracts. Treating it as a utility ignores its governance value. \|
67	\| "JSON Schema is easier so we'll use it everywhere." \| JSON Schema provides weaker guarantees and no binary efficiency. Format choice should match the system's durability, performance, and typing needs. \|
68	\| "Compatibility can be set to NONE during development and tightened later." \| Teams rarely tighten compatibility retroactively. Setting policies early prevents breaking changes from accumulating. \|
69
70	## Red Flags
71
72	- no compatibility policy defined on production subjects
73	- schemas are modified directly in the registry without CI validation
74	- producer and consumer deploy simultaneously with no coordination order
75	- no documentation of which consumers depend on which schema versions
76	- schema registry is a single point of failure with no availability plan
77	- field semantics change without version bumps or consumer notification
78	- deprecated schema versions are never cleaned up or documented

People who liked this also viewed

6 picks

Type