spring-batch

Name: spring-batch
Author: rrezartprebreza

174stars33forks

How to install spring-batch?

rrezartprebreza/spring-boot-skills/spring-batch

$npx -y skills add rrezartprebreza/spring-boot-skills --skill spring-batch

Installs into the current project.

Summary

Use when building batch jobs, ETL pipelines, scheduled imports/exports, or any chunk-oriented bulk processing with Spring Batch. Covers the Spring Batch 5 / Boot 3 builder API, restartability and idempotent job parameters, reader/writer thread-safety, fault tolerance, and chunk t

Files · 1

View on GitHub

SKILL.md

1	# Spring Batch
2
3	Spring Boot 3.x ships Spring Batch 5. The API changed significantly from 4.x — most online
4	examples are wrong. The two rules that break the most agent-generated code:
5
6	1. Do NOT add `@EnableBatchProcessing`. Boot auto-configures the `JobRepository`,
7	`JobLauncher`, and transaction manager. Adding `@EnableBatchProcessing` disables that
8	auto-configuration and you lose all the wired beans.
9	2. `JobBuilderFactory` and `StepBuilderFactory` are gone. Build jobs and steps with
10	`new JobBuilder(name, jobRepository)` / `new StepBuilder(name, jobRepository)`, injecting the
11	`JobRepository` and `PlatformTransactionManager` Boot already created.
12
13	## Job & Step (Spring Batch 5 API)
14
15	```java
16	@Configuration
17	@RequiredArgsConstructor
18	public class OrderExportJobConfig {
19
20	@Bean
21	public Job orderExportJob(JobRepository jobRepository, Step exportStep) {
22	return new JobBuilder("orderExportJob", jobRepository)
23	.incrementer(new RunIdIncrementer()) // lets the same job be re-run; see "Idempotency"
24	.start(exportStep)
25	.build();
26	}
27
28	@Bean
29	public Step exportStep(JobRepository jobRepository,
30	PlatformTransactionManager txManager, // Boot's, injected — do NOT new one up
31	ItemReader<Order> reader,
32	ItemProcessor<Order, OrderRow> processor,
33	ItemWriter<OrderRow> writer) {
34	return new StepBuilder("exportStep", jobRepository)
35	.<Order, OrderRow>chunk(500, txManager) // chunk size is the commit interval — and a TX boundary
36	.reader(reader)
37	.processor(processor)
38	.writer(writer)
39	.faultTolerant()
40	.skip(FlatFileParseException.class)
41	.skipLimit(50)
42	.build();
43	}
44	}
45	```
46
47	`chunk(500, txManager)` means: read 500 items, process each, hand the list of 500 to the writer,
48	commit one transaction, repeat. The chunk is the unit of restart and the unit of rollback.
49
50	## Idempotency & Restartability — the #1 operational gotcha
51
52	A `JobInstance` is identified by its identifying `JobParameters`. Launch the same job with the
53	same identifying parameters twice and you get:
54
55	```
56	JobInstanceAlreadyCompleteException: A job instance already exists and is complete
57	```
58
59	This is by design — Batch refuses to re-run completed work. Two ways to handle it:
60
61	```java
62	// Option A — RunIdIncrementer on the job (above) + JobLauncherApplicationRunner bumps run.id each launch.
63	// Option B — add a unique identifying parameter yourself when launching:
64	JobParameters params = new JobParametersBuilder()
65	.addString("status", "COMPLETED") // identifying — part of the instance key
66	.addLong("run.id", System.currentTimeMillis()) // identifying & unique — makes each run a new instance
67	.toJobParameters();
68	```
69
70	Mark a parameter non-identifying with the `false` flag when it's metadata that shouldn't change
71	the instance identity (e.g. a request id you log but don't key on):
72
73	```java
74	.addString("requestId", requestId, false) // non-identifying — excluded from the instance key
75	```
76
77	A failed job, by contrast, is resumed when relaunched with the same parameters — it skips
78	completed steps and restarts the failed step from the last committed chunk. That is the point of the
79	metadata tables. Don't defeat it by always passing a unique parameter if you want resume-on-failure.
80
81	## ItemReader — sort key and thread-safety
82
83	```java
84	@Bean
85	@StepScope // required: late-binds jobParameters at step execution, not context startup
86	public JpaPagingItemReader<Order> orderReader(
87	EntityManagerFactory emf,
88	@Value("#{jobParameters['status']}") String status) {
89	return new JpaPagingItemReaderBuilder<Order>()
90	.name("orderReader")
91	.entityManagerFactory(emf)
92	.queryString("SELECT o FROM Order o WHERE o.status = :status ORDER BY o.id") // ORDER BY is MANDATORY
93	.parameterValues(Map.of("status", OrderStatus.valueOf(status)))
94	.pageSize(500) // keep pageSize == chunk size
95	.build();
96	}
97	```
98
99	- Paging readers require a deterministic `ORDER BY` on a unique column. Without it the DB returns
100	rows in arbitrary order across pages → rows get skipped or processed twice. This is silent
101	data corruption, not an error.
102	- `JdbcCursorItemReader` is NOT thread-safe. `JdbcPagingItemReader` / `JpaPagingItemReader` are
103	safe for multi-threaded steps. For a non-thread-safe reader in a multi-threaded step, wrap it in
104	`SynchronizedItemStreamReader`.
105	- **Don't mutate the column y

People who liked this also viewed

6 picks

Type

spring-batch

174stars33forks

View on GitHub

How to install spring-batch?

rrezartprebreza/spring-boot-skills/spring-batch

$npx -y skills add rrezartprebreza/spring-boot-skills --skill spring-batch

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Spring Batch
2
3	Spring Boot 3.x ships Spring Batch 5. The API changed significantly from 4.x — most online
4	examples are wrong. The two rules that break the most agent-generated code:
5
6	1. Do NOT add `@EnableBatchProcessing`. Boot auto-configures the `JobRepository`,
7	`JobLauncher`, and transaction manager. Adding `@EnableBatchProcessing` disables that
8	auto-configuration and you lose all the wired beans.
9	2. `JobBuilderFactory` and `StepBuilderFactory` are gone. Build jobs and steps with
10	`new JobBuilder(name, jobRepository)` / `new StepBuilder(name, jobRepository)`, injecting the
11	`JobRepository` and `PlatformTransactionManager` Boot already created.
12
13	## Job & Step (Spring Batch 5 API)
14
15	```java
16	@Configuration
17	@RequiredArgsConstructor
18	public class OrderExportJobConfig {
19
20	@Bean
21	public Job orderExportJob(JobRepository jobRepository, Step exportStep) {
22	return new JobBuilder("orderExportJob", jobRepository)
23	.incrementer(new RunIdIncrementer()) // lets the same job be re-run; see "Idempotency"
24	.start(exportStep)
25	.build();
26	}
27
28	@Bean
29	public Step exportStep(JobRepository jobRepository,
30	PlatformTransactionManager txManager, // Boot's, injected — do NOT new one up
31	ItemReader<Order> reader,
32	ItemProcessor<Order, OrderRow> processor,
33	ItemWriter<OrderRow> writer) {
34	return new StepBuilder("exportStep", jobRepository)
35	.<Order, OrderRow>chunk(500, txManager) // chunk size is the commit interval — and a TX boundary
36	.reader(reader)
37	.processor(processor)
38	.writer(writer)
39	.faultTolerant()
40	.skip(FlatFileParseException.class)
41	.skipLimit(50)
42	.build();
43	}
44	}
45	```
46
47	`chunk(500, txManager)` means: read 500 items, process each, hand the list of 500 to the writer,
48	commit one transaction, repeat. The chunk is the unit of restart and the unit of rollback.
49
50	## Idempotency & Restartability — the #1 operational gotcha
51
52	A `JobInstance` is identified by its identifying `JobParameters`. Launch the same job with the
53	same identifying parameters twice and you get:
54
55	```
56	JobInstanceAlreadyCompleteException: A job instance already exists and is complete
57	```
58
59	This is by design — Batch refuses to re-run completed work. Two ways to handle it:
60
61	```java
62	// Option A — RunIdIncrementer on the job (above) + JobLauncherApplicationRunner bumps run.id each launch.
63	// Option B — add a unique identifying parameter yourself when launching:
64	JobParameters params = new JobParametersBuilder()
65	.addString("status", "COMPLETED") // identifying — part of the instance key
66	.addLong("run.id", System.currentTimeMillis()) // identifying & unique — makes each run a new instance
67	.toJobParameters();
68	```
69
70	Mark a parameter non-identifying with the `false` flag when it's metadata that shouldn't change
71	the instance identity (e.g. a request id you log but don't key on):
72
73	```java
74	.addString("requestId", requestId, false) // non-identifying — excluded from the instance key
75	```
76
77	A failed job, by contrast, is resumed when relaunched with the same parameters — it skips
78	completed steps and restarts the failed step from the last committed chunk. That is the point of the
79	metadata tables. Don't defeat it by always passing a unique parameter if you want resume-on-failure.
80
81	## ItemReader — sort key and thread-safety
82
83	```java
84	@Bean
85	@StepScope // required: late-binds jobParameters at step execution, not context startup
86	public JpaPagingItemReader<Order> orderReader(
87	EntityManagerFactory emf,
88	@Value("#{jobParameters['status']}") String status) {
89	return new JpaPagingItemReaderBuilder<Order>()
90	.name("orderReader")
91	.entityManagerFactory(emf)
92	.queryString("SELECT o FROM Order o WHERE o.status = :status ORDER BY o.id") // ORDER BY is MANDATORY
93	.parameterValues(Map.of("status", OrderStatus.valueOf(status)))
94	.pageSize(500) // keep pageSize == chunk size
95	.build();
96	}
97	```
98
99	- Paging readers require a deterministic `ORDER BY` on a unique column. Without it the DB returns
100	rows in arbitrary order across pages → rows get skipped or processed twice. This is silent
101	data corruption, not an error.
102	- `JdbcCursorItemReader` is NOT thread-safe. `JdbcPagingItemReader` / `JpaPagingItemReader` are
103	safe for multi-threaded steps. For a non-thread-safe reader in a multi-threaded step, wrap it in
104	`SynchronizedItemStreamReader`.
105	- **Don't mutate the column y

People who liked this also viewed

6 picks

Type