spring - Spring Batch 分区不起作用复合项目处理器
问题描述
我有一个 Spring Batch 分区作业。我正在使用CompositeProcessor
,从数据库中读取数据并将这些项目保存到CopyOnWriteArrayList
. 因为环境是并发的,但我CopyOnWriteArrayList
被用于其他线程和混合信息,我不知道为什么以及我做错了什么,以及将它们写入每个线程的文件的输出。
public class CustomerItemProcessor implements ItemProcessor<beangenerico,CopyOnWriteArrayList<beanCustomer>> {
private CustomerDAO customerDAO;
private CopyOnWriteArrayList<beanCustomer> listbean;
public CopyOnWriteArrayList<beanCustomer> process(beangenerico rangos) throws Exception {
listbean = customerDAO.getAccAgentes(rangos);
if(listbean != null) {
//customer.setId(currentCustomer.getId());
return listbean;
}else{
return null;
}
}
我的批处理 im XML 的配置:
<batch:job id="partitionJob" xmlns="http://www.springframework.org/schema/batch">
<batch:step id="masterStep">
<batch:partition step="slave" partitioner="rangePartitioner">
<batch:handler grid-size="10" task-executor="taskExecutor"/>
</batch:partition>
</batch:step>
</batch:job>
<!-- each thread will run this job, with different stepExecutionContext values. -->
<batch:step id="slave" xmlns="http://www.springframework.org/schema/batch">
<batch:tasklet task-executor="taskExecutor" throttle-limit="1">
<batch:chunk reader="beaniniendreader" writer="tempRecordsWriter" processor="completeItemProcessor" commit-interval="1" />
</batch:tasklet>
</batch:step>
<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
<bean id="rangePartitioner" class="my.package.springbatch.RangePartitioner" />
<bean id="beaniniendreader" class="my.package.springbatch.FormiikReader" scope="step"></bean>
<bean id="beanprocessor" class="my.package.springbatch.FormiikProcessor" scope="step">
<property name="accountExecutiveDao" ref="accountExecutiveDao"/>
</bean>
<bean id="beanprocessor2" class="my.package.springbatch.CustomerItemProcessor" scope="step">
<property name="customerDAO" ref="customerAccDao"/>
</bean>
<bean id="completeItemProcessor" class="org.springframework.batch.item.support.CompositeItemProcessor">
<property name="delegates">
<list>
<ref bean="beanprocessor2"/>
<ref bean="accItemprocessor"/>
<ref bean="beanaccDataItem"/>
</list>
</property>
</bean>
<bean id="tempRecordsWriter" class="my.package.springbatch.ListDelegateWriter" scope="step">
<property name="delegate" ref="flatFileItemWriterPartition"/>
</bean>
<!-- csv file writer -->
<bean id="flatFileItemWriterPartition" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step" >
<property name="resource"
value="file:csv/outputs/users.processed#{stepExecutionContext[fromId]}-#{stepExecutionContext[toId]}.csv" />
<property name="appendAllowed" value="false" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="cuenta, name, purchasedPackage" />
</bean>
</property>
</bean>
</property>
</bean>
解决方案
我回去讨论我的代码主题,建议我使用 Threadlocal 来存储线程特定的数据,这样它就可以工作了。在这里我再次输入我的代码。感谢您的回复。
public class CustomerItemProcessor implements ItemProcessor<beangenerico,ThreadLocal<CopyOnWriteArrayList<beanCustomer>>> {
private CustomerDAO customerDAO;
private ThreadLocal<CopyOnWriteArrayList<beanCustomer>> listbean = new ThreadLocal<CopyOnWriteArrayList<beanCustomer>>();
public ThreadLocal<CopyOnWriteArrayList<beanCustomer>> process(beangenerico rangos) throws Exception {
listbean.set(new CopyOnWriteArrayList<beanCustomer>());
listbean = customerDAO.getAccAgentes(rangos);
if(listbean != null) {
return listbean;
} else {
return null;
}
}
public void setCustomerDAO(CustomerDAO customerDAO) {
this.customerDAO = customerDAO;
}
}
推荐阅读
- visual-studio-code - 保存格式(更漂亮)不起作用VScode
- json - JSON Schema 的 Swagger UI 更改格式
- c# - 成员类未序列化定义的方式
- sql-server - Azure 数据工厂插入具有父标识的子代
- pine-script - 如何使用 pine-script 中的时间戳函数获取昨天的日期
- python - Windows IIS 服务器在 2 分钟后停止 Django 进程
- qt - 未显示 QChart 的刻度值
- html - div 两侧的径向渐变(但为负)
- kubernetes - Kubernetes - 限制对应用程序中私有 URI 的访问
- html - 停止键入出现在 Angular 中的非表单元素上的插入符号