spring-batch - 自定义 ItemReader 的 Spring Batch MultiResourceItemReader 配置问题
问题描述
只要在我的 publicFigureItemReader() 中我只是将 Resource 设置为我要读取的文件，我当前的下面的实现就可以正常工作。然而，现在我已经到了这一点，我正在尝试实现一个 MultiResourceItemReader，它将传递给 publicFigureItemReader() 一堆文件。因为我在一个文件夹中有大约 100 个文件。
我已经创建了 MultiResourceItemReader，现在如果我只使用经典的 ItemReader 我应该将其注册到我的步骤.reader(multiResourceItemReader())但这样做不起作用，因为我使用的是自定义 ItemReader 所以我不确定如何注册我的 MultiResourceItemReader。任何指针？
如果我尝试像这样注册，customItemWriter 将停止工作，我得到一个空指针异常。
@Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
.stream(multiResourceItemReader())
//.stream((ItemStream) publicFigureItemReader())
.build();
}
如果我尝试不注册 multiResourceItemReader() 异常是明显的java.lang.IllegalArgumentException: The Resource must not be null。
@Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
//.stream(multiResourceItemReader())
.stream((ItemStream) publicFigureItemReader())
.build();
}
这是我当前的作业配置类的复制粘贴。
@Value("classpath*:/data/mydata*.xml")
private Resource[] inputFiles;
@Bean
public MultiResourceItemReader<PublicFigure> multiResourceItemReader() {
MultiResourceItemReader<PublicFigure> reader = new MultiResourceItemReader<>();
reader.setDelegate((ResourceAwareItemReaderItemStream<? extends PublicFigure>) publicFigureItemReader());
reader.setResources(inputFiles);
return reader;
}
@Bean
ItemReader<PublicFigure> publicFigureItemReader() {
StaxEventItemReader<PublicFigure> xmlFileReader = new StaxEventItemReader<>();
//xmlFileReader.setResource(new FileSystemResource("C:\\Users\\da\\myfile1.xml"));
xmlFileReader.setFragmentRootElementName("PublicFigure");
Jaxb2Marshaller publicFigMarshaller = new Jaxb2Marshaller();
publicFigMarshaller.setClassesToBeBound(Associate.class, PublicFigure.class);
publicFigMarshaller.setUnmarshallerListener(new PublicFigureAssociateListener());
xmlFileReader.setUnmarshaller(publicFigMarshaller);
return xmlFileReader;
}
@Bean
ItemReader<Associate> customItemReader() {
return new AssociateItemReader(publicFigureItemReader());
}
@Bean
ItemWriter<Associate> associateItemWriter(DataSource dataSource, NamedParameterJdbcTemplate jdbcTemplate) {
JdbcBatchItemWriter<Associate> databaseItemWriter = new JdbcBatchItemWriter<>();
databaseItemWriter.setDataSource(dataSource);
databaseItemWriter.setJdbcTemplate(jdbcTemplate);
databaseItemWriter.setSql(QUERY_INSERT);
ItemPreparedStatementSetter<Associate> associateItemPreparedStatementSetter = new AssociatePreparedStatementSetter();
databaseItemWriter.setItemPreparedStatementSetter(associateItemPreparedStatementSetter);
return databaseItemWriter;
}
@Bean
Step associateXmlFileToDatabaseStep() {
return stepBuilderFactory.get("associateXmlFileToDatabaseStep")
.<Associate, Associate>chunk(1000)
.reader(customItemReader())
.writer(associateItemWriter(super.dataSource,super.namedParameterJdbcTemplate))
//.stream(multiResourceItemReader())
.stream((ItemStream) publicFigureItemReader())
.build();
}
上面的配置会抛出异常，即使修改customItemReader返回multiREsourceItemReader，见下图：
@Bean
ItemReader<Associate> customItemReader() {
//return new AssociateItemReader(publicFigureItemReader());
return new AssociateItemReader(multiResourceItemReader());
}
结果是堆栈跟踪
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:147) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
Caused by: java.lang.IllegalArgumentException: The Resource must not be null.
at org.springframework.util.Assert.notNull(Assert.java:134) ~[spring-core-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.item.xml.StaxEventItemReader.doOpen(StaxEventItemReader.java:189) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
... 12 common frames omitted
当我注册 multiReader 并注释 publicFigureItemReader()) 时，将出现以下堆栈跟踪，如下所示：
.stream(multiResourceItemReader())
//.stream((ItemStream) publicFigureItemReader())
- 这是堆栈跟踪：
org.springframework.oxm.UncategorizedMappingException: Unknown JAXB exception; nested exception is com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
org.springframework.core.io.Resource is an interface, and JAXB can't handle interfaces.
this problem is related to the following location:
at org.springframework.core.io.Resource
at private org.springframework.core.io.Resource io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure.resource
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.PublicFigure io.spring.batch.configuration.jobs.djsplit.djpersonsentities.Associate.publicFigure
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.Associate
at org.springframework.oxm.jaxb.Jaxb2Marshaller.convertJaxbException(Jaxb2Marshaller.java:915) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.getJaxbContext(Jaxb2Marshaller.java:483) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.createUnmarshaller(Jaxb2Marshaller.java:858) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.unmarshal(Jaxb2Marshaller.java:761) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.unmarshal(Jaxb2Marshaller.java:753) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.item.xml.StaxEventItemReader.doRead(StaxEventItemReader.java:240) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:88) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.readFromDelegate(MultiResourceItemReader.java:140) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.readNextItem(MultiResourceItemReader.java:119) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.item.file.MultiResourceItemReader.read(MultiResourceItemReader.java:108) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.AssociateItemReader.read(AssociateItemReader.java:23) ~[classes/:na]
at io.spring.batch.configuration.jobs.djsplit.djpersonsentities.AssociateItemReader.read(AssociateItemReader.java:11) ~[classes/:na]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:157) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) [spring-batch-core-3.0.9.RELEASE.jar:3.0.9.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
Caused by: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions
at com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:445) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:277) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:124) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1123) ~[na:1.8.0_144]
at com.sun.xml.internal.bind.v2.ContextFactory.createContext(ContextFactory.java:147) ~[na:1.8.0_144]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_144]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:247) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:234) ~[na:1.8.0_144]
at javax.xml.bind.ContextFinder.find(ContextFinder.java:462) ~[na:1.8.0_144]
at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:641) ~[na:1.8.0_144]
at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:584) ~[na:1.8.0_144]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.createJaxbContextFromClasses(Jaxb2Marshaller.java:523) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
at org.springframework.oxm.jaxb.Jaxb2Marshaller.getJaxbContext(Jaxb2Marshaller.java:476) ~[spring-oxm-4.3.17.RELEASE.jar:4.3.17.RELEASE]
... 37 common frames omitted
解决方案
推荐阅读
- python-3.x - Python Pillow：用数据加载图像
- flutter - 如何在 Flutter 中使用 OneSignal 通知按钮
- java - 如何使用自定义拆分器拆分由分隔符分隔的 csv 列？
- python - 如何根据单元格值对表格数据进行不同的样式设置？
- java - 为什么 Pattern.compile 在 android studio 和 Java SE 中的行为不同？
- html - 跨浏览器上的字体排版问题
- java - 如何根据以下字符/单词匹配 URI 字符串的特定部分
- ios - 我在 macbook 上为 ios 运行 react-native 时出错
- angular - 如何使用Angular 6中的选择显示和隐藏从数组动态创建的元素
- c# - 来自 PiCamera 的 AForge.NET 流 - AForge.NET 不接收任何字节