首页 > 解决方案 > Spring Batch PathMatchingResourcePatternResolver.getResources() 不适用于 https url

问题描述

我正在尝试读取“https://raw.githubusercontent.com/Shrutika09/SpringBatchTemplateUploaderPOC/main/order-data-*.csv”上存在的多个 csv 文件,并使用 Spring Batch 将其并行插入数据库。

当我将 URL 用于单个 csv 文件(https://raw.githubusercontent.com/Shrutika09/SpringBatchTemplateUploaderPOC/main/order-data-1.csv)时,所有记录都被读取并插入到数据库中。但是,当我尝试读取具有特定命名模式(https://raw.githubusercontent.com/Shrutika09/SpringBatchTemplateUploaderPOC/main/order-data-*.csv)的所有文件时,它无法识别该文件,因此不会t 按预期工作。

有什么方法可以让我们从 github 位置读取与特定命名模式匹配的所有文件。

我正在使用 Spring Batch Partitioner

Partitioner:
@Bean
    public Partitioner partitioner() throws Exception {
        System.out.println("In Partitioner");
        MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
        PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
        partitioner.setResources(resolver.getResources("https://raw.githubusercontent.com/Shrutika09/SpringBatchTemplateUploaderPOC/main/order-data-*.csv"));
        partitioner.partition(5);
        return partitioner;
    }

Reader: 
@Bean
    @StepScope
    public FlatFileItemReader<Orders> reader(@Value("#{stepExecutionContext['fileName']}") String path)
            throws MalformedURLException {
        System.out.println("In Reader");
        System.out.println("In Reader" +path);
        FlatFileItemReader<Orders> reader = new FlatFileItemReader<Orders>();

        reader.setResource(new UrlResource(path));
        reader.setLineMapper(new DefaultLineMapper<Orders>() {
            {
                setLineTokenizer(new DelimitedLineTokenizer() {
                    {
                        setNames(new String[] { "id", "firstName", "lastName" });
                    }
                });
                setFieldSetMapper(new BeanWrapperFieldSetMapper<Orders>() {
                    {
                        setTargetType(Orders.class);
                    }
                });
            }
        });
        return reader;
    }

标签: javaspringparallel-processingspring-batchpartitioning

解决方案


推荐阅读