java - 在一个事务中多次使用刷新时，休眠过度计数刷新的实体

Question

我正在使用 jpa 和 hibernate 准备 spring 应用程序，以在 db 中保存数亿行。

我正在对具有 500 万行的较小数据源进行一些测试。为了不将所有数据保留在内存中，直到处理结束，我每隔 n 行刷新一次数据。代码如下所示：

    AtomicLong counter = new AtomicLong();

    destinationService.doInTransaction(() -> {

        IntStream.range(0, (int) numberOfPages)
                .boxed()
                .peek(integer -> log.debug("Getting page: {}", integer + 1))
                .flatMap(i -> sourceRepository.findAll(PageRequest.of(i, pageSize)).get())
                .flatMap(ETLService::map)
                .peek(destinationService::save)
                .map(measurement -> counter.getAndIncrement())
                .peek(i -> {
                    if (i % jdbcBatchSize == 0) {
                        log.debug("Flushing. Objects counted: {}", i);
                        destinationService.flush();
                    }
                })
                .count();
        return null;
    });

我启用了会话指标，它显示了更多的实体被刷新：

2021-09-18 23:58:10.807  INFO 4680 --- [           main] i.StatisticalLoggingSessionEventListener : Session Metrics {
    3140100 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    21488800 nanoseconds spent preparing 584 JDBC statements;
    0 nanoseconds spent executing 0 JDBC statements;
    251780906500 nanoseconds spent executing 5828 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    2494336312500 nanoseconds spent executing 584 flushes (flushing a total of 1702357144 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}

但是，如果我要从流中删除冲洗部分，它会显示一个具有正确行数的冲洗。

 2021-09-18 23:04:55.562  INFO 11476 --- [           main] i.StatisticalLoggingSessionEventListener : Session Metrics {
    4869500 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    2912100 nanoseconds spent preparing 1 JDBC statements;
    0 nanoseconds spent executing 0 JDBC statements;
    250479981900 nanoseconds spent executing 5827 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    302878192900 nanoseconds spent executing 1 flushes (flushing a total of 5826561 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}

为什么有更多的刷新休眠过多的实体？

标签： javahibernatespring-data-jpa