时间戳

首页 > 解决方案 > 写入大约 2 GB Vertx 的文件时出现内存不足错误

file - 写入大约 2 GB Vertx 的文件时出现内存不足错误

问题描述

我正在尝试将文件从一个位置上传到另一个位置，文件大小可以达到 5GB，首先我尝试在本地使用 vertx 从一个位置写入另一个位置。

我正在读取文件并将其写入另一个位置，我添加了代码片段，但出现内存不足错误。任何人都可以提出更好的解决方案。

这是代码片段：-

vertx.fileSystem().readFile(fileToPut,readHandler -> {
                       if (readHandler.succeeded()){
                           vertx.fileSystem().writeFile("C:\\Users\\xyz\\Desktop\\abc\\files",readHandler.result(),writeHandler -> {
                              if (writeHandler.succeeded()){
                                  log.info("file is successfully written");
                                  blockingHandler.complete();
                              } else {
                                  blockingHandler.fail(writeHandler.cause());
                              }
                           });

                       } else {
                           log.error("");
                           blockingHandler.fail(readHandler.cause());
                       }
                    });

                },resultHandler -> {
                    if (resultHandler.succeeded()){
                        log.debug("Blocking call succeed");
                    } else {
                        log.error("Error in blocking call : ",resultHandler.cause());
                    }
                });

错误日志：-

java.lang.OutOfMemoryError: Java heap space
    at io.netty.util.internal.PlatformDependent.allocateUninitializedArray(PlatformDependent.java:281) ~[netty-common-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledUnsafeHeapByteBuf.allocateArray(UnpooledUnsafeHeapByteBuf.java:39) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.allocateArray(UnpooledByteBufAllocator.java:144) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledHeapByteBuf.<init>(UnpooledHeapByteBuf.java:59) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledUnsafeHeapByteBuf.<init>(UnpooledUnsafeHeapByteBuf.java:34) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.<init>(UnpooledByteBufAllocator.java:139) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:168) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.netty.buffer.Unpooled.buffer(Unpooled.java:136) ~[netty-buffer-4.1.49.Final.jar:4.1.49.Final]
    at io.vertx.core.buffer.impl.BufferImpl.<init>(BufferImpl.java:44) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.buffer.impl.BufferFactoryImpl.buffer(BufferFactoryImpl.java:49) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.buffer.Buffer.buffer(Buffer.java:94) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.file.impl.FileSystemImpl$16.perform(FileSystemImpl.java:862) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.file.impl.FileSystemImpl$16.perform(FileSystemImpl.java:857) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.file.impl.FileSystemImpl$BlockingAction.handle(FileSystemImpl.java:971) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.file.impl.FileSystemImpl$BlockingAction.handle(FileSystemImpl.java:951) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:313) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.impl.ContextImpl$$Lambda$175/0x000000084027b440.run(Unknown Source) ~[?:?]
    at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[vertx-core-3.9.1.jar:3.9.1]
    at io.vertx.core.impl.TaskQueue$$Lambda$151/0x0000000840255c40.run(Unknown Source) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.49.Final.jar:4.1.49.Final]
    at java.lang.Thread.run(Thread.java:834) [?:?]

标签： filevert.x

解决方案

该readFile方法将整个文件加载到内存中，这就是您获得OutOfMemoryError大文件的原因。

Vert.x FileSystemAPI 有一个copy你应该使用的方法：

FileSystem fs = vertx.fileSystem();

// Copy file from foo.txt to bar.txt
fs.copy("foo.txt", "bar.txt", res -> {
  if (res.succeeded()) {
    // Copied ok!
  } else {
    // Something went wrong
  }
});

推荐阅读