apache-camel - Camel 2.18 流缓存：java.lang.OutOfMemoryError：Java 堆空间

Question

我正在使用 Camel 2.18 和 Java 8。下面是一些带有路由的代码。我将此类 JSON 发布到此服务：

{
    "bigField": "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttest",
    "smallField": "234"
}

问题是 bigField 相当大，可以是 256MB。这就是为什么我正在考虑在 Camel 中进行流缓存：http ://camel.apache.org/stream-caching.html不确定是否可以在我的场景中应用它？我的理解是这种机制应该将流一点一点地存储在磁盘上？我创建了 /tmp/cachedir 并看到了 Camel 如何创建和删除一些临时文件。

6:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos7138988885969104934.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos7138988885969104934.tmp with result: true
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos8802932624903146810.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos8802932624903146810.tmp with result: true
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos91165405800652292.tmp
16:15:23.505 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos91165405800652292.tmp with result: true
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Retrying attempt 0 to delete file: /tmp/cachedir/cos6540772182497948066.tmp
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.apache.camel.util.FileUtil - Tried 1 to delete file: /tmp/cachedir/cos6540772182497948066.tmp with result: true
16:15:23.506 [qtp422330142-21 - /myservice/v1/oper] DEBUG org.eclipse.jetty.server.Server - RESPONSE /myservice/v1/oper  201

所以它似乎正在工作。但我正在使用这种机制来保护自己免受 OutOfMemory 异常的影响。所以我写了 Junit 发送 bigField 是这样创建的：

headers.setAll(map);
        char[] chars = new char[20000000];
        Arrays.fill(chars, 'a');
        String bigField = new String(chars);

我开始了我的 Spring Boot 应用程序 JAVA_OPTS="-Xms60m -Xmx128m"

我错过了什么？我认为这种流缓存机制可以保护我免受客户端发送大于 Xmx - 128MB 的主体事件的影响，但它在 20MB 时失败。

@SpringBootApplication
public class Application extends RouteBuilder {

@Override
    public void configure() throws Exception {

        setupStreamCaching();

        restConfiguration().host("0.0.0.0").port(PORT)
                .endpointProperty("headerFilterStrategy", "#myHeaderFilterStrategy")
                .endpointProperty("matchOnUriPrefix", "true")
                .endpointProperty("sendServerVersion", "false")
                .bindingMode(RestBindingMode.json);

        rest(API_CONTEXT + V1)
                .post(API_OPERATION)
                .type(RequestModel.class)
                .outType(Response.class)
                .consumes(APPLICATION_JSON)
                .produces(APPLICATION_JSON)
                .to(MAIN_ROUTE);

        from(MAIN_ROUTE)
                .routeId(MAIN_ROUTE)
                .process(requestValidator)
                .to(SERVICE_CALL)
                .end();


    private void setupStreamCaching() {
            getContext().getStreamCachingStrategy().setSpoolDirectory("/tmp/cachedir");
    getContext().getStreamCachingStrategy().setSpoolThreshold(64 * 1024);
    getContext().getStreamCachingStrategy().setBufferSize(16 * 1024);
        }

http://camel.apache.org/stream-caching.html

3:30:24.584 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.io.ChannelEndPoint - 填充 3432 SelectChannelEndPoint@44b2fa12{/10.0.2.2:57125<->1080,Open,in,out ,-,-,0/30000,HttpConnection}{io=0,kio=0,kro=1} 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 oejetty.server.HttpConnection - HttpConnection@ 13dbbbcc[REFILLING,SelectChannelEndPoint@44b2fa12{/10.0.2.2:57125<->1080,打开,输入,输出,-,-,0/30000,HttpConnection}{io=0,kio=0,kro=1}][ p=HttpParser{s=CONTENT,19996672 of 20000104},g=HttpGenerator{s=START},c=HttpChannelOverHttp@4b6953ae{r=1,c=false,a=DISPATCHED,uri=/myservice/v1/oper}]填充 3432 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.http.HttpParser - parseNext s=CONTENT HeapByteBuffer@42f6e192[p=0,l=3432,c=8192, r=3432]={<<>>aaaaaaaaaaaaaaaaa...aaaaaaaaaaaaaaa} 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@4b6953ae{r=1,c=false,a=DISPATCHED,uri=/myservice /v1/oper} 内容 java.nio.HeapByteBufferR[pos=0 lim=3432 cap=8192] 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.http.HttpParser - parseNext s=CONTENT HeapByteBuffer@42f6e192[p=3432,l=3432,c=8192,r=0]={aaaaaaaaaaaaaaaaaa...-21T17:32:28Z"}<<<>>>aaaaaaaaaaaaaaaa...aaaaaaaaaaaaaaaa} 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.http.HttpParser - 内容 --> 结束 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper]调试 org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@4b6953ae{r=1,c=false,a=DISPATCHED,uri=/myservice/v1/oper} messageComplete 13:30:24。585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.server.HttpInput - HttpInputOverHTTP@63602aee EOF 13:30:24.585 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse。 jetty.io.ChannelEndPoint - 填充 0 SelectChannelEndPoint@44b2fa12{/10.0.2.2:57125<->1080,Open,in,out,-,-,1/30000,HttpConnection}{io=0,kio=0,kro= 1} 13:30:24.586 [qtp1492219097-24 - /myservice/v1/oper] 调试 oejetty.server.HttpConnection - HttpConnection@13dbbbcc[REFILLING,SelectChannelEndPoint@44b2fa12{/10.0.2.2:57125<->1080,Open,in ,out,-,-,2/30000,HttpConnection}{io=0,kio=0,kro=1}][p=HttpParser{s=END,20000104 of 20000104},g=HttpGenerator{s=START}, c=HttpChannelOverHttp@4b6953ae{r=1,c=false,a=DISPATCHED,uri=/myservice/v1/oper}] 填充 0 13:30:24.586 [qtp1492219097-24 - /myservice/v1/oper] 调试组织。 eclipse.jetty.server。HttpInput - HttpInputOverHTTP@63602aee eof EOF 13:30:24.586 [qtp1492219097-24 - /myservice/v1/oper] 调试 org.eclipse.jetty.server.HttpInput - HttpInputOverHTTP@63602aee eof EOF 13:30:27.462 [qtp1492219097-24 - /myservice/v1/oper] WARN oejetty.servlet.ServletHandler - /myservice/v1/oper org.apache.camel.TypeConversionException：从类型转换时出错：java.lang.String 到所需类型：java.lang.String由于 java.lang.OutOfMemoryError: Java heap space 的值 [Body is instance of org.apache.camel.StreamCache]camel.TypeConversionException：从类型转换期间出错：java.lang.String 到所需类型：java.lang.String 值 [Body is instance of org.apache.camel.StreamCache] 由于 java.lang.OutOfMemoryError：Java 堆空间camel.TypeConversionException：从类型转换期间出错：java.lang.String 到所需类型：java.lang.String 值 [Body is instance of org.apache.camel.StreamCache] 由于 java.lang.OutOfMemoryError：Java 堆空间

java.lang.OutOfMemoryError: Java 堆空间在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.createTypeConversionException(BaseTypeConverterRegistry.java:629) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:150) ) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:78) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:53) 在 org.apache.camel.util.MessageHelper。 extractBodyAsString(MessageHelper.java:84) at org.apache.camel.component.rest.RestConsumerBindingProcessor.process(RestConsumerBindingProcessor.java:162) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org .apache.camel.processor.RedeliveryErrorHandler.process（RedeliveryErrorHandler.java：542) 在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) 在 org.apache.camel.processor.Pipeline.process(Pipeline.java:120) 在 org.apache.camel.processor.Pipeline .process(Pipeline.java:83) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) at org.apache.camel.component.jetty.CamelContinuationServlet.doService(CamelContinuationServlet.java:191) org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle( ServletHolder.java:812) 在 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) 在 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127）在 org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) 在 org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) 在 org.eclipse.jetty.server .handler.ScopedHandler.handle(ScopedHandler.java:141) 在 org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) 在 org.eclipse.jetty.server.Server.handle(Server.java :499) 在 org.eclipse.jetty.io 的 org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) 的 org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)。 AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool. java:555) 在 java.lang.Thread.run(Thread.java:748) 引起：org.apache.camel.RuntimeCamelException: java.lang.OutOfMemoryError: Java heap space at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1763) at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1358) 在 org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) 在 org.apache.camel.impl.converter。 BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个常见框架org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1358) 的 wrapRuntimeCamelException(ObjectHelper.java:1763) org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) .apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个常用帧org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1358) 的 wrapRuntimeCamelException(ObjectHelper.java:1763) org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) .apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个常用帧camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个常用帧camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) ...省略了27个常用帧

引起：java.lang.OutOfMemoryError：java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) 处 java.util.Arrays.copyOf(Arrays.java:3332) 处 java.lang.AbstractStringBuilder.append 处的 Java 堆空间(AbstractStringBuilder.java:596) 在 java.lang.StringBuilder.append(StringBuilder.java:190) 在 org.apache.camel.converter.IOConverter.toString(IOConverter.java:318) 在 org.apache.camel.converter。 IOConverter.toString(IOConverter.java:307) at org.apache.camel.converter.IOConverter.toString(IOConverter.java:364) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) 在 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 在 java.lang.reflect.Method。在 org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:59) at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1354) 调用(Method.java:498) .apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:306) 在 org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:133) 在 org.apache.camel.impl.MessageSupport .getBody(MessageSupport.java:78) 在 org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:53) 在 org.apache.camel.util.MessageHelper.extractBodyAsString(MessageHelper.java:84) 在 org. org.apache.camel.management 中的 apache.camel.component.rest.RestConsumerBindingProcessor.process(RestConsumerBindingProcessor.java:162)。InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) at org .apache.camel.processor.Pipeline.process(Pipeline.java:120) 在 org.apache.camel.processor.Pipeline.process(Pipeline.java:83) 在 org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor .java:197）在 org.apache.camel.component.jetty.CamelContinuationServlet.doService(CamelContinuationServlet.java:191) 在 org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) 在 javax。 servlet.http.HttpServlet.service(HttpServlet.java:790) 在 org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) 在 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) 在 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)

标签： apache-camel