logstash - 需要帮助 - logstash 因 OOM 错误而崩溃

Question

Logstash 版本 - 6.2.0 OpenJDK

日志存储.yml

pipeline.workers: 8
pipeline.batch.size: 250
pipleline.batch.delay: 5
log.level: warn
path.log : /usr/share/test/logstash

它在 kubernetes 的 pod 中运行，最小 4g 和最大 6g 内存。pod 使用一段时间后因 OOM 错误而崩溃。在分析堆时发现了这一点。

问题嫌疑人1

由 "org.jruby.util.JRubyClassLoader @ 0x6c3992e80" 加载的 68 个 "io.netty.util.concurrent.DefaultEventExecutor" 实例占用 4,364,500,264 (83.46%) 字节。

最大实例：

   io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3828eb0 - 112,381,328 
    (2.15%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f63c8 - 108,620,792 
    (2.08%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817e08 - 100,920,304     
     (    1.93%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456138 - 100,211,048     
     (1.92%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456888 - 95,707,344 
    (1.83%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456820 - 91,473,176     
    (1.75%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817e70 - 89,591,048 
    (1.71%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d381a9e8 - 88,260,400 
    (1.69%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39a9538 - 88,139,568 
     (1.69%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817c68 - 85,195,688 
    (1.63%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f3578 - 83,102,248 
    (1.59%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456750 - 82,410,056 
    (1.58%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f3510 - 81,845,608 
     (1.56%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f35e0 - 81,734,736 
     (1.56%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3923900 - 81,014,544 
    (1.55%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d381a918 - 80,729,792 
     (1.54%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d38412d8 - 78,563,144 
    (1.50%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3840f98 - 77,632,384 
    (1.48%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39267b8 - 77,357,744 
    (1.48%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39a8ff0 - 75,636,776 
    (1.45%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d34566e8 - 75,467,808 
    (1.44%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39a8f20 - 75,290,592 
    (1.44%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39a8f88 - 75,134,256 
    (1.44%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d383e280 - 74,776,152 
   (1.43%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d34564e0 - 72,343,896 
    (1.38%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f6228 - 71,541,520 
    (1.37%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456478 - 71,489,152 
     (1.37%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f36b0 - 71,307,352 
    (1.36%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3844878 - 70,976,424 
    (1.36%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3828e48 - 70,949,224     
    (1.36%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3844810 - 70,415,992 (1.35%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d34565b0 - 69,949,520 
    (1.34%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d381a980 - 69,782,704 
     (1.33%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d381a8b0 - 69,389,880 (1.33%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3841000 - 66,237,624 
     (1.27%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3926750 - 65,789,608 
     (1.26%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d34561a0 - 65,720,384 
     ( 1.26%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39266e8 - 64,684,776 
     (1.24%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3828de0 - 63,547,248 
     (1.22%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817b30 - 63,459,752 
    (1.21%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39a90c0 - 61,840,136 
    (1.18%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3923968 - 61,683,728 
    (1.18%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456618 - 61,615,632 
    (1.18%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817d38 - 60,920,120 
     (1.16%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817cd0 - 60,482,864 
    (1.16%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f3648 - 59,936,400 
     (1.15%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d38449b0 - 58,924,864 
    (1.13%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d38448e0 - 57,296,960 
    (1.10%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d381a7e0 - 56,577,712 
    (1.08%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d39f6290 - 55,650,808 
    (1.06%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d34567b8 - 55,516,976 
    (1.06%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3456680 - 54,447,392 
    (1.04%) bytes.
     io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3844948 - 53,432,888 
    (1.02%) bytes.
    io.netty.util.concurrent.DefaultEventExecutor @ 0x6d3817c00 - 52,301,264 
    (1.00%) bytes.

标签： logstash