首页 > 解决方案 > java.lang.OutOfMemoryError:直接缓冲内存 Apache Ignite

问题描述

最近我们使用 ignite 2.10.0 迁移到 64 位 JVM,我们的缓存配置如下所示(在 2.9.0 上工作正常),

_cfg = new IgniteConfiguration()
{                           
    IgniteInstanceName = MKT_TICK_DATA_CACHE,                               
    DiscoverySpi = new TcpDiscoverySpi
    {
        LocalPort = 48500,
        LocalPortRange = 60,
        IpFinder = new TcpDiscoveryStaticIpFinder
        {
            Endpoints = new[] { "127.0.0.1:48500..48560" }
        }
    },
    CommunicationSpi = new TcpCommunicationSpi
    {
        LocalPort = 48100
    },
    DataStorageConfiguration = new DataStorageConfiguration
    {
        DefaultDataRegionConfiguration = new DataRegionConfiguration
        {
            Name = MKT_TICK_DATA_CACHE,
            InitialSize = 500L * 1024 * 1024,
            MaxSize =     4L * 1024 * 1024 * 1024,
            PersistenceEnabled = false,
            PageEvictionMode = Apache.Ignite.Core.Configuration.DataPageEvictionMode.Random2Lru
        },
        SystemRegionInitialSize = 100 * 1024 * 1024
    },
    ClientConnectorConfiguration = new ClientConnectorConfiguration
    {
        // Set a port range from 10000 to 10005
        Port = 10800,
        PortRange = 60,
        HandshakeTimeout = TimeSpan.FromSeconds(30),
        IdleTimeout = Timeout.InfiniteTimeSpan,
        MaxOpenCursorsPerConnection = 10000,
        SocketReceiveBufferSize = 100 * 1024 * 1024,
        SocketSendBufferSize = 100 * 1024 * 1024,
        TcpNoDelay = true,
        ThinClientEnabled = true,
        ThreadPoolSize = 256
    },
    JvmOptions = new System.Collections.Generic.List<String>() { "-Xms1g", "-Xmx20g", "-XX:+AlwaysPreTouch", "-XX:+UseG1GC", "-XX:+ScavengeBeforeFullGC", "-XX:+DisableExplicitGC", "-Djava.net.preferIPv4Stack=true", "-XX:MaxDirectMemorySize=30G" }
    //JvmOptions = new System.Collections.Generic.List<String>() { "-XX:+AlwaysPreTouch", "-XX:+UseG1GC", "-XX:+ScavengeBeforeFullGC", "-XX:+DisableExplicitGC", "-Djava.net.preferIPv4Stack=true" }
    //JvmOptions = new[] { "-Xmx10g" },//, "-XX:+AlwaysPreTouch", "-XX:+UseG1GC", "-XX:+ScavengeBeforeFullGC", "-XX:+DisableExplicitGC" }
};

_cfg.ServiceThreadPoolSize = 256;
_cfg.SystemThreadPoolSize = 128;
_cfg.StripedThreadPoolSize = 128;
_cfg.WorkDirectory = _dir;
var cacheConfig = new CacheConfiguration
{
    Name = MKT_TICK_DATA_CACHE,
    CacheMode = CacheMode.Partitioned,
    Backups = 0,                            
    AtomicityMode = CacheAtomicityMode.Atomic,
    LoadPreviousValue = false,
    OnheapCacheEnabled = true,
    WriteBehindEnabled = false,
    WriteSynchronizationMode = CacheWriteSynchronizationMode.FullAsync,
    MaxConcurrentAsyncOperations = 300000
};
_cfg.CacheConfiguration = new CacheConfiguration[] { cacheConfig };
Ignition.ClientMode = false;
var ignite = Ignition.Start(_cfg);
_cache = ignite.GetOrCreateCache<Int32, MarketTickS>(MKT_TICK_DATA_CACHE);
_oCache = ignite.GetOrCreateCache<Int64, byte[]>(ORION_L_CACHE);
_oSecCache = ignite.GetOrCreateCache<Int64, CacheSecurity>(ORION_SEC_CACHE);

我们的缓存在运行大约 15 分钟后崩溃。这个缓存的作用是存储来自市场的报价数据,以每秒 60K 报价的速率传入。

[14:03:01,528][SEVERE][grid-nio-worker-client-listener-13-#197%QTICKDATA%][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: Direct buffer memory]]

你能帮忙吗?

*********** 更新 ******************************

更新,在服务器上重新安装 JDK 后。连续运行 60 分钟后未发生崩溃。然而,仍然好奇 Ignite 2.9 需要 3.X GB 总内存,但 ignite 2.10 需要 32.x GB 总内存,具有相同的缓存配置(相同的 C# 二进制文件集,除了 ignite 使用 32.x GB 内存是 2.10,并且使用 3.x GB 内存。 x GB 内存为 2.9)。下面附上截图,

Ignite 2.10 - 任务管理器

2.9, Ignite 2.9 - 任务管理器

标签: jvmignite

解决方案


尝试移除-XX:+DisableExplicitGC开关,已知会干扰直接缓冲内存分配,因此不再推荐。

不知道为什么它比以前需要更多的 RAM,需要更多详细信息。


推荐阅读