java - 内存已满和 GC 需要很长时间

Question

1. 当上游接口变慢时，应用程序 GC 需要很长时间并且没有响应。当我使用 jstack pid 时线程被阻塞。

java -Xmx4g -Xms4g -Xmn1g -XX:MetaspaceSize=256M -XX:+PrintGCDateStamps -Xloggc:/tmp/http_gc.log -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC

2019-03-26T22:58:15.846+0800: 25665.934: [GC (Allocation Failure) [PSYoungGen: 4175385K->14818K(4179456K)] 4411887K->541974K(8373760K), 19.0466380 secs] [Times: user=149.46 sys=0.56, real=19.05 secs]
2019-03-26T22:58:37.604+0800: 25687.692: [GC (Allocation Failure) [PSYoungGen: 4179426K->14832K(3154432K)] 4706582K->1684012K(7348736K), 68.8601905 secs] [Times: user=537.37 sys=4.03, real=68.86 secs]
2019-03-26T22:59:48.585+0800: 25758.673: [GC (Allocation Failure) [PSYoungGen: 3154416K->527352K(3666944K)] 4823596K->2421091K(7861248K), 47.9907468 secs] [Times: user=373.32 sys=2.99, real=47.99 secs]
2019-03-26T23:00:38.452+0800: 25808.540: [GC (Allocation Failure) [PSYoungGen: 3666936K->527346K(2592768K)] 5560675K->3195781K(6787072K), 62.5120261 secs] [Times: user=485.91 sys=5.77, real=62.51 secs]
2019-03-26T23:01:42.212+0800: 25872.300: [GC (Allocation Failure) [PSYoungGen: 2592754K->424209K(3129856K)] 5261189K->3619241K(7324160K), 53.6535209 secs] [Times: user=417.13 sys=4.36, real=53.65 secs]
2019-03-26T23:02:37.104+0800: 25927.192: [GC (Allocation Failure) [PSYoungGen: 2489617K->417182K(3204096K)] 5684649K->4033835K(7398400K), 50.0956018 secs] [Times: user=389.61 sys=4.15, real=50.09 secs]
2019-03-26T23:03:27.200+0800: 25977.288: [Full GC (Ergonomics) [PSYoungGen: 417182K->0K(3204096K)] [ParOldGen: 3616653K->3881724K(4194304K)] 4033835K->3881724K(7398400K), [Metaspace: 109373K->108606K(1153024K)], 81.2032824 secs] [Times: user=620.26 sys=16.53, real=81.21 secs]
2019-03-26T23:04:49.719+0800: 26059.807: [Full GC (Ergonomics) [PSYoungGen: 2176000K->95467K(3204096K)] [ParOldGen: 3881724K->4194180K(4194304K)] 6057724K->4289648K(7398400K), [Metaspace: 108606K->108598K(1153024K)], 73.8241548 secs] [Times: user=562.17 sys=16.43, real=73.83 secs]
2019-03-26T23:06:04.738+0800: 26134.826: [Full GC (Ergonomics) [PSYoungGen: 2176000K->521564K(3204096K)] [ParOldGen: 4194180K->4194270K(4194304K)] 6370180K->4715835K(7398400K), [Metaspace: 108598K->108594K(1153024K)], 121.4262195 secs] [Times: user=925.74 sys=25.90, real=121.43 secs]
2019-03-26T23:08:06.936+0800: 26257.024: [Full GC (Ergonomics)

2. 我们必须使用带有上游接口的 hystrix 断路器，以及可调用我们自己的接口。

    @HystrixCommand(fallbackMethod = "fallBack",
            threadPoolProperties = {
                    @HystrixProperty(name = "coreSize", value = "20"),
                    @HystrixProperty(name = "maxQueueSize", value = "50"),
                    @HystrixProperty(name = "queueSizeRejectionThreshold", value = "20")},
            commandProperties = {
                    //使用线程隔离时，调用的超时时间，默认1s
                    @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1000"),
                    @HystrixProperty(name = "metrics.rollingStats.timeInMilliseconds", value = "5000"),
                    //出错率超过30%时开始熔断,默认50%
                    @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "30"),
                    @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10")
            })


        ResponseCallable callable = new ResponseCallable() {
            @Override
            public Response call() throws Exception {
                Response response = new Response();
                response.setData(commonService.getInfo(data));
                response.setSuccess();
                return response;

问题内存信息如下：

  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT
  0.00   0.00 100.00 100.00  92.82  89.17    122  306.980     4  276.454  583.434

 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT
1028096.0 990208.0  0.0    0.0   2176000.0 2176000.0 4194304.0  4193820.7  116992.0 108641.8 13312.0 11870.2    122  306.980  13   1515.771 1822.751
1028096.0 990208.0  0.0    0.0   2176000.0 2176000.0 4194304.0  4193820.7  116992.0 108641.8 13312.0 11870.2    122  306.980  13   1515.771 1822.751

下面是 jmap 实例：

 num     #instances         #bytes  class name
----------------------------------------------
   1:     137721956     3305326944  java.util.LinkedList$Node
   2:     137703668     2203258688  com.dangdang.ddframe.rdb.sharding.parsing.parser.expression.SQLIgnoreExpression

我无法附加进度以获取线程信息：

jstack 1095
1095: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

Server compiler detected.
JVM version is 25.65-b01

using thread-local object allocation.
Parallel GC with 38 thread(s)

Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 8589934592 (8192.0MB)
   NewSize                  = 4294967296 (4096.0MB)
   MaxNewSize               = 4294967296 (4096.0MB)
   OldSize                  = 4294967296 (4096.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 536870912 (512.0MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 2228224000 (2125.0MB)
   used     = 2228201192 (2124.9782485961914MB)
   free     = 22808 (0.02175140380859375MB)
   99.99897640452666% used
From Space:
   capacity = 1052770304 (1004.0MB)
   used     = 0 (0.0MB)
   free     = 1052770304 (1004.0MB)
   0.0% used
To Space:
   capacity = 1013972992 (967.0MB)
   used     = 0 (0.0MB)
   free     = 1013972992 (967.0MB)
   0.0% used
PS Old Generation
   capacity = 4294967296 (4096.0MB)
   used     = 4294824496 (4095.863815307617MB)
   free     = 142800 (0.1361846923828125MB)
   99.99667517840862% used

标签： java

Answer 1

首先，您必须检查您的错误代码。

其次，您可以使用这样的垃圾收集器调用程序

@Component
public class GarbageCollectorInvoker {
    private final Logger logger = LoggerFactory.getLogger(GarbageCollectorInvoker.class);

    @Scheduled(cron = "0 0/30 * * * *")
    public void execute() {
        logger.warn("============GC Status B4=================");
        logger.warn("Heap size free (mb): " + Runtime.getRuntime().freeMemory() / 1024 / 1024);
        logger.warn("Heap size total (mb): " + Runtime.getRuntime().totalMemory() / 1024 / 1024);
        logger.warn("Heap size max (mb): " + Runtime.getRuntime().maxMemory() / 1024 / 1024);

        System.gc();

        logger.warn("============GC Status After=================");
        logger.warn("Heap size free (mb): " + Runtime.getRuntime().freeMemory() / 1024 / 1024);
    }
}