java - 调用外部 Python 进程时超出 GC 开销限制

Question

我有一个调用python脚本的java程序script.py如下

Process p1 = Runtime.getRuntime().exec("python3 /home/user/script.py " + id + " " + text);
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p1.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(p1.getErrorStream()));
String s;
StringBuilder t = new StringBuilder();
boolean isT = false;
while ((s = stdInput.readLine()) != null) {
    isT = true;
    t.append(s);
}
if (isT) {
    stdInput.close();
    stdError.close();
    p1.destroy();
    return t.toString();
}
logger.info(text);
StringBuilder error_text = new StringBuilder();
while ((s = stdError.readLine()) != null) {
    error_text.append(s);
}
logger.info(error_text.toString());
stdInput.close();
stdError.close();
p1.destroy();
return text;

每当满足某些条件时，都会在循环中调用这段代码。我正在从 python 获取输出并使用p1.getInputStream().

python脚本如下

import sys
text = ''
id_ = sys.argv[1]

for w in sys.argv[2:]:
    text += w + ' '

text = text.replace('\n', ' ')

list_ = [text]
ret_val = my_function(list_)
with open('/home/user/output.txt', 'a') as out:
    for r in ret_val:
        print(r)
        out.write(id_ + '\t\t' + r + '\n')

该脚本在开始时运行良好，但随着时间的推移，我收到以下错误

Exception in thread "..." java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Arrays.java:3664)
    at java.lang.StringBuffer.toString(StringBuffer.java:669)
    at java.io.BufferedReader.readLine(BufferedReader.java:359)
    at java.io.BufferedReader.readLine(BufferedReader.java:389)
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.util.LinkedHashMap.newNode(LinkedHashMap.java:256)
    at java.util.HashMap.putVal(HashMap.java:631)
    at java.util.HashMap.put(HashMap.java:612)
    ...

即使在此之后重复调用python脚本并且一切正常，但经过一段时间后，我不断收到如下错误

Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "..." java.lang.OutOfMemoryError: GC overhead limit exceeded
...

我的 python 脚本停止响应。

我试图通过使用stdInput.close()and来停止输入流读取器，stdError.close()并且我正在通过 using 杀死进程p1.destroy()，但我认为这并没有破坏子进程，或者我还缺少其他什么东西，我还有什么其他方法可以实现这一点。

谢谢你。

标签： javapythonprocess