java - debezium mysql 连接器因 java 堆空间问题而失败
问题描述
我们在 GKE 集群上部署了debezium mysql连接器。但是 pod 失败了 Caused by: java.lang.OutOfMemoryError: Java heap space 。将 jvm 选项增加到 10g,但仍然因同样的问题而失败。
以下是我的配置属性:
singleTopicMode=true
databaseManagementSystem=mysql
debezium.offset.flush.timeout.ms=60000
debezium.offset.flush.interval.ms=10000
debezium.max.request.size=1048
debezium.max.queue.size=65536
debezium.max.batch.size=16384
debezium.HEAP_OPTS='-Xms10g -Xmx10g'
完整的日志:
org.apache.kafka.connect.errors.ConnectException: java.lang.OutOfMemoryError: Java heap space
at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:275)
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:49)
at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:791)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596)
at java.lang.StringBuilder.append(StringBuilder.java:190)
at java.io.BufferedReader.readLine(BufferedReader.java:358)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at java.nio.file.Files.readAllLines(Files.java:3205)
at io.debezium.relational.history.FileDatabaseHistory.lambda$recoverRecords$2(FileDatabaseHistory.java:136)
at io.debezium.relational.history.FileDatabaseHistory$$Lambda$181/230925968.run(Unknown Source)
at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:96)
at io.debezium.relational.history.FileDatabaseHistory.recoverRecords(FileDatabaseHistory.java:133)
at io.debezium.relational.history.AbstractDatabaseHistory.recover(AbstractDatabaseHistory.java:73)
at io.debezium.connector.mysql.MySqlSchema.loadHistory(MySqlSchema.java:252)
at io.debezium.connector.mysql.MySqlTaskContext.loadHistory(MySqlTaskContext.java:165)
at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:105)
... 7 more
解决方案
推荐阅读
- hyperledger-composer - 如何在 hyperledger-composer 事务处理器中定义 BusinessNetworkConnection?
- mysql - 安装 file_key_management
- android - 带有动画的自定义圆形切换按钮
- java - 无法在 AWS-Lambda 项目中导入“com.amazonaws.services.lambda.runtime.events.SQSEvent”
- javascript - 过滤/排序数组
- android - 即使安装了 USB 驱动程序,我也无法找到我的设备
- java - Shrinkwrap maven-resolver
- python - klepto 中的过期缓存
- php - PHP如何将文件内容加载为XML变量
- ballerina - 无法使 MySQL 客户端示例工作