首页 > 解决方案 > 基于 Postgresql 上的 LSN 跟踪异常事务

问题描述

我的 PostgreSQL 的 Debezium kafka 连接器有问题。它表明我有超过 wal 大小 1GB 的记录。这是完整的错误:

"org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:197)
tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:101)
tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
tat java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.postgresql.util.PSQLException: ERROR: out of memory
 Detail: Cannot enlarge string buffer containing 1073741385 bytes by 2500 more bytes.
 Where: slot "debezium", output plugin "wal2json", in the change callback, associated LSN 3D/66648BE8
tat org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2505)
tat org.postgresql.core.v3.QueryExecutorImpl.processCopyResults(QueryExecutorImpl.java:1165)
tat org.postgresql.core.v3.QueryExecutorImpl.readFromCopy(QueryExecutorImpl.java:1072)
tat org.postgresql.core.v3.CopyDualImpl.readFromCopy(CopyDualImpl.java:37)
tat org.postgresql.core.v3.replication.V3PGReplicationStream.receiveNextData(V3PGReplicationStream.java:158)
tat org.postgresql.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:123)
tat org.postgresql.core.v3.replication.V3PGReplicationStream.readPending(V3PGReplicationStream.java:80)
tat io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:402)
tat io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:125)
t... 6 more"

我想追踪引发此错误的“异常”操作在哪里。我只有一个来自错误的 LSN。我对数据库维护尤其是 PostgreSQL 还是很陌生。任何帮助表示赞赏。

更新 这是我从 pg_log 得到的完全相同的消息,感谢@laurenz-albe 指出我应该在哪里搜索问题:

2020-07-08 13:59:13 ICT [25137-23] kafka@aaa ERROR:  out of memory
2020-07-08 13:59:13 ICT [25137-24] kafka@aaa DETAIL:  Cannot enlarge string buffer containing 1073741385 bytes by 2500 more bytes.
2020-07-08 13:59:13 ICT [25137-25] kafka@aaa CONTEXT:  slot "debezium", output plugin "wal2json", in the change callback, associated LSN 3D/66648BE8

但是,如果我无法了解触发事件的 LSN 目前发生了什么,那仍然是一无所知

标签: postgresql

解决方案


推荐阅读