mongodb - 辅助数据库意外崩溃
问题描述
Primary 和 Secondary 在 Windows server 2012 R2 Standard 上运行。今天主要机器下来了。之后，由于 Got 信号的以下错误：22 (SIGABRT)，Secondary Mongo 实例关闭。然后 Primary 进入恢复模式。主服务器不打印任何清晰的日志。
主服务器日志：
2021-07-23T19:13:53.237+0530 I REPL [replexec-23] Member SecondaryHost:27017 is now in state RS_DOWN 2021-07-23T19:13:53.237+0530 I REPL [replexec-23] can't see a majority of the set, relinquishing primary 2021-07-23T19:13:53.237+0530 I REPL [replexec-23] Stepping down from primary in response to heartbeat 2021-07-23T19:13:53.237+0530 I REPL [replexec-23] transition to RECOVERING from PRIMARY 2021-07-23T19:13:53.238+0530 I NETWORK [replexec-23] Skip closing connection for connection # 11139
今天我们的 UAT 主服务器突然进入恢复模式，辅助服务器没有起床。从日志中，我得到了来自辅助服务器的错误。只是想知道为什么会这样。
辅助服务器日志：
2021-07-23T19:05:26.900+0530 E STORAGE [WTCheckpointThread] WiredTiger error (13) [1627047326:876858][1540:140704437702784], file:ReconDB_UAT/collection/26--5782211086932514962.wt, WT_SESSION.checkpoint: __win_open_file, 543: D:\MongoData\Data\ReconDB_UAT/collection/26--5782211086932514962.wt: handle-open: CreateFileW: Access is denied.
: Permission denied Raw: [1627047326:876858][1540:140704437702784], file:ReconDB_UAT/collection/26--5782211086932514962.wt, WT_SESSION.checkpoint: __win_open_file, 543: D:\MongoData\Data\ReconDB_UAT/collection/26--5782211086932514962.wt: handle-open: CreateFileW: Access is denied.
: Permission denied
2021-07-23T19:05:26.900+0530 F - [WTCheckpointThread] Invariant failure: s->checkpoint(s, "use_timestamp=true") resulted in status UnknownError: 13: Permission denied at src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp 352
2021-07-23T19:05:26.900+0530 F - [WTCheckpointThread]
***aborting after invariant() failure
2021-07-23T19:05:28.595+0530 I ASIO [Replication] Connecting to ArbHost:27017
2021-07-23T19:05:28.642+0530 I ASIO [Replication] Failed to connect to ArbHost:27017 - AuthenticationFailed: Authentication failed.
2021-07-23T19:05:28.642+0530 I CONNPOOL [Replication] Dropping all pooled connections to ArbHost:27017 due to AuthenticationFailed: Authentication failed.
2021-07-23T19:05:28.642+0530 I REPL_HB [replexec-99] Error in heartbeat (requestId: 1211726) to ArbHost:27017, response status: AuthenticationFailed: Authentication failed.
2021-07-23T19:05:30.643+0530 I ASIO [Replication] Connecting to ArbHost:27017
2021-07-23T19:05:30.689+0530 I ASIO [Replication] Failed to connect to ArbHost:27017 - AuthenticationFailed: Authentication failed.
2021-07-23T19:05:30.689+0530 I CONNPOOL [Replication] Dropping all pooled connections to ArbHost:27017 due to AuthenticationFailed: Authentication failed.
2021-07-23T19:05:30.690+0530 I REPL_HB [replexec-99] Error in heartbeat (requestId: 1211729) to ArbHost:27017, response status: AuthenticationFailed: Authentication failed.
2021-07-23T19:05:32.691+0530 I ASIO [Replication] Connecting to ArbHost:27017
2021-07-23T19:05:32.736+0530 I ASIO [Replication] Failed to connect to ArbHost:27017 - AuthenticationFailed: Authentication failed.
2021-07-23T19:05:32.736+0530 I CONNPOOL [Replication] Dropping all pooled connections to ArbHost:27017 due to AuthenticationFailed: Authentication failed.
2021-07-23T19:05:32.736+0530 I REPL_HB [replexec-99] Error in heartbeat (requestId: 1211732) to ArbHost:27017, response status: AuthenticationFailed: Authentication failed.
2021-07-23T19:05:32.952+0530 F - [WTCheckpointThread] Got signal: 22 (SIGABRT).
mongod.exe ...\src\mongo\util\stacktrace_windows.cpp(247) mongo::printStackTrace+0x43
mongod.exe ...\src\mongo\util\signal_handlers_synchronous.cpp(184) mongo::`anonymous namespace'::printSignalAndBacktrace+0x74
mongod.exe ...\src\mongo\util\signal_handlers_synchronous.cpp(240) mongo::`anonymous namespace'::abruptQuit+0x85
ucrtbase.DLL raise+0x1e7
ucrtbase.DLL abort+0x31
mongod.exe ...\src\mongo\util\assert_util.cpp(129) mongo::invariantOKFailed+0x228
mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp(352) mongo::WiredTigerKVEngine::WiredTigerCheckpointThread::run+0x661
mongod.exe ...\src\mongo\util\background.cpp(153) mongo::BackgroundJob::jobBody+0x131
mongod.exe c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread(247) std::_LaunchPad<std::unique_ptr<std::tuple<<lambda_8929d5e2502e4de5883694c1a7a329eb> >,std::default_delete<std::tuple<<lambda_8929d5e2502e4de5883694c1a7a329eb> > > > >::_Run+0x77
mongod.exe c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread(210) std::_Pad::_Call_func+0x9
ucrtbase.DLL o__realloc_base+0x60
KERNEL32.DLL BaseThreadInitThunk+0x22
2021-07-23T19:05:33.006+0530 F CONTROL [WTCheckpointThread] *** unhandled exception 0x0000000E at 0x00007FF84DC27AFC, terminating
2021-07-23T19:05:33.006+0530 F CONTROL [WTCheckpointThread] *** stack trace for unhandled exception:
2021-07-23T19:05:33.021+0530 I - [WTCheckpointThread] KERNELBASE.dll RaiseException+0x6c
mongod.exe ...\src\mongo\util\signal_handlers_synchronous.cpp(241) mongo::`anonymous namespace'::abruptQuit+0x9d
ucrtbase.DLL raise+0x1e7
ucrtbase.DLL abort+0x31
mongod.exe ...\src\mongo\util\assert_util.cpp(129) mongo::invariantOKFailed+0x228
mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp(352) mongo::WiredTigerKVEngine::WiredTigerCheckpointThread::run+0x661
mongod.exe ...\src\mongo\util\background.cpp(153) mongo::BackgroundJob::jobBody+0x131
mongod.exe c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread(247) std::_LaunchPad<std::unique_ptr<std::tuple<<lambda_8929d5e2502e4de5883694c1a7a329eb> >,std::default_delete<std::tuple<<lambda_8929d5e2502e4de5883694c1a7a329eb> > > > >::_Run+0x77
mongod.exe c:\program files (x86)\microsoft visual studio 14.0\vc\include\thr\xthread(210) std::_Pad::_Call_func+0x9
ucrtbase.DLL o__realloc_base+0x60
KERNEL32.DLL BaseThreadInitThunk+0x22
2021-07-23T19:05:33.022+0530 I CONTROL [WTCheckpointThread] writing minidump diagnostic file C:\Program Files\MongoDB\Server\4.2021-07-23T13-35-33.mdmp
2021-07-23T19:13:01.047+0530 I NETWORK [listener] connection accepted from PrimaryHost:51006 #182621 (15 connections now open)