cassandra - 将数据从本地 cassandra 集群流式传输到 AWS 时出错(版本:2.1.19)
问题描述
尝试在目标 DC 上重建源 DC 时出现错误。我想将源数据复制到 AWS 中的目标。请帮忙。
INFO [MemtableFlushWriter:5] 2021-05-29 20:19:40,664 Memtable.java:382 - Completed flushing /data/system_auth/users-473588ad9c7938be8b59e06c10456ba0/system_auth-users-tmp-ka-1-Data.db (0.000KiB) for commitlog position ReplayPosition(segmentId=1622318449434, position=9380187)
INFO [STREAM-IN-/10.0.31.101] 2021-05-29 20:19:40,679 StreamResultFuture.java:181 - [Stream #327c5b00-c0bb-11eb-aa90-bb6530dd5190] Session with /10.0.31.101 is complete
INFO [STREAM-IN-/10.0.31.101] 2021-05-29 20:19:40,681 StreamResultFuture.java:213 - [Stream #327c5b00-c0bb-11eb-aa90-bb6530dd5190] All sessions completed
INFO [HANDSHAKE-/10.0.31.101] 2021-05-29 20:20:37,126 OutboundTcpConnection.java:496 - Handshaking version with /10.0.31.101
INFO [CompactionExecutor:4] 2021-05-29 20:20:50,217 CompactionTask.java:141 - Compacting [SSTableReader(path='/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/system-hints-ka-2-Data.db')]
INFO [RMI TCP Connection(36)-127.0.0.1] 2021-05-29 20:21:27,805 StorageService.java:1052 - rebuild from dc: DC1
INFO [RMI TCP Connection(36)-127.0.0.1] 2021-05-29 20:21:28,131 StreamResultFuture.java:87 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86] Executing streaming plan for Rebuild
INFO [StreamConnectionEstablisher:1] 2021-05-29 20:21:28,132 StreamSession.java:224 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86] Starting streaming to /172.0.0.147
INFO [StreamConnectionEstablisher:1] 2021-05-29 20:21:28,134 StreamCoordinator.java:209 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86, ID#0] Beginning stream session with /172.0.0.147
INFO [STREAM-IN-/172.0.0.147] 2021-05-29 20:21:28,353 StreamResultFuture.java:167 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86 ID#0] Prepare completed. Receiving 15 files(85742295230 bytes), sending 0 files(0 bytes)
ERROR [STREAM-IN-/172.0.0.147] 2021-05-29 20:26:18,026 StreamSession.java:512 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86] Streaming error occurred
java.io.EOFException: null
at java.io.DataInputStream.readInt(DataInputStream.java:392) ~[na:1.7.0_151]
at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.deserialize(CompressionInfo.java:68) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.compress.CompressionInfo$CompressionInfoSerializer.deserialize(CompressionInfo.java:47) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.messages.FileMessageHeader$FileMessageHeaderSerializer.deserialize(FileMessageHeader.java:188) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:42) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(IncomingFileMessage.java:38) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:56) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:276) ~[apache-cassandra-2.1.19.jar:2.1.19]
at java.lang.Thread.run(Thread.java:748) [na:1.7.0_151]
INFO [STREAM-IN-/172.0.0.147] 2021-05-29 20:26:18,202 StreamResultFuture.java:181 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86] Session with /172.0.0.147 is complete
WARN [STREAM-IN-/172.0.0.147] 2021-05-29 20:26:18,203 StreamResultFuture.java:208 - [Stream #728af9e0-c0bb-11eb-9dde-0d382aaf9f86] Stream failed
ERROR [RMI TCP Connection(36)-127.0.0.1] 2021-05-29 20:26:18,203 StorageService.java:1075 - Error while rebuilding node
org.apache.cassandra.streaming.StreamException: Stream failed
at org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85) ~[apache-cassandra-2.1.19.jar:2.1.19]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1172) ~[guava-16.0.jar:na]
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na]
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) ~[guava-16.0.jar:na]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) ~[guava-16.0.jar:na]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202) ~[guava-16.0.jar:na]
at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:209) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:185) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:413) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.StreamSession.onError(StreamSession.java:518) ~[apache-cassandra-2.1.19.jar:2.1.19]
at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:294) ~[apache-cassandra-2.1.19.jar:2.1.19]
at java.lang.Thread.run(Thread.java:748) ~[na:1.7.0_151]
这是我的节点工具状态
root@e7816844afae:/bin# nodetool status
Datacenter: DC1
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 172.0.0.147 80.31 GB 256 100.0% 38f37d99-496b-4316-bca7-b10a1f161904 RAC1
UN 172.0.0.161 80.45 GB 256 100.0% cfb05b5a-fcbe-4686-873a-be7aaa840784 RAC1
UN 172.0.0.234 80.38 GB 256 100.0% 2b2a5739-df20-4083-8fd0-ab9de8a228c9 RAC1
Datacenter: DC3
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 10.0.3.61 123.34 KB 256 100.0% 1b3299a7-3003-4c98-bd0d-ce2ef8666edc RAC1
UN 10.0.31.101 109.36 KB 256 100.0% d0fb8a71-a38e-4449-b6f4-2ff7448fb263 RAC1
UN 10.0.30.118 160.96 KB 256 100.0% ebe0f32c-9d72-4327-9fc1-bd9a2c7b1b82 RAC1
[root@ip-172-0-0-234 bin]# ./nodetool describecluster
Cluster Information:
Name: XXXXXX
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
0850c5dc-99c9-3770-9c7c-d23e19d194b6: [172.0.0.147, 172.0.0.161, 10.0.3.61, 10.0.31.101, 10.0.30.118, 172.0.0.234]
cassandra@cqlsh> select * from system.schema_keyspaces;
keyspace_name | durable_writes | strategy_class | strategy_options
---------------------+----------------+------------------------------------------------------+-----------------------
user_activity_vault | True | org.apache.cassandra.locator.NetworkTopologyStrategy | {"DC1":"3","DC3":"3"}
system_auth | True | org.apache.cassandra.locator.NetworkTopologyStrategy | {"DC1":"3","DC3":"3"}
system | True | org.apache.cassandra.locator.LocalStrategy | {}
system_traces | True | org.apache.cassandra.locator.NetworkTopologyStrategy | {"DC1":"3","DC3":"3"}
非常感谢任何解决此问题的帮助和建议
谢谢, 加吉
解决方案
推荐阅读
- python - Django如何制作数据透视表?
- python - 使用 Python 观察“成功”或“失败”字符串的日志
- python - 时间戳 YYYY-MM-DDThh:mmTZD python
- python - 在不创建 Web 应用程序的情况下呈现交互式仪表板的最佳方式是什么
- python - 使用 Python Pandas 合并 .dat 文件
- javascript - 如何从数组中提取经纬度对象并将它们推入新数组
- python - 股票预测的特征提取
- loops - 如何在 Ansible 中循环inventory_hostname
- angular - 发生未处理的异常:NGCC 失败。当我使用 ng serve 编译时
- c# - 当我使用 PropertyChanged 时 WPF ItemsControl 不刷新