首页 > 解决方案 > 与 Cassandra 1.2 相比,Cassandra 3 读取延迟巨大

问题描述

我们正在将我们的数据从旧 Cassandra (1.2) 迁移到最新的 3.11.6。我已经编写了迁移脚本,它将从旧 Cassandra 读取数据并触发对新 Cassandra 的查询(因此它将是新的插入)。

迁移后,当我尝试读取数据时,我发现新 Cassandra 中的一个表(2.5 秒)与旧表(600 毫秒)相比存在巨大延迟。

此表中的总列数接近 60k。没有聚类列和一个对每一行都是唯一的主键。

要求是为 150 个唯一的 flow_id 获取 40-50 列。我正在使用从我的应用程序异步触发的准备好的语句,一旦所有 CompletionFuture 完成,我正在解析最终结果。获取数据本身需要超过 2.5 秒。获取唯一 flow_id 结果的代码

allRows = keyList.stream().map((key) -> {

                    return session.executeAsync(boundStmt.setToken(partitionKeyName, key)
                            .setExecutionProfile(profile).setTracing(isTracingEnabled)).toCompletableFuture();

                }).collect(Collectors.toList()).stream().map(completableFuture -> {
                    AsyncResultSet asyncResultSet;
                    try {

                        //traceIfEnabled(asyncResultSet.getExecutionInfo(), isTracingEnabled);
                        asyncResultSet = completableFuture.get();

                    } catch (InterruptedException | ExecutionException e) {
                        throw new RuntimeGestorException(CassandraErrors.FAILURE, e, "moveForward");
                    }
                    return asyncResultSet.one();
                }).filter(row -> row != null).collect(Collectors.toList());     

它将生成的查询类型

SELECT src_flow_id,failures,author,authored_at,version_num,description,step_1_time,source,type,title,steps,url, labels,tags,title_lowercase,ent,user,host,votes,published_time,runs,views,flow_id FROM flows WHERE token(flow_id) = tokien(?)

占位符(?)将在哪里获取“钥匙”的令牌

行缓存被禁用。请让我知道我在这里缺少什么。

**Details -** 

 Old Partitoner OrderPreserve
 New Partitioner MurMur3

节点总数:4 复制因子:2

Machine configuration :

 12 Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz processor
 48GB of RAM

比照统计


  Read Count: 738  
  Read Latency: 8.633516260162601 ms   
  Write Count: 57  
  Write Latency: 0.1060701754385965 ms  
Pending Flushes: 0  
    Table: flows  
    SSTable count: 4  
    Space used (live): 2310544421  
    Space used (total): 2310544421  
    Space used by snapshots (total): 0  
    Off heap memory used (total): 595544  
    SSTable Compression Ratio: 0.9655157119293131  
    Number of partitions (estimate): 56942  
    Memtable cell count: 0  
    Memtable data size: 0  
    Memtable off heap memory used: 0  
    Memtable switch count: 0  
    Local read count: 37  
    Local read latency: 10.126 ms  
    Local write count: 0  
    Local write latency: NaN ms  
    Pending flushes: 0  
    Percent repaired: 100.0  
    Bloom filter false positives: 0  
    Bloom filter false ratio: 0.00000  
    Bloom filter space used: 234440  
    Bloom filter off heap memory used: 234408  
    Index summary off heap memory used: 70416  
    Compression metadata off heap memory used: 290720  
    Compacted partition minimum bytes: 51  
    Compacted partition maximum bytes: 43388628  
    Compacted partition mean bytes: 13867  
    Average live cells per slice (last five minutes): 1.0  
    Maximum live cells per slice (last five minutes): 1  
    Average tombstones per slice (last five minutes): 75.75956284153006  
    Maximum tombstones per slice (last five minutes): 372  
    Dropped Mutations: 0  

标签: cassandra

解决方案


推荐阅读