elasticsearch - 为桶弹性搜索聚合中的每个键设置阈值
问题描述
我的系统日志包含2个字段,ServiceName和ResponseCode(000-999),response保存服务的处理结果,000表示成功。我需要统计每个服务的成功和失败次数并计算成功率。
"aggs": {
"group_by_service": {
"terms": {
"field": "ServiceName.keyword"
},
"aggs": {
"group_by_count": {
"value_count": {
"field": "ServiceName.keyword"
}
},
"group_by_success": {
"filter": {
"terms": {
"ResponseCode": "000"
}
},
"aggs": {
"group_by_count_succ": {
"value_count": {
"field": "ServiceName.keyword"
}
}
}
},
"success_percent": {
"bucket_script": {
"buckets_path": {
"numbersucess": "group_by_success>group_by_count_succ",
"totalRequests": "group_by_count"
},
"script": "params.numbersucess / params.totalRequests * 100",
"format": "0.00"
}
}
}
}
}
返回的结果:
"aggregations": {
"group_by_service": {
"doc_count_error_upper_bound": 1859,
"sum_other_doc_count": 94338,
"buckets": [
{
"doc_count": 34361,
"success_percent": {
"value_as_string": "100.00",
"value": 100
},
"group_by_count": {
"value": 34361
},
"group_by_success": {
"doc_count": 34361,
"group_by_count_succ": {
"value": 34361
}
},
"key": "AAA"
},
{
"doc_count": 20474,
"success_percent": {
"value_as_string": "89.27",
"value": 89.26931718276839
},
"group_by_count": {
"value": 20474
},
"group_by_success": {
"doc_count": 18277,
"group_by_count_succ": {
"value": 18277
}
},
"key": "BBB"
},
我需要为桶中的每个键设置阈值:
如果 key = AAA,success_percent.value 必须大于 80
如果 key = BBB,success_percent.value 必须大于 90
...
不满足条件的key会从bucket中移除,这样当我发送alert email时,我只会收到有效key,我该怎么做呢?
解决方案
无法访问 bucket_Selector 聚合中的存储桶键。所以我们不能根据条款进行检查。这个问题应该在客户端解决或者数据应该用预先计算的success_percent索引
通过使用多个术语聚合,有一种肮脏的方法
"aggs": {
"group_by_service_A": {
"terms": {
"field": "ServiceName.keyword",
"include":"A" ---> aggregation for "A" only, can also be replaced by filter
},
"aggs": {
"group_by_count": {
"value_count": {
"field": "ServiceName.keyword"
}
},
"group_by_success": {
"filter": {
"term": {
"ResponseCode": "000"
}
},
"aggs": {
"group_by_count_succ": {
"value_count": {
"field": "ServiceName.keyword"
}
}
}
},
"success_percent": {
"bucket_script": {
"buckets_path": {
"numbersucess": "group_by_success>group_by_count_succ",
"totalRequests": "group_by_count"
},
"script": "params.numbersucess / params.totalRequests * 100",
"format": "0.00"
}
},
"filter_bucket": {
"bucket_selector": {
"buckets_path": {
"percent":"success_percent"
},
"script": "if(params.percent > 20) return true;" --> percent for "A"
}
}
}
},
"group_by_service_B": {
"terms": {
"field": "ServiceName.keyword",
"include":"B"---> aggregation for "B" only, can also be replaced by filter
},
"aggs": {
"group_by_count": {
"value_count": {
"field": "ServiceName.keyword"
}
},
"group_by_success": {
"filter": {
"term": {
"ResponseCode": "000"
}
},
"aggs": {
"group_by_count_succ": {
"value_count": {
"field": "ServiceName.keyword"
}
}
}
},
"success_percent": {
"bucket_script": {
"buckets_path": {
"numbersucess": "group_by_success>group_by_count_succ",
"totalRequests": "group_by_count"
},
"script": "params.numbersucess / params.totalRequests * 100",
"format": "0.00"
}
},
"filter_bucket": {
"bucket_selector": {
"buckets_path": {
"percent":"success_percent"
},
"script": "if(params.percent > 30) return true;" --> percent for "B"
}
}
}
}
}
推荐阅读
- javascript - 即使使用时区设置,两个日期之间的时间差也会返回错误
- google-sheets - 当行被移动并且上面的行发生变化时在上面添加列
- r - 如何在r中为回归线上方和下方的ggplot点着色?
- python - 尽管 Big-O 相同,为什么这两个函数的性能差异如此之大?
- arrays - How to sort list by sub list value in flutter?
- kubernetes-ingress - haproxy-ingress 多个后端规则
- google-apps-script - Google Apps 脚本“脚本属性”实际上是“项目属性”吗?正确的范围是多少?
- java - Web 应用程序监听服务器事件的最佳方式
- android - 了解android中的隐藏过程
- javascript - 反应导航 - 无法读取未定义的属性“推送”