java - 如何通过 Elasticsearch 中的嵌套字段计算唯一文档的数量?
问题描述
我正在尝试计算具有唯一嵌套字段值的文档(接下来也是文档本身)。看起来获得独特的文件是可行的。但是,当我尝试执行对 的请求时count
,出现如下错误:
抑制:org.elasticsearch.client.ResponseException:方法 [POST],主机 [ http://localhost:9200],URI [/package/_count?ignore_throttled=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true],状态行 [HTTP/1.1 400 错误请求] {"error":{"root_cause":[{"type":"parsing_exception","reason":"请求不支持 [collapse]","line":1,"col":216} ],"type":"parsing_exception","reason":"request 不支持 [collapse]","line":1,"col":216},"status":400}
编码:
BoolQueryBuilder innerTemplNestedBuilder = QueryBuilders.boolQuery();
NestedQueryBuilder templatesNestedQuery = QueryBuilders.nestedQuery("attachment", innerTemplNestedBuilder, ScoreMode.None);
BoolQueryBuilder mainQueryBuilder = QueryBuilders.boolQuery().must(templatesNestedQuery);
if (!isEmpty(templateName)) {
innerTemplNestedBuilder.filter(QueryBuilders.termQuery("attachment.name", templateName));
}
SearchSourceBuilder searchSourceBuilder = SearchSourceBuilder.searchSource()
.collapse(new CollapseBuilder("attachment.uuid"))
.query(mainQueryBuilder);
// NEXT LINE CAUSE ERROR
long count = client.count(new CountRequest("package").source(searchSourceBuilder), RequestOptions.DEFAULT).getCount(); <<<<<<<<<< ERROR HERE
// THIS WORKS
SearchResponse searchResponse = client.search(
new SearchRequest(
new String[] {"package"},
searchSourceBuilder.timeout(new TimeValue(20, TimeUnit.SECONDS)).from(offset).size(limit)
).indices("package").searchType(SearchType.DFS_QUERY_THEN_FETCH),
RequestOptions.DEFAULT
);
return ....;
方法的总体意图是获取部分文档和所有此类文档的数量。可能已经存在另一种方法来满足这种需求。如果我正在尝试count
使用aggregations
并且cardinality
- 我得到的结果为零,并且它看起来不适用于嵌套字段。
计数请求:
{
"query": {
"bool": {
"must": [
{
"nested": {
"query": {
"bool": {
"adjust_pure_negative": true,
"boost": 1.0
}
},
"path": "attachment",
"ignore_unmapped": false,
"score_mode": "none",
"boost": 1.0
}
}
],
"adjust_pure_negative": true,
"boost": 1.0
}
},
"collapse": {
"field": "attachment.uuid"
}
}
映射是如何创建的:
curl -X DELETE "localhost:9200/package?pretty"
curl -X PUT "localhost:9200/package?include_type_name=true&pretty" -H 'Content-Type: application/json' -d '{
"settings" : {
"number_of_shards" : 1,
"number_of_replicas" : 1
}}'
curl -X PUT "localhost:9200/package/_mappings?pretty" -H 'Content-Type: application/json' -d'
{
"dynamic": false,
"properties" : {
"attachment": {
"type": "nested",
"properties": {
"uuid" : { "type" : "keyword" },
"name" : { "type" : "text" }
}
},
"uuid" : {
"type" : "keyword"
}
}
}
'
代码生成的结果查询应该是这样的:
curl -X POST "localhost:9200/package/_count?&pretty" -H 'Content-Type: application/json' -d' { "query" :
{
"bool": {
"must": [
{
"nested": {
"query": {
"bool": {
"adjust_pure_negative": true,
"boost": 1.0
}
},
"path": "attachment",
"ignore_unmapped": false,
"score_mode": "none",
"boost": 1.0
}
}
],
"adjust_pure_negative": true,
"boost": 1.0
}
},
"collapse": {
"field": "attachment.uuid"
}
}'
解决方案
折叠只能在_search
上下文中使用,不能在_count
.
其次,您的查询甚至可以做什么?你有很多冗余参数,比如boost:1
等等。你不妨说:
POST /package/_count?&pretty
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "attachment",
"query": {
"match_all": {}
}
}
}
]
}
}
}
这并没有真正做任何事情:)
要回答“计算具有唯一嵌套字段值的文档”的原始问题,
让我们想象 3 个文档,其中 2 个具有相同的attachment.uuid
值:
[
{
"attachment":{
"uuid":"04144e14-62c3-11ea-bc55-0242ac130003"
}
},
{
"attachment":{
"uuid":"04144e14-62c3-11ea-bc55-0242ac130003"
}
},
{
"attachment":{
"uuid":"100b9632-62c3-11ea-bc55-0242ac130003"
}
}
]
要获得 s 的terms
细分uuid
,请运行
GET package/_search
{
"size": 0,
"aggs": {
"nested_uniques": {
"nested": {
"path": "attachment"
},
"aggs": {
"subagg": {
"terms": {
"field": "attachment.uuid"
}
}
}
}
}
}
产生
...
{
"aggregations":{
"nested_uniques":{
"doc_count":3,
"subagg":{
"doc_count_error_upper_bound":0,
"sum_other_doc_count":0,
"buckets":[
{
"key":"04144e14-62c3-11ea-bc55-0242ac130003",
"doc_count":2
},
{
"key":"100b9632-62c3-11ea-bc55-0242ac130003",
"doc_count":1
}
]
}
}
}
}
为了获得唯一嵌套字段的父文档计数,我们将不得不变得更加聪明:
GET package/_search
{
"size": 0,
"aggs": {
"nested_uniques": {
"nested": {
"path": "attachment"
},
"aggs": {
"scripted_uniques": {
"scripted_metric": {
"init_script": "state.my_map = [:];",
"map_script": """
if (doc.containsKey('attachment.uuid')) {
state.my_map[doc['attachment.uuid'].value.toString()] = 1;
}
""",
"combine_script": """
def sum = 0;
for (c in state.my_map.entrySet()) {
sum += 1
}
return sum
""",
"reduce_script": """
def sum = 0;
for (agg in states) {
sum += agg;
}
return sum;
"""
}
}
}
}
}
}
返回
...
{
"aggregations":{
"nested_uniques":{
"doc_count":3,
"scripted_uniques":{
"value":2
}
}
}
}
这scripted_uniques: 2
正是你所追求的。
注意:我使用嵌套脚本度量 aggs 解决了这个用例,但如果你们中的任何人知道更简洁的方法,我很乐意学习它!
推荐阅读
- python-3.x - 在 matplotlib.text 中填充文本
- swift - 如何为按钮添加遮罩以使隐藏区域不可点击?
- postgresql - 我正在尝试使用 spark 从 postgres 中删除数据,但无法删除相同的代码适用于 select 语句
- microsoft-graph-api - Microsoft Graph - 如何将邮件移动到公用文件夹
- r - R中的SQL查询使用基于数据框中值的多个AND语句
- javascript - Javascript:使用单个属性删除对象数组中的重复项
- linux - 如何在centos 7服务器中的每个用户(域)下的每个.htaccess文件中插入文本?
- android - RecyclerView 未按预期显示数据库内容
- postgresql - 在 postgresql 中用于 mssql 中的 Trancount 的替代方法是什么
- javascript - 无法解析axios错误