elasticsearch - 如何更新映射
问题描述
如何更新索引映射以包含以下字段doc_as_upsert : true
我从 s3 摄取 cloudtrail 日志的 logstash 在日志上显示以下内容
Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"cloudtrail-2018.10.08", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x251f932>], :response=>{"index"=>{"_index"=>"cloudtrail-2018.10.08", "_type"=>"doc", "_id"=>"t2mmVWYBVQr-RbWuAQIS", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [requestParameters.disableApiTermination]", "caused_by"=>{"type"=>"json_parse_exception", "reason"=>"Current token (START_OBJECT) not of boolean type\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@133a6c; line: 1, column: 1509]"}}}}}
映射是动态的,而且很长,所以不能全部放在这里,但这是我喜欢的
{
"cloudtrail-2018.10.08": {
"mappings": {
"_default_": {
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "keyword"
},
"geoip": {
"dynamic": "true",
"properties": {
"ip": {
"type": "ip"
},
"latitude": {
"type": "half_float"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "half_float"
}
}
}
}
},
"doc": {
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "keyword"
},
"additionalEventData": {
"properties": {
"configRuleArn": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"configRuleInputParameters": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"configRuleName": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"managedRuleIdentifier": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"notificationJobType": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"vpcEndpointId": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"apiVersion": {
"type": "date"
},
"awsRegion": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"errorCode": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"errorMessage": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"eventID": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"eventName": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"eventSource": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"eventType": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"eventVersion": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
当我尝试使用这些更新映射时,出现以下错误
PUT cloudtrail-*/_mapping/_doc
{
"properties": {
"doc_as_upsert": true
}
}
我收到错误
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Expected map for property [fields] on field [doc_as_upsert] but got a class java.lang.String"
}
],
"type": "mapper_parsing_exception",
"reason": "Expected map for property [fields] on field [doc_as_upsert] but got a class java.lang.String"
},
"status": 400
}
解决方案
doc_as_upsert 是一个标志,用于告诉 elasticsearch 您想要使用 doc 的内容作为 upsert 值来更新文档。它与更新索引映射无关。
假设您要更新 id 1 和索引 test 的文档(更新名称)。
POST test/_doc/1/_update
{
"doc" : {
"name" : "new_name"
},
"doc_as_upsert" : true
}
推荐阅读
- python - 模式匹配并在字符串出现中提取它
- javascript - 我将如何检测何时在 jquery 中持有一个键?
- java - 简单的 Java 邮件 API:发送邮件队列但未收到任何电子邮件
- node.js - 检查 ejs 模板中的未定义变量
- python - python包初始化一个数据库到无处
- android - 默认 Firebase 应用程序未初始化
- amazon-s3 - 创建签名 URL - 生成的 URL 缺少签名
- android - 从 Android 控制第三方 Cast 会话?
- ios - 没有这样的模块'SocketIO'
- android - 如何将外部库依赖项添加到我的库中?