时间戳

首页 > 解决方案 > Logstash 6.2.3 将 json 值转换为 GeoPoint 问题

json - Logstash 6.2.3 将 json 值转换为 GeoPoint 问题

问题描述

嗨，我是 ELK 的新手，所以首先我想设置使用 logstash 读取的自定义 json 文件。我想在 kibana 地图中显示客户位置。这是我要转换的json格式。

{"id":1,"first_name":"Freeman","last_name":"Jowers","email":"fjowers0@mashable.com","gender":"Male","ip_address":"15.128.77.162","latitude":9.9004655,"longitude":13.0544185,"date":"2017-10-29T17:47:59Z","country":"Nigeria"}

这是我用于 logstash 的配置文件。

    input {
  file{
    path => ["/home/sajith/Desktop/scripts/logstash-data/sample-data-006.json"]
    type => "json"
    start_position => "beginning"
  }
}
filter {
  grok {
    match => ['message','(?<body>\"id\":.*\"country\":\"[^"]+\")']
    add_field => ["json_body","{%{body}}"]
  }
  json {
    source => "json_body"
    remove_field => ["message","body","json_body"]
  }
  mutate{
    add_field => ["[geoip][location]","%{[latitude]}"]
    add_field => ["[geoip][location]","%{[longitude]}"]
  }
  mutate{
    convert => ["[geoip][location]","float"]
  }
}
output {
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "my-mock"
  }
}

问题出在显示为 number 的 kibana geoip.location 类型中。我需要将 geoip.location 显示为 geo_point。 在此处输入图像描述

谁能给我如何解决这个问题。我正在使用 ELK 6.2.3

标签： jsonelasticsearchlogstashgeopoints

解决方案

有几个问题。首先，您需要使用以下映射设置正确的索引映射：

PUT my-index
{
  "mappings" : {
    "_default_" : {
      "dynamic_templates" : [ {
        "message_field" : {
          "path_match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text",
            "norms" : false
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text", "norms" : false,
            "fields" : {
              "keyword" : { "type": "keyword", "ignore_above": 256 }
            }
          }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date"},
        "@version": { "type": "keyword"},
        "geoip"  : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" }
          }
        }
      }
    }
  }
}

其次，您需要geoip像这样正确创建字段：

  mutate{
    add_field => ["[geoip][ip]","%{[ip_address]}"]
    add_field => ["[geoip][location][latitude]","%{[latitude]}"]
    add_field => ["[geoip][location][longitude]","%{[longitude]}"]
  }

最后，您需要删除最后一个mutate/convert过滤器。

推荐阅读