elasticsearch - 在 elasticSearch 中创建管道的问题

Question

我正在尝试摄取一个包含 grok、date 和 remove 处理器的管道，但是尽管在文档下明确提到了“message”字段，但我收到了一个丢失的字段错误

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              "field" : "message", 
                "pattern" : "%{COMMONAPACHELOG}"
            }
          },
          {
            "date" : {
              "match_field" : "timestamp", 
              "match_formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
              
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 -- [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }

我得到这个错误请帮忙

{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "[patterns] required property is missing",
        "property_name" : "patterns",
        "processor_type" : "grok",
        "suppressed" : [
          {
            "type" : "parse_exception",
            "reason" : "[field] required property is missing",
            "property_name" : "field",
            "processor_type" : "date"
          }
        ]
      }
    ],
    "type" : "parse_exception",
    "reason" : "[patterns] required property is missing",
    "property_name" : "patterns",
    "processor_type" : "grok",
    "suppressed" : [
      {
        "type" : "parse_exception",
        "reason" : "[field] required property is missing",
        "property_name" : "field",
        "processor_type" : "date"
      }
    ]
  },
  "status" : 400
}

我试图在 youtube 上寻找一个视频，我发现有人使用相同的代码并且它执行得很好这是视频 https://www.youtube.com/watch?v=PEHnBa19Gxs&t=1s 它在第 34 分钟

标签： elasticsearch

Answer 1

事实证明，它适用于 youtube 的人，因为它使用的是旧版本。这将适用于较新的版本

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              
              "field" : "message", 
              "patterns" : ["%{COMMONAPACHELOG}"]
            }
          },
          {
            "date" : {
              "field" : "timestamp", 
              "formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 - - [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }