时间戳

首页 > 解决方案 > 将 csv 数据加载到 Elasticsearch 失败，翻译问题

java - 将 csv 数据加载到 Elasticsearch 失败，翻译问题

问题描述

我正在尝试将 CSV 文件导入弹性文件，但失败并引发错误

由于错误 {:pipeline_id=>"main", :exception=>#, :backtrace=>["/usr/local/Cellar/logstash/7.6.1/libexec/vendor/bundle/jruby/2.5.0，管道中止/gems/logstash-filter-mutate-3.5.0/lib/logstash/filters/mutate.rb:222:in block in register'", "org/jruby/RubyHash.java:1428:ineach'", "/usr/local/Cellar/logstash/7.6.1/libexec/vendor/bundle/ jruby/2.5.0/gems/logstash-filter-mutate-3.5.0/lib/logstash/filters/mutate.rb:220:in register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in register'", "/usr/local/Cellar/logstash/7.6.1/libexec /logstash-core/lib/logstash/java_pipeline.rb:200:in block in register_plugins'", "org/jruby/RubyArray.java:1814:in each'", "/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:199:在 register_plugins'", "/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:502:in Maybe_setup_out_plugins'", "/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:212:in start_workers'", "/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:154:in 运行'", "/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:109:in `block in start'"], "pipeline.sources"=> ["/Users/user/Document/Esk-Data/xudaxia.conf"], :thread=>"#"}

下面是conf文件

input
{
    file{
    path => ["/test.csv"]
    start_position => "beginning"
    }
}
filter{
    csv{
        separator => ","
         columns => ["comment_time","comment", "id", "video_time"]
       }
    mutate{
        convert => {
            "comment_time" => "date_time"
            "comment" => "string"
            "id" => "integer"
            "video_time" => "float"
      }
    }
}
output{
    elasticsearch{
        hosts => ["localhost:9200"]
        index => "test"
    }
}

测试.csv

comment_time         comment        id        video_time
2020/03/22 15:59:41  バイ             a        123.100
2020/03/22 15:59:45  บาย             b        100.100
2020/04/22 15:59:50  ByeBye          c        80.210

谁能帮忙？

标签： javarubycsvelasticsearchlogstash

解决方案

根据文档，插件-doc heredate_time上的转换操作选项不存在。但是，此插件用于将类型转换为另一种类型，这不是您的用例。如果不被识别为日期字段，则应在此处使用plugin-doc对其进行转换。mutatecomment_timedate

所以你应该删除这个块：

mutate{
    convert => {
        "comment_time" => "date_time"
        "comment" => "string"
        "id" => "integer"
        "video_time" => "float"
  }
}

并用这个替换它：

date {
    match => [ "comment_time", "yyyy/MM/dd HH:mm:ss"
}

推荐阅读