elasticsearch - 无法使用 docker logstash 建立索引
问题描述
我正在使用最新的存储库代码来使用命令git@github.com:deviantony/docker-elk.git
托管 ELK 堆栈。docker-compose up
弹性搜索和 kibana 运行良好。
虽然我无法使用我的 logstash.conf 索引到 logstash,如下所示:
input {
file {
# Configure your path below
path => ["C:/Users/matt/Desktop/temp/logs/*.txt*"]
ignore_older => "141 days"
start_position => "beginning"
file_sort_by => "last_modified"
file_sort_direction => "desc"
sincedb_path => "NUL"
type => "appl"
codec => multiline {
pattern => "^<log4j:event"
negate => true
what => "previous"
}
}
}
filter {
if [type] == "appl" {
grok {
add_tag => [ "groked" ]
match => ["message", ".*"]
remove_tag => ["_grokparsefailure"]
}
mutate {
gsub => ["message", "log4j:", ""]
}
xml {
source => "message"
remove_namespaces => true
target => "log4jevent"
xpath => [ "//event/@timestamp", "timestamp" ]
xpath => [ "//event/@level", "loglevel" ]
xpath => [ "/event/message/text()", "message" ]
xpath => [ "/event/throwable/text()", "exception" ]
xpath => [ "//event/properties/data[@name='log4jmachinename']/@value", "machinename" ]
xpath => [ "//event/properties/data[@name='log4japp']/@value", "app" ]
xpath => [ "//event/properties/data[@name='log4net:UserName']/@value", "username" ]
xpath => [ "//event/properties/data[@name='log4net:Identity']/@value", "identity" ]
xpath => [ "//event/properties/data[@name='log4net:HostName']/@value", "hostname" ]
}
mutate {
remove_field => ["type"]
gsub => [
"message", "&", "&",
"message", "<", "<",
"message", ">", ">",
"message", """, "\"",
"message", "'", "'"
]
}
date {
match => [ "[timestamp][0]","UNIX_MS" ]
target => "@timestamp"
remove_field => ["timestamp"]
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "log4jevents"
user => "elastic"
password => "changeme"
ecs_compatibility => disabled
}
stdout {
codec => rubydebug
}
}
我想用我的logstash索引的日志文件如下所示
<log4j:event logger="Microsoft.Unity.ApplicationBlocks.Logging.Logger" timestamp="1615025506621" level="DEBUG" thread="13"><log4j:message>SSO->AccountController->Login->Before ClientID Check</log4j:message><log4j:properties><log4j:data name="log4jmachinename" value="hostname01" /><log4j:data name="log4japp" value="/LM/W3SVC/2/ROOT-1-132594985694777790" /><log4j:data name="log4net:UserName" value="IIS APPPOOL\default" /><log4j:data name="log4net:Identity" value="" /><log4j:data name="log4net:HostName" value="hostname01" /></log4j:properties><log4j:locationInfo class="Microsoft.Unity.ApplicationBlocks.Logging.Logger" method="Debug" file="F:\somefolder\Agent\_work\1\s\Unity\Microsoft.Unity.ApplicationBlocks\Logging\Logging.cs" line="353" /></log4j:event>
启动时显示的问题如下docker-compose up
所示为logstash
Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
相同的 logstash.conf 在早期的 EK 版本 6.8 中工作。我的 logstash.conf 有什么问题?
解决方案
在您的输出elasticsearch
插件中,将hosts
属性设置为elasticsearch:9200
.
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "log4jevents"
user => "elastic"
password => "changeme"
ecs_compatibility => disabled
}
stdout {
codec => rubydebug
}
}
推荐阅读
- python - 在python中使用selenium以这种形式提取值
- python - 如何在 tkinter 中创建一个保持纵横比的可调整大小的框架?
- tensorflow - 如何在 keras 中实现类似 Alexnet 的数据增强
- firebase - Firebase 通配符重定向到变量
- python-3.x - jinja flask通过python方法渲染侧边栏菜单
- java - 使用扫描仪时如何检测空白点并用预定答案填写?(爪哇)
- uinavigationcontroller - 未生成 SafeArgs
- jquery - 成功的 AJAX 请求后 jQuery 数据表不绘制
- r - R:将嵌套的for循环转换为应用函数
- c# - 消费(超时)超时 > max.poll.interval.ms 不触发重新平衡