c# - 在峰值负载期间，并非所有日志都在使用 Serilog 的 AWS Elasticsearch Service 中

Question

技术栈：AWS ECS (.net core) + Serilog + Serilog.Sinks.Elasticsearch + AWS Elasticsearch

我注意到有时我在 ES 中找不到一些日志。

我做了一些测试来重现这样的问题，结果如下：

| Raised log entries (in parallel)  | Found log enries in ES    | 
-----------------------------------------------------------------
| 100000                            | 100000                    |
| 500000                            | 172244                    |
| 500000                            | 163841                    |
| 500000                            | 174947                    |
| 500000                            | 155788                    |

所以你可以看到它是非常可重复的。

我为 Serilog 打开了https://github.com/serilog/serilog/wiki/Debugging-and-Diagnostics但它什么也没说。CloudWatch 还说一切都很好。

Serilog的原始配置：

builder.UseSerilog(
    (hostingContext, loggerConfiguration) => loggerConfiguration
        .ReadFrom.Configuration(hostingContext.Configuration)
        .Enrich.FromLogContext()
        .Enrich.WithServiceName(serviceName)
        .WriteTo.Console(new JsonFormatter(renderMessage: true))
        .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(hostingContext.Configuration["elasticsearch"] ?? "http://localhost:9200"))
        {
            AutoRegisterTemplate = true,
            IndexDecider = (ev, offset) => string.Format("{0}.{1}", index, System.DateTime.Now.ToString("yyyy.MM.dd")),
        })
        .Destructure.UsingAttributes()
        );

也许有人已经遇到过这样的问题，所以如果您分享您的经验，我将不胜感激

更新

我从 AppInsights 中找到了下一个日志

System.Net.Http.HttpRequestException：在 System.Net.Http.ConnectHelper+d__2.MoveNext（System.Net.Http，版本=4.2.1.0，Culture=neutral，PublicKeyToken=b03f5f7f11d50a3a）在 System.Runtime.ExceptionServices.ExceptionDispatchInfo。在 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken) 抛出 (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) =7cec85d7bea7798e) 在 System.Threading.Tasks.ValueTask1.get_Result (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Net.Http.HttpConnectionPool+<CreateConnectionAsync>d__40.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Threading.Tasks.ValueTask1.get_Result (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.HttpConnectionPool+d__45.MoveNext (System.Net.Http, Version=4.2.1.0, Culture =中性，PublicKeyToken=b03f5f7f11d50a3a) 在 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib，版本=4.0.0.0，Culture=neutral，PublicKeyToken=7cec85d7bea7798e) 在 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System .Private.CoreLib，版本=4.0.0.0，文化=中性，PublicKeyToken=7cec85d7bea7798e) 在 System.Threading.Tasks.ValueTask1.get_Result (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Net.Http.HttpConnectionPool+<SendWithRetryAsync>d__37.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1+ConfiguredTaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.RedirectHandler+d__4.MoveNext (System.Net.Http, Version=4.2.1.0 , Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 在 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification （System.Private.CoreLib，版本=4.0.0.0，文化=中性，PublicKeyToken=7cec85d7bea7798e）在 System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult（System.Private.CoreLib，版本=4.0.0.0，文化=中性，PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.DiagnosticsHandler+d__2。MoveNext（System.Net.Http，版本=4.2.1.0，Culture=neutral，PublicKeyToken=b03f5f7f11d50a3a）在 System.Net.Http.ConnectHelper+d__2.MoveNext 处理的内部异常 System.Net.Sockets.SocketException：在 System.Runtime。 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture =中性，PublicKeyToken=7cec85d7bea7798e）在 System.Net.Http.ConnectHelper+d__2.MoveNext （System.Net.Http，版本=4.2.1.0，文化=中性，PublicKeyToken=b03f5f7f11d50a3a）Net.Http.ConnectHelper+d__2.MoveNext：在 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw（System.Private.CoreLib，Version=4.0.0.0，Culture=neutral，PublicKeyToken=7cec85d7bea7798e）在 System.Runtime.CompilerServices.TaskAwaiter。 HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.ConnectHelper+d__2.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral , PublicKeyToken=b03f5f7f11d50a3a)Net.Http.ConnectHelper+d__2.MoveNext：在 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw（System.Private.CoreLib，Version=4.0.0.0，Culture=neutral，PublicKeyToken=7cec85d7bea7798e）在 System.Runtime.CompilerServices.TaskAwaiter。 HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.ConnectHelper+d__2.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral , PublicKeyToken=b03f5f7f11d50a3a)Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.ConnectHelper+d__2.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 在 System.Net.Http.ConnectHelper+d__2.MoveNext (System.Net.Http, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

在http://elasticsearch:9200/_bulk操作期间

标签： c#amazon-web-serviceselasticsearch.net-coreserilog