logging - Best way to find out which log statements are generating the most log volume?

Question

I'm hosting a backend service in AWS, and for some reason the service has been logging excessively for about a week now. I downloaded the logs of the service, and now I'm trying to find some pattern in those log files to see which log statement(s) are responsible for the excessive logging volume.

I did this before by basically cutting away the timestamps, sorting alphabetically and then counting how often each line occurs. That sometimes works, but of course as soon as there is a dynamic part in the log line this approach fails.

So ideally I'd like some kind of tool that can analyse my logs and fuzzy match log lines to determine which patterns occur the most. Does any such tool exist?

标签： logginganalytics