python - ProcessPoolExecutor 输出

Question

我有以下代码，我试图根据给定的 grok 模式解析日志行。我正在使用 concurrent.futures.ProcessPoolExecutor 进行并行调用。该程序执行良好，但输出未按预期打印。

这是我的代码：

from concurrent.futures import ProcessPoolExecutor, as_completed
from korg import LineGrokker, PatternRepo

pr = PatternRepo()
patterns = [
    '%{COMBINEDAPACHELOG}',
    '%{COMMONAPACHELOG}',
    '%{HTTPD_ERRORLOG}'
]

text = [
    '37.162.60.195 - - [06/Jun/2018:17:31:29 -0400] "PUT /app/main/posts HTTP/1.0" 200 5055 "http://harris-johnson.com/main/register/" "Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5311 (KHTML, like Gecko) Chrome/13.0.850.0 Safari/5311"',
    '::1 - - [26/Dec/2016:16:16:29 +0200] "GET /favicon.ico HTTP/1.1" 404 209',
    '[Mon Dec 26 16:22:08 2016] [error] [client 192.168.33.1] File does not exist: /var/www/favicon.ico'
]

pool = ProcessPoolExecutor(len(patterns))

def grok_parser(pattern, text):
    return LineGrokker(pattern, pr).grok(text)

def parse_type(ia):
    futures = [pool.submit(grok_parser, pattern, ia) for pattern in patterns]
    for r in as_completed(futures):
        return r.result()

for i in text:
    print(parse_type(i))

输出：

{'clientip': '37.162.60.195', 'ident': '-', 'auth': '-', 'timestamp': '06/Jun/2018:17:31:29 -0400', 'verb': 'PUT', 'request': '/app/main/posts', 'httpversion': '1.0', 'rawrequest': None, 'response': '200', 'bytes': '5055'}
None
None

预期输出：

{'clientip': '37.162.60.195', 'ident': '-', 'auth': '-', 'timestamp': '06/Jun/2018:17:31:29 -0400', 'verb': 'PUT', 'request': '/app/main/posts', 'httpversion': '1.0', 'rawrequest': None, 'response': '200', 'bytes': '5055', 'referrer': '"http://harris-johnson.com/main/register/"', 'agent': '"Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5311 (KHTML, like Gecko) Chrome/13.0.850.0 Safari/5311"'}
{'clientip': '::1', 'ident': '-', 'auth': '-', 'timestamp': '26/Dec/2016:16:16:29 +0200', 'verb': 'GET', 'request': '/favicon.ico', 'httpversion': '1.1', 'rawrequest': None, 'response': '404', 'bytes': '209'}
{'timestamp': 'Mon Dec 26 16:22:08 2016', 'loglevel': 'error', 'clientip': '192.168.33.1', 'message': 'File does not exist: /var/www/favicon.ico', 'module': None, 'pid': None, 'tid': None, 'proxy_errorcode': None, 'proxy_message': None, 'clientport': None, 'errorcode': None}

我如何从上面的代码中得到想要的？任何帮助表示赞赏。

标签： pythonpython-3.xconcurrency