python - Python Scrapy - 附加日志文件但以压缩格式

Question

我们启动了几个爬虫。如果 1 遇到错误，则会发送一封电子邮件。问题是日志有时太大（它们是平面文件）。

问题：我们如何以最简单的方式压缩和附加文件？

帮助表示赞赏

except Exception as e:
    from db_tool.settings import IS_DEV

    if not IS_DEV:
        from scrapy_app.settings import EMAIL_TO, LOG_DIR
        from django.template.loader import render_to_string
        from scrapy_app.utils import send_email
        import os

        logging.exception("exception occured while starting the spider. sending admins email")
        for user_email in EMAIL_TO:
            email_subject = f'sohb2bcrawler: General error encountered. Crawler did not finish '
            email_message = render_to_string('general_error.html')
            send_email(email_subject, email_message, email_to=user_email, files=[os.path.join(LOG_DIR, 'log_info.log'), os.path.join(LOG_DIR, 'log_error.log')])
    else:
        logging.exception("exception occured while starting the spider.")

标签： pythonscrapy