python - Scrapy - NameError:未定义全局名称'logger'
问题描述
我正在尝试通过修改中间件来稍微修改 Scrapy 重试。我使用这个中间件:
class Retry500Middleware(RetryMiddleware):
def _retry(self, request, reason, spider):
retries = request.meta.get('retry_times', 0) + 1
if retries <= self.max_retry_times:
logger.debug("Retrying %(request)s (failed %(retries)d times): %(reason)s",
{'request': request, 'retries': retries, 'reason': reason},
extra={'spider': spider})
retryreq = request.copy()
retryreq.meta['retry_times'] = retries
retryreq.meta['download_timeout'] = 600
retryreq.dont_filter = True
retryreq.priority = request.priority + self.priority_adjust
return retryreq
else:
logger.error("Gave up retrying %(request)s (failed %(retries)d times): %(reason)s",
{'request': request, 'retries': retries, 'reason': reason},
extra={'spider': spider})
然后我得到这个错误。
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
result = g.send(result)
File "/usr/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 53, in process_response
spider=spider)
File "/usr/lib/python2.7/site-packages/scrapy/downloadermiddlewares/retry.py", line 54, in process_response
return self._retry(request, reason, spider) or response
File "/home/<user_name>/<project_folder>/<project_name>/<project_name>/middlewares.py", line 48, in _retry
logger.debug("Retrying %(request)s (failed %(retries)d times): %(reason)s",
NameError: global name 'logger' is not defined
2018-08-15 14:01:44 [scrapy.core.engine] INFO: Closing spider (finished)
我在我的机器上使用了它,中间件工作得很好。我应该怎么做才能避免这个错误?
解决方案
最后,我改用这段代码
import logging
logging.log(logging.ERROR, "Gave up retrying %(request)s (failed %(retries)d times): %(reason)s",
{'request': request, 'retries': retries, 'reason': reason},
extra={'spider': spider})
推荐阅读
- flutter - Flutter:listview 和 expandTile 的类
- javascript - 是否可以在不使用 UI/WKWebView 的情况下加载和操作 HTML DOM?
- python - Python/Numpy:有没有一种有效的方法来计算多个窗口的移动平均值?
- java - 我的单元测试用例在 Spring Boot 中的 PUT api 控制器失败
- ruby-on-rails - Ruby on Rails(敏捷书示例)错误:Webpacker::Manifest::MissingEntryError in Store#index
- validation - NestJS 在 DTO 创建期间执行验证之前使用 ValidationPipe 转换属性
- angular - 当其他人调用相同的方法时取消 HTTP 请求
- python - 404 Django URL 错误,即使它包含在 url 模式中
- php - 使用 FOR EACH 将动态下拉列表作为一列的 PHP ECHO 表
- python - 设置一个变量以在python中的导入中使用