首页 > 解决方案 > 导入 settings.py Scrapy

问题描述

我有一个 testmultiple 文件夹,其中包含一个__init__.py文件、管道、设置和一个core.py文件,我用它来启动位于子文件夹(蜘蛛)中的几个蜘蛛。我注意到我必须导入设置才能将管道与CrawlerProcess. 这是我的代码:

import scrapy
from scrapy.crawler import CrawlerProcess
from scrapy.settings import Settings
import settings as my_settings
from spiders.DemoSpider import DemoSpider
from spiders.DemoSpider2 import DemoSpider2


crawler_settings = Settings()
crawler_settings.setmodule(my_settings)
process = CrawlerProcess(settings=crawler_settings)

process.crawl(DemoSpider)
process.crawl(DemoSpider2)
process.start() # the script will block here until the crawling is finished

但它在第 4 行失败。通过这次尝试,我有:

ModuleNotFoundError: No module named 'testmultiple'

当我尝试:

from testmultiple.settings import settings as my_settings

我有同样的错误,也有这一行:

from testmultiple import settings as my_settings

如何导入settings.py

标签: pythonscrapy

解决方案


推荐阅读