python - 我正在尝试从网站上抓取,但它给了我一个错误
问题描述
File "/usr/local/bin/scrapy", line 10, in <module>
sys.exit(execute())
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 149, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 251, in __init__
super(CrawlerProcess, self).__init__(settings)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 137, in __init__
self.spider_loader = _get_spider_loader(settings)
File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 338, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "/usr/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 61, in from_settings
return cls(settings)
File "/usr/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 25, in __init__
self._load_all_spiders()
File "/usr/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
for module in walk_modules(name):
File "/usr/local/lib/python2.7/site-packages/scrapy/utils/misc.py", line 71, in walk_modules
submod = import_module(fullpath)
File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/sid/scrapytutorial/quotetutorial/quotetutorial/spiders/quotes_spider.py", line 28, in <module>
cd
NameError: name 'cd' is not defined
我无法弄清楚为什么会出现此错误。这与文件结构有关还是某种内部错误。我必须制作一个不同的文件夹然后再试一次吗?
这是我到目前为止所做的:
quote_spider.py 文件
import scrapy
from ..items import QuotetutorialItem
class QuoteScrapy(scrapy.Spider):
name = 'quotes'
start_urls = [
'http://quotes.toscrape.com/'
]
def parse(self, response):
items = QuotetutorialItem()
all_div_quotes = response.css('div.quote')
for quotes in all_div_quotes:
title = quotes.css('span.text::text').extract()
author = quotes.css('.author::text').extract()
tag = quotes.css('.tag::text').extract()
items['title'] = title
items['author'] = author
items['tag'] = tag
yield items
项目.py
import scrapy
class QuotetutorialItem(scrapy.Item):
title = scrapy.Field()
author = scrapy.Field()
tag = scrapy.Field()
请调查一下并告诉我我做错了什么
解决方案
推荐阅读
- npm - 通过私有存储库和 GitLab Runner Pipeline 安装 aws-cdk
- visual-studio-code - 开发与特定代码相邻显示信息(“查看问题”)的 VS Code 扩展
- xpath - 如何为表中存在的相同对象获取 xpath
- apache-spark - 如何将 Spark 配置参数传递给 DBT?
- c++ - 为什么当 std::fstream 都移动一个指针时存在 seekp/seekg?
- java - CaptainCasa WebSockets
- java - 将 hmac 函数从 python 转换为 Java
- window - 为什么我的 neovim 弹窗浮动窗口边框线断了?
- c++ - 如何在没有指针的情况下插入类对象以正确映射?
- c++ - 如何隐藏dll对JUMP的描述