python - Scrapy 找不到 div.title

问题描述

import scrapy


class BookSpider(scrapy.Spider):
    name = "books"
    start_urls = [
        'http://books.toscrape.com/catalogue/page-1.html'
    ]

    def parse(self, response):
        page = response.url.split(".")[-1]
        filename = f'BooksHTML-{page}.html'
        with open(filename, 'wb') as f:
            f.write(response.body)
        self.log(f'Saved file {filename}')

所以我正在使用这个蜘蛛来练习网页抓取，我正在尝试收集这个页面上所有书籍的标题。当我进入终端并输入

刮壳'http://books.toscrape.com/catalogue/page-1.html'

接着

response.css("div.title").getall()

它只返回一个空列表。

[]

任何澄清将不胜感激。

标签： pythonweb-scrapingscrapy

就像蒂姆罗伯茨在评论中指出的那样，没有带有classof 的div title。

页面上每本书的完整标题位于标签（锚标签）的title属性中a，其中锚标签链接到该特定书籍的页面。

您可以获得具有以下title属性的所有锚标记的属性值title：

response.css("a::attr(title)").getall()

['A Light in the Attic', 'Tipping the Velvet', 'Soumission', 'Sharp Objects', 'Sapiens: A Brief History of Humankind', 'The Requiem Red', 'The Dirty Little Secrets of Getting Your Dream Job', 'The Coming Woman: A Novel Based on the Life of the Infamous Feminist, Victoria Woodhull', 'The Boys in the Boat: Nine Americans and Their Epic Quest for Gold at the 1936 Berlin Olympics', 'The Black Maria', 'Starving Hearts (Triangular Trade Trilogy, #1)', "Shakespeare's Sonnets", 'Set Me Free', "Scott Pilgrim's Precious Little Life (Scott Pilgrim #1)", 'Rip it Up and Start Again', 'Our Band Could Be Your Life: Scenes from the American Indie Underground, 1981-1991', 'Olio', 'Mesaerion: The Best Science Fiction Stories 1800-1849', 'Libertarianism for Beginners', "It's Only the Himalayas"]

python - Scrapy 找不到 div.title

问题描述

解决方案

推荐阅读