web-scraping - 为什么这显示“DNS 查找失败”？

Question

scrapers.here是我的代码。我正在使用scrapy基本蜘蛛模板，并且出现DNS查找失败错误。我的错误在哪里？

class TopmoviesSpider(scrapy.Spider):
    name = 'topmovies'
    allowed_domains = ['www.imdb.com']
    start_urls = ['https://https://www.imdb.com/chart/top/']



     def parse(self, response):
            movies = response.xpath("//td[@class='titleColumn']/a")
            for movie in movies:
                link = movie.xpath(".//@href").get()
                yield response.follow(url=link, callback=self.scrape_movie)


        def scrape_movie(self,response):

            rating = response.xpath("//span[@itemprop='ratingValue']/text()").get()
            for mov in response.xpath("//div[@class='title_wrapper']"):
                yield {
                    'title': mov.xpath(".//h1/text()").get(),
                    'year_of_release': mov.xpath(".//span/a/text()").get(),
                    'duration': mov.xpath(".//div[@class='subtext']/time/text()").get(),
                    'genre': mov.xpath(".//div[@class='subtext']/a/text()").get(),
                    'date_of_release': mov.xpath("//div[@class='subtext']/a[2]/text()"),
                    'rating': rating
                }

标签： web-scrapingscrapy

Answer 1

检查 start_urls。您提供的网址无效。如果您尝试爬取 imdb，请查看此帖子。