时间戳

首页 > 解决方案 > 如何在网页抓取期间将其他参数传递给scrapy中的蜘蛛解析函数

parsing - 如何在网页抓取期间将其他参数传递给scrapy中的蜘蛛解析函数

问题描述

我正在尝试将其他信息传递给 parse 函数，但它给出了类型错误。

TypeError: parse() got an unexpected keyword argument 'body' 我无法解决这个问题。

    """
    return [scrapy.Request(url=website.search_url.format(prod), callback=self.parse,
                           cb_kwargs = {"body":website.body_xpath,"product_list":website.products_list_xpath,
                                 "names":website.products_name_xpath,"selling_price":website.selling_price_xpath,
                                 "market_price":website.market_price_xpath}) for website in websites for prod in modified_products]

    def parse(self, response):
            body = response.cb_kwargs.get("body")
            product_list = response.cb_kwargs.get("product_list")
            name = response.cb_kwargs.get("names")
            selling_price = response.cb_kwargs.get("selling_price")
            market_price = response.cb_kwargs.get("market_price")

    """

标签： parsingscrapyparameter-passing

解决方案

我忘了在解析函数定义中写这些名字，添加它们后我得到了正确的结果。感谢您查看它。

    """
    return [scrapy.Request(url=website.search_url.format(prod), callback=self.parse,
                           cb_kwargs = dict(body = website.body_xpath, product_list  = website.products_list_xpath,
                                  name  = website.products_name_xpath, selling_price = website.selling_price_xpath,
                                  market_price  = website.market_price_xpath)) for website in websites for prod in modified_products]

    def parse(self, response, body, product_list, name, selling_price, market_price):
            body = response.cb_kwargs["body"]
            product_list = response.cb_kwargs["product_list"]
            name_ = response.cb_kwargs["name"]
            selling_price_ = response.cb_kwargs["selling_price"]
            market_price_ = response.cb_kwargs["market_price"]
    """

推荐阅读