首页 > 解决方案 > 访问类范围python之外的变量

问题描述

我想在从页面上抓取它后更新__VIEWSTATE和值。__EVENTVALIDATION我正在浏览页面,并且需要在数据字典中引用一个唯一值__VIEWSTATE和值。__EVENTVALIDATION我想知道是否可以访问范围response之外的变量,class以便我可以将值发送到数据字典?

从输入导入计数器

import requests
from scrapy.selector import Selector


class Scraper:
    val = ''
    def bot(self, event_state='some-values', event_validate='some-valus'):
        cookies = {}
        headers = {}
        
        counter = 0
        while True:
            data = {
                '__EVENTTARGET': f'dgSearchResults$ctl24$ctl0{counter}',
                '__VIEWSTATE': self.event_state,
                '__VIEWSTATEGENERATOR': '11C1F95B',
                '__EVENTVALIDATION': self.event_validate,
                'cboType': 'RE',
                'DateOfFilingFrom': '01/01/2021',
                'DateOfFilingTo': '10/20/2021',
                'cboPartyType': 'Decedent',
                'cmdSearch': 'Search'
            }

            url = 'https://registers.maryland.gov/RowNetWeb/Estates/frmEstateSearch2.aspx'
            r = requests.post(url=url, headers=headers, cookies=cookies, data=data)
            
            response = Selector(text=r.text)
            
            links = response.xpath('//a[contains(@href, "RecordId")]')
            if links == []:
                break
            for link in links:
                ids = link.xpath('.//@href').get()
                print(ids)
            counter += 1


event_state_value = response.xpath('//input[@name="__VIEWSTATE"]/@value').get()
event_validate_value = response.xpath('//input[@name="__EVENTVALIDATION"]/@value').get()

scraper = Scraper()
scraper.bot(event_state_value, event_validate_value)

标签: pythonclassoopweb-scrapingpython-requests

解决方案


推荐阅读