时间戳

首页 > 解决方案 > 我正在尝试从 espn 中提取一些数据作为表格并将其作为列表

python - 我正在尝试从 espn 中提取一些数据作为表格并将其作为列表

问题描述

start_urls = http://www.espncricinfo.com/series/18679/scorecard/1144998/australia-vs-india-2nd-odi-india-in-aus-2018-19

我抓取了这个网站并提取了比赛结果（获胜球队），然后我产生了球员 URL，我想打印球员姓名和击球方式。我的第一个问题是 1. 我无法抽象出球员的击球方式。它在<pclass="ciPlayerinformationtxt"><b>Batting style</b> <span>Right-hand bat</span>. 我只能提取文本“击球风格”。如何提取“右手球棒” 2.我无法将提取的整个数据生成为表格。我得到的结果就像

所有播放器的 p 链接 http://www.espncricinfo.com/ci/content/player/326434.html

Player_name 国家 Alex Carey 澳大利亚
Kuldeep Yadav 印度
Mohammed Siraj India
Winning_Team:India

class ScoreSpider(scrapy.Spider):
    name = 'score'
    allowed_domains = ['espncricinfo.com']

    def parse(self, response):
        Player_URLs=[]

        #got the result

        result= response.xpath('//div[@class="cscore_notes"]/span/text()').extract_first()
        result=result.split(" ")
        Winning_Team =result[0]

        #extracted player ulrs

        Batting_Player_URLs=response.xpath('//div[@class="cell batsmen"]/a/@href').extract()
        Bowling_Player_URLs=response.xpath('//*[@class="scorecard-section bowling"]/table/tbody/tr/td/a/@href').extract()

        #added to a list

        Player_URLs.extend(Batting_Player_URLs)
        Player_URLs.extend(Bowling_Player_URLs)
        for p in Player_URLs:
            yield Request(p,callback=self.parse_players,meta={'p':p})
            yield{'Winning_Team':Winning_Team}

    def parse_players(self,response):
        Player_name=response.xpath('//div[@class="ciPlayernametxt"]/div/h1/text()').extract_first()
        Country=response.xpath('//div[@class="ciPlayernametxt"]/div/h3/b/text()').extract_first()

        #this wont give the batting style but the 'batting style' as text
        Batting_style=response.xpath('//div[@class="ciPlayerinformationtxt"]/p/text()').extract_first()

        yield{'Player_name':Player_name,
              'Country':Country, 
              'Batting_style':Batting_style}

我想要的是提取的数据作为一个表，我想避免重复。

          yield{'Winning_Team':Winning_Team,
                'Player_name':Player_name,
                'Country':Country,
                'Batting_style':Batting_style}

提前致谢

标签： pythonweb-scrapingscrapy

解决方案

您需要调整 XPath： 

batting_style = response.xpath('//p[@class="ciPlayerinformationtxt"]/b[.="Batting style"]/following-sibling::span[1]/text()').get()

更新

for p in Player_URLs:
    yield Request(p,callback=self.parse_players,meta={'Winning_Team':Winning_Team})

然后：

def parse_players(self,response):
    Winning_Team = response.meta["Winning_Team"]

推荐阅读