时间戳

首页 > 解决方案 > Scrapy使文本超出范围

python - Scrapy使文本超出范围

问题描述

网址：https ://myanimelist.net/anime/236/Es_Otherwise

我试图在 URL 中抓取以下内容：

在此处输入图像描述

我试过了 ：

for i in response.css('span[class = dark_text]') :
    i.xpath('/following-sibling::text()')

或者当前的 XPath 不起作用，或者我错过了一些东西......

aired_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[11]/text()')

producer_xpath = response.xpath("//*[@id='content']/table/tbody/tr/td[1]/div/div[12]/span/a/@href/text()")
licensor_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[13]/a/text()')
studio_xpath response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[14]/a/@href/title/text()')
studio_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[17]/text()')
str_rating_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[18]/text()')
ranked_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[20]/span/text()')
japanese_title_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[7]/text()')
source_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[15]/text()')
genre_xpath = [response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[16]/a[{0}]'.format(i)) for i in range(1,4)]
genre_xpath_v2 = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[16]/a/@href/text()')
number_of_users_rated_anime_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[19]/span[3]/text()')
popularity_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[21]/span/text()')
members_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[22]/span/text()')
favorite_xpath =  response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[23]/span/text()')

但我发现有些文本超出了跨度类，所以我想使用 css/XPath 公式使该文本超出跨度。

标签： pythonhtmlcssscrapy

解决方案

只循环遍历表格内的 div 更简单

foundH2 = False
response =  Selector(text=htmlString).xpath('//*[@id="content"]/table/tr/td[1]/div/*')

for resp in response:
  tagName = resp.xpath('name()').extract_first()
  if 'h2' == tagName:
    foundH2 = True
  if foundH2:
    # start adding 'info' after <h2>Alternative Titles</h2> found
    info = None
    if 'div' == tagName:
      for item in resp.xpath('.//text()').extract():
        if 'googletag.' in item: break
        item = item.strip()
        if item and item != ',':
          info = info + " " + item if info else item
      if info:
        print info

只是我的看法，beautifulSoup 比 scrapy 更快更好。

推荐阅读