python-3.x - 如何使用 scrapy 从该 url 刮取多个表格行？

Question

import scrapy

class SsoSpider(scrapy.Spider):
    name = 'sso'
    allowed_domains = ['www.sso.agc.gov.sg']
    start_urls = ['https://sso.agc.gov.sg/Browse/Act/Current']

def parse(self, response):
    acts = response.xpath("//table[@class='table browse-list']/tbody")

    for act in acts:
        yield {
            #'Act title': act.xpath(".//tr[@class='alternate']/td/a/text()").get(),
            'Act title': act.xpath(".//tr/td/a/text()").get(),
            #'Short-hand code': act.xpath(".//tr[@class='alternate']/td/a/@href").get()
            'Short-hand code': act.xpath(".//tr/td/a/@href").get()
        }

所以这是我上面用于抓取的代码。运行后，我只得到 1 个抓取结果。

我认为问题与如何创建表行有关？就像有些人有课而有些人没有。

我是新手，所以任何帮助将不胜感激！

标签： python-3.xscrapy

Answer 1

试试这个，我希望它会工作。

import scrapy

class SsoSpider(scrapy.Spider):
    name = 'sso'
    allowed_domains = ['www.sso.agc.gov.sg']
    start_urls = ['https://sso.agc.gov.sg/Browse/Act/Current']

def parse(self, response):
    acts = response.xpath("//*[@id='listPanel']/table/tbody/tr")
    for act in acts:
        yield {
            'Act title': act.xpath("td[1]/a/text()").extract_first(),
            'Short-hand code': act.xpath("td/a/@href").extract_first()
        }

让我知道。