时间戳

首页 > 解决方案 > Beautifulsoup 迭代以获取 sometext 或 url

python - Beautifulsoup 迭代以获取 sometext 或 url

问题描述

我想创建一个包含键值对的列表。以<thead>物品为重点。对于我想要获取除存在 a<th>的项目之外的所有项目的文本的值，然后我想获取 url。<th><a href='url'>

目前我只能从所有项目中获取文本。但是我该怎么做才能得到'/someurl'而不是Makulerad和Detaljer？

<table class="table table-bordered table-hover table-striped zero-margin-top">
  <thead>
    <tr>
      <th>Volymsenhet</th>
      <th>Pris</th>
      <th>Valuta</th>
      <th>Handelsplats</th>
      <th>url1</th>
      <th>url2</th>
    </tr>
  </thead>
  <tbody>
      
     <tr  class="iprinactive">
        <td>Antal</td>
        <td>5,40</td>
        <td>SEK</td>
        <td>NASDAQ STOCKHOLM AB</td>
        <td><a href="/someurl">Makulerad</a></td>
        <td>
          <a href="/someurl">Detaljer</a>
        </td>
      </tr>
    
    </tbody>
  </table>

我的代码：

raw_html = simple_get('https://example.com/')
soup = BeautifulSoup(raw_html, 'html.parser')

table = soup.find("table", attrs={"class":"table"})
head = [th.get_text() for th in table.find("tr").find_all("th")]

datasets = []
for row in table.find_all("tr")[1:]:
    dataset = dict(zip(head,(td.get_text() for td in row.find_all("td"))))
    datasets.append(dataset)

标签： pythonbeautifulsoup

解决方案

尝试这个：

只需获取文本数据，<td>如果它没有<a>. 否则获取href值。

from bs4 import BeautifulSoup
raw_html = '''<table class="table table-bordered table-hover table-striped zero-margin-top">
  <thead>
    <tr>
      <th>Volymsenhet</th>
      <th>Pris</th>
      <th>Valuta</th>
      <th>Handelsplats</th>
      <th>url1</th>
      <th>url2</th>
    </tr>
  </thead>
  <tbody>          
     <tr class="iprinactive">
        <td>Antal</td>
        <td>5,40</td>
        <td>SEK</td>
        <td>NASDAQ STOCKHOLM AB</td>
        <td><a href="/someurl">Makulerad</a></td>
        <td>
          <a href="/someurl">Detaljer</a>
        </td>
      </tr>

    </tbody>
  </table>'''
soup = BeautifulSoup(raw_html, 'html.parser')

table = soup.find("table", attrs={"class":"table"})
head = [th.get_text() for th in table.find("tr").find_all("th")]

datasets = []
for row in table.find_all("tr")[1:]:
    dataset = dict(zip(head, [td.get_text() if not td.a else td.a['href'] for td in row.find_all("td")]))
    datasets.append(dataset)
print(datasets)

输出：

[{'Volymsenhet': 'Antal', 'Pris': '5,40', 'Valuta': 'SEK', 'Handelsplats': 'NASDAQ STOCKHOLM AB', 'url1': '/someurl', 'url2': '/someurl'}]

推荐阅读