python - Beautifulsoup 为文本和链接解析 html 表

Question

我有一张有几列的桌子。最后一列可能包含指向文档的链接，每个单元格的链接数未确定（从 0 到无穷大）。

<tbody>
     <tr>
        <td>
          <h2>Table Section</h2>
        </td>
      </tr>
    <tr>
      <td>
        <a href="#">Object 1</a>
      </td>
      <td>Param 1</td>
      <td>
        <span class="text-nowrap">Param 2</span>        
      </td>
      <td class="text-nowrap"></td>
    </tr>

    <tr>
      <td>
        <a href="#">Object 2</a>
      </td>
      <td>Param 1</td>
      <td>
        <span class="text-nowrap">Param 2</span>
      <td>
          <ul>
            <li>
              <small>
                <a href="link_to.doc">Title</a>Notes
              </small>
            </li>

            <li>
              <small>
                <a href="another_link_to.doc">Title2</a>Notes2
              </small>
            </li>
          </ul>
      </td>
    </tr>
</tbody>

所以基本解析不是问题。我坚持使用标题和注释获取这些链接并将它们附加到 python 的列表（或 numpy 数组）。

from bs4 import BeautifulSoup

with open("new 1.html", encoding="utf8") as dump:
    soup = BeautifulSoup(dump, features="lxml")

data = []

table_body = soup.find('tbody')
rows = table_body.find_all('tr')

for row in rows:
    cols = row.find_all('td')
    cols = [ele.text.strip() for ele in cols]
    data.append(cols)
    a = row.find_all('a')
    for ele1 in a:
        if ele1.get('href') != "#":
            data.append([ele1.get('href')])
print(*data, sep='\n')

输出：

['Table Section']
['Object 1', 'Param 1', 'Param 2', '']
['Object 2', 'Param 1', 'Param 2', 'TitleNotes\n\t\t\t  \n\n\n\nTitle2Notes2']
['link_to.doc']
['another_link_to.doc']

有没有办法将链接附加到第一个列表？我希望第二行的列表如下所示：

['Object 2', 'Param 1', 'Param 2', 'Title', 'Notes', 'link_to.doc', ' Title2', 'Notes2', 'another_link_to.doc']

标签： pythonbeautifulsoup

Answer 1

像这样的东西

from bs4 import BeautifulSoup


html = '''<tbody>
     <tr>
        <td>
          <h2>Table Section</h2>
        </td>
      </tr>
    <tr>
      <td>
        <a href="#">Object 1</a>
      </td>
      <td>Param 1</td>
      <td>
        <span class="text-nowrap">Param 2</span>        
      </td>
      <td class="text-nowrap"></td>
    </tr>

    <tr>
      <td>
        <a href="#">Object 2</a>
      </td>
      <td>Param 1</td>
      <td>
        <span class="text-nowrap">Param 2</span>
      <td>
          <ul>
            <li>
              <small>
                <a href="link_to.doc">Title</a>Notes
              </small>
            </li>

            <li>
              <small>
                <a href="another_link_to.doc">Title2</a>Notes2
              </small>
            </li>
          </ul>
      </td>
    </tr>
</tbody>'''


soup = BeautifulSoup(html, features="lxml")
smalls = soup.find_all('small')
links = [s.contents[1].attrs['href'] for s in smalls]
print(links)

输出

['link_to.doc', 'another_link_to.doc']