web-scraping - 如何通过 BeautifulSoup 提取带有 href 的列表项？

Question

HTML 如下所示：

<h2 class="block-title">Content</h2>
<div class="catalog-list column-1">
<ol>
<li class="level1">
<span class="index">1</span>
<span class="text"><a href="#1">Basic Info</a></span>
</li>
<li class="level1">
<span class="index">2</span>
<span class="text"><a href="#2">Life</a></span>
</li>
<li class="level1">
<span class="index">3</span>
<span class="text"><a href="#3">Achievements</a></span>
</li>
</ol>

</div>
</div>
</div>
<div class="anchor-list ">
<a name="1" class="lemma-anchor para-title" ></a>
<a name="sub20964854_1" class="lemma-anchor " ></a>
<a name="Basic Info" class="lemma-anchor " ></a>
</div><div class="para-title level-2" label-module="para-title">
<h2 class="title-text"><span class="title-prefix">John Smith</span>Basic Info</h2>
<a class="edit-icon j-edit-link" data-edit-dl="1" href="javascript:;"><em class="cmn-icon wiki-lemma-icons wiki-lemma-icons_edit-lemma"></em>edit</a>
</div>
<div class="para" label-module="para">Name：John Smith</div>
<div class="para" label-module="para">Nationality：USA</div>
<div class="para" label-module="para">Sex：Male</div>
<div class="para" label-module="para">Height：184cm</div>
<div class="para" label-module="para">Birthday：1994-06-01</div>
<div class="para" label-module="para">Place：CA, USA<sup class="sup--normal" data-sup="1" data-ctrmap=":1,">

我想提取“基本信息”部分，其href没有。是“#1”，下面是我要提取的属性列表。

我正在使用 BeautifulSoup。我遇到了处理其他标签的方法，但不知道如何处理这些标签，即'label-module'。

标签： web-scrapingbeautifulsoup

Answer 1

尝试这个。

from simplified_scrapy import SimplifiedDoc
html = ''''''
doc = SimplifiedDoc(html)
baseInfoH2 = doc.getElementByText('Basic Info','h2')
baseInfo = doc.getElements('div',attr='label-module',value='para',start=baseInfoH2._end).text # Use h2 to locate
print (baseInfo)

结果：

['Name：John Smith', 'Nationality：USA', 'Sex：Male', 'Height：184cm', 'Birthday：1994-06-01']