时间戳

首页 > 解决方案 > 使用冗余标签解析 HTML 的精确答案

python-3.x - 使用冗余标签解析 HTML 的精确答案

问题描述

我正在寻找解析Bert as a service的常见问题解答。

我对这个 HTML 很感兴趣：

<h5>
    <a id="user-content-q-how-do-you-get-the-fixed-representation-did-you-do-pooling-or-something" class="anchor" aria-hidden="true" href="#q-how-do-you-get-the-fixed-representation-did-you-do-pooling-or-something">
    <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true">
        <path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45">
        </path>
    </svg>
    </a>
    <strong>Q:</strong> How do you get the fixed representation? Did you do pooling or something?
</h5>
<p><strong>A:</strong> Yes, pooling is required to get a fixed representation of a sentence. In the default strategy <code>REDUCE_MEAN</code>, I take the second-to-last hidden layer of all of the tokens in the sentence and do average pooling.</p>

我已经成功地将问题与答案分开检索。但是答案的标签形式并不是多余的。这是我解析此 HTML 的代码：

import requests
from bs4 import BeautifulSoup  

wp = requests.get("https://github.com/hanxiao/bert-as-service")
soup = BeautifulSoup(wp.text, "html.parser")

# Parse the questions    
results = soup.find_all("h5") 

questions = []
for result in results:
    question = result.contents[2]
    questions.append(question)

# Parse the answers
new_tag = soup.find_all("p")
new_tag = new_tag[114:165] # specify the tag of the answers


answers = []
for new in new_tag:
    answer = new.contents[1]

我的答案非常糟糕，因为<p>标签非常频繁。

标签： python-3.xbeautifulsouphtml-parsing

解决方案

您还可以执行以下操作

import requests
from bs4 import BeautifulSoup  

wp = requests.get("https://github.com/hanxiao/bert-as-service")
soup = BeautifulSoup(wp.text, "lxml")
titles = [item.text.lstrip('Q: ') for item in soup.select('h5')]
initial_paras = [item.text.lstrip('A: ') for item in soup.select('h5 + p')]
print(len(titles), len(initial_paras))

推荐阅读