python-3.x - Beautiful Soup Error: Trying to retrieve data from web page returns empty array
问题描述
I am trying to download a list of voting intention opinion polls from this web page using beautiful soup. However, the code I wrote returns an empty array or nothing. The code I used is below:
The page code is like this:
<div class="ST-c2-dv1 ST-ch ST-PS" style="width:33px"></div>
<div class="ST-c2-dv2">41.8</div>
That's what I tried:
import requests
from bs4 import BeautifulSoup
request = requests.get(quote_page) # take the page link
page = request.content # extract page content
soup = BeautifulSoup(page, "html.parser")
# extract all the divs
for each_div in soup.findAll('div',{'class':'ST-c2-dv2'}):
print each_div
At this point, it prints nothing. I've tried also this:
tutti_a = soup.find_all("html_element", class_="ST-c2-dv2")
and also:
tutti_a = soup.find_all("div", class_="ST-c2-dv2")
But I get an empty array [] or nothing at all
解决方案
我认为您可以使用以下网址
import requests
from bs4 import BeautifulSoup as bs
import pandas as pd
r = requests.get('https://www.marktest.com/wap/a/sf/v~[73D5799E1B0E]/name~Dossier_5fSondagensLegislativas_5f2011.HighCharts.Sondagens.xml.aspx')
soup = bs(r.content, 'lxml')
results = []
for record in soup.select('p'):
results.append([item.text for item in record.select('b')])
df = pd.DataFrame(results)
print(df)
第 5、6、7、8、9、10 列对应于 PS、PSD、CDS、CDU、Bloco、Outros/Brancos/Nulos
您可以删除不需要的列,添加适当的标题等。
推荐阅读
- python - bottle.py - AttributeError 'NoneType' 对象没有属性 'write'
- math - 增加多边形面积后求多边形内部坐标值
- c# - 我可以将 X509 证书转换为字符串/字节以供以后使用吗?
- windows - 使用 WebRTC 通过应用程序运行 Chrome
- python - While循环内部的异步函数调用
- navigationview - NavigationView 汉堡按钮样式
- css - 如何在与元素相同的行上显示 ::after 内容?
- ios - iOS 应用程序有时会在发布模式下崩溃 (EXC_BAD_ACCESS (SIGSEGV))
- angular - Angular 9 - 如何将文本数据数组从父组件传递到多层子组件?
- javascript - React JS,webpack 配置不正确以加载 mp3 播放器