时间戳

首页 > 解决方案 > python网页抓取，网页解析器

python - python网页抓取，网页解析器

问题描述

我刚开始学习python，但我在抓取时遇到了问题。代码可以正常工作，但是当我抓取时，却只得到空列表 []。我做错了什么？我找不到同样的问题，感谢您的时间！

`import requests
from bs4 import BeautifulSoup as bs4

headers = {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"

}

url = "https://www.worldometers.info/geography/alphabetical-list-of-countries/"
session = requests.session()


try:
    req = session.get(url, headers=headers)
    if req.status_code == 200:
        soup =  bs4(req.content, "html.parser")
        divs = soup.find_all("div", attrs={"style" : "font-weight"})
        name = soup.find_all()
        print(divs)
except Exception:
    print("ERORR IN URL ADRESS")`

标签： pythonlistweb-scrapingrequesthtml-parsing

解决方案

您可以获取带有类的表table-condensed并找到您需要的数据。请检查以下代码：

import requests
from bs4 import BeautifulSoup

headers = {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"

}

url = "https://www.worldometers.info/geography/alphabetical-list-of-countries/"
session = requests.session()


try:
    req = session.get(url, headers=headers)
    if req.status_code == 200:
        soup =  BeautifulSoup(req.content, "html.parser")
        countries = soup.find("table", {"class": "table-condensed"}).find("tbody").findAll("tr")
        for country in countries:
            print(country.findAll("td")[1].text)
            
except Exception:
    print("ERORR IN URL ADRESS")

推荐阅读