时间戳

首页 > 解决方案 > Python Beautifulsoup（bs4）findAll没有找到所有元素

python - Python Beautifulsoup（bs4）findAll没有找到所有元素

问题描述

从代码中的 url，我最终试图从页面中收集所有玩家的名字。但是，当我使用 .findAll 来获取所有列表元素时，我还没有成功。请指教。

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

players_url = 'https://stats.nba.com/players/list/?Historic=Y'

# Opening up the Connection and grabbing the page
uClient = uReq(players_url)
page_html = uClient.read()

players_soup = soup(page_html, "html.parser")

# Taking all of the elements from the unordered lists that contains all of the players.

list_elements = players_soup.findAll('li', {'class': 'players-list__name'})

标签： pythonweb-scrapingbeautifulsoup

解决方案

正如@Oluwafemi Sule建议的那样，最好与以下内容selenium一起使用BS：

from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Firefox()
driver.get('https://stats.nba.com/players/list/?Historic=Y')
soup = BeautifulSoup(driver.page_source, 'lxml')
for div in soup.findAll('li', {'class': 'players-list__name'}):
    print(div.find('a').contents[0])

输出：

Abdelnaby, Alaa
Abdul-Aziz, Zaid
Abdul-Jabbar, Kareem
Abdul-Rauf, Mahmoud
Abdul-Wahad, Tariq

等等

推荐阅读