时间戳

首页 > 解决方案 > Python 的表 Web 抓取问题

python - Python 的表 Web 抓取问题

问题描述

我在从该网站抓取数据时遇到问题：https ://fantasy.premierleague.com/player-list

我有兴趣从不同的表中获取玩家的姓名和分数。

我对 python 比较陌生，对网络抓取完全陌生。这是我到目前为止所拥有的：

from urllib.request import urlopen
from bs4 import BeautifulSoup


url = 'https://fantasy.premierleague.com/player-list'


html = urlopen(url)
soup = BeautifulSoup(html, "lxml")

rows = soup.find_all('tr')
print(rows)

从这里我会继续查找所有“td”信息。

但是我没有得到'tr'的结果。我可以将“a”作为参数传递，并获得该站点的链接，但无法从表中获取任何数据。我的理解是通过“tr”将找到网站内任何表格的所有行

有什么想法我哪里出错了吗？谢谢你的帮助

标签： pythonhtmlweb-scrapingbeautifulsoup

解决方案

您可以使用 来获取所有表数据webdriver，pandas并且BeautifulSoup.

from bs4 import BeautifulSoup
import requests
from selenium import webdriver
import pandas as pd
url = "https://fantasy.premierleague.com/player-list"

driver = webdriver.Firefox()
driver.get(url)

html = driver.page_source
soup = BeautifulSoup(html)
table = soup.find_all('table', {'class': 'Table-ziussd-1 fVnGhl'})

df = pd.read_html(str(table))

print(df)

输出将是：

[             Player            Team  Points  Cost
0           Alisson       Liverpool      99  £6.2
1           Ederson        Man City      89  £6.0
2              Kepa         Chelsea      72  £5.4
3        Schmeichel       Leicester     122  £5.4
4            de Gea         Man Utd     105  £5.3
5            Lloris           Spurs      56  £5.3
6         Henderson   Sheffield Utd     135  £5.3
7          Pickford         Everton      93  £5.2
8          Patrício          Wolves     122  £5.2
9          Dubravka       Newcastle     124  £5.1
10             Leno         Arsenal     114  £5.0
11           Guaita  Crystal Palace     122  £5.0
12             Pope         Burnley     129  £4.9
13           Foster         Watford     113  £4.9
14        Fabianski        West Ham      61  £4.9
15        Caballero         Chelsea       7  £4.8
16             Ryan        Brighton     105  £4.7
17            Bravo        Man City      11  £4.7
18            Grant         Man Utd       0  £4.7
19           Romero         Man Utd       0  £4.6
20             Krul         Norwich      94  £4.6
21         Mignolet       Liverpool       0  £4.5
22         McCarthy     Southampton      74  £4.5
23         Ramsdale     Bournemouth      97  £4.5
24         Fahrmann         Norwich       1  £4.4




and so on........................................]

推荐阅读