时间戳

首页 > 解决方案 > 美汤如何选择某类下的所有数据

python - 美汤如何选择某类下的所有数据

问题描述

当我使用findAll我的班级和我的标签时，代码找不到任何东西
这是我的尝试：

import requests
from bs4 import BeautifulSoup

# get the data
data = requests.get('https://www.twitch.tv/directory/game/Counter-Strike%3A%20Global%20Offensive/clips?range=24hr/')

# load data into bs4
soup = BeautifulSoup(data.text, 'html.parser')

clip = soup.find_all("a", ["tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit"])

print(clip)

有什么问题？

标签： pythonbeautifulsoup

解决方案

数据是通过 Ajax 动态加载的，但是你可以使用requests/ jsonmodules 来加载它：

import json
import requests

url = 'https://gql.twitch.tv/gql#origin=twilight'

params = [{"operationName":"ClipsCards__Game","variables":{"gameName":"Counter-Strike: Global Offensive","limit":20,"criteria":{"languages":[],"filter":"LAST_WEEK"}},"extensions":{"persistedQuery":{"version":1,"sha256Hash":"0d8d0eba9fc7ef77de54a7d933998e21ad7a1274c867ec565ac14ffdce77b1f9"}}}]
headers = {'Client-Id': 'kimne78kx3ncx6brgo4mv6wki5h1ko'}

data =  requests.post(url, headers=headers, json=params).json()

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

for g in data[0]['data']['game']['clips']['edges']:
    print('{:<60} {:<10} {}'.format(g['node']['title'], g['node']['viewCount'], g['node']['url']))

印刷：

12 y/o ace                                                   121409     https://clips.twitch.tv/GoldenBlitheBillKeepo
Dupreeh feeling himself                                      51384      https://clips.twitch.tv/TenderBlatantCattleWoofer
ZyWoo Deagle Ace                                             49288      https://clips.twitch.tv/FurryWiseLemurLitty
ART + GAU JOGO DA TEAM ONE                                   34676      https://clips.twitch.tv/FamousDeliciousCasetteSMOrc
flusha insane 1v4                                            32522      https://clips.twitch.tv/NaiveSavoryLadiesOneHand
UNLUCKY XDD                                                  31698      https://clips.twitch.tv/AveragePoisedDelicataCmonBruh
lindinho clutchzero 1x3                                      24808      https://clips.twitch.tv/SmokyIronicMangetoutDatBoi
Stewie2k Selfless Play                                       24693      https://clips.twitch.tv/TallViscousSlothBloodTrail
bedzie magik flesh                                           20393      https://clips.twitch.tv/TangibleAverageDugongNononoCat
moe vs autimatic on stream                                   20311      https://clips.twitch.tv/CrowdedYummyNigiriPastaThat
Tarik insane 1v5 his first EU FPL game                       20104      https://clips.twitch.tv/HappyTangibleChowderRiPepperonis
Snax zaskakuje przeciwnika i wygrywa przegraną rundę!        19614      https://clips.twitch.tv/SpinelessFragileEchidnaImGlitch
Welcome to EU Tarik                                          18400      https://clips.twitch.tv/StormySavageDotterelYee
csgo'd                                                       17318      https://clips.twitch.tv/EmpathicAwkwardOtterHeyGuys
epic sauce                                                   15501      https://clips.twitch.tv/AttractiveRacyPhoneYouWHY
NiKo 1v2 2x 1taps                                            15088      https://clips.twitch.tv/SuspiciousJoyousSalsifyPogChamp
morda                                                        14394      https://clips.twitch.tv/PuzzledLightMonkeyMingLee
RAIN 3k vs sAw                                               14387      https://clips.twitch.tv/TastySavoryKangarooPRChase
Synchronous grenade throwing                                 13881      https://clips.twitch.tv/TransparentFrailCoffeePipeHype
sneaky fail ;p                                               13668      https://clips.twitch.tv/SquareRockyBunnyCclamChamp

推荐阅读