python - 为什么我在 HTML 标签之间没有得到任何内容？

Question

我正在尝试从该网站“ https://www.physiotherapyexercises.com/ ”中提取数据。但是当我运行以下代码时，我只得到了 html 正文，没有得到 HTMl 标签之间的内容。

import requests
import bs4

url = "https://www.physiotherapyexercises.com/"
res = requests.get(url)
soup = bs4.BeautifulSoup(res.text, 'lxml')

print(soup)

标签： pythonpython-3.xweb-scrapingbeautifulsoup

Answer 1

您正在搜索的数据是使用 Javascript 从其他位置加载的。您可以使用requests//模块来获取信息re。json

例如：

import re
import json
import requests

url = 'https://www.physiotherapyexercises.com/Js/Data/ExerciseData_English_2019_12_06_00_49_56.js'

data = json.loads(re.search(r'exerciseRecords=(.*?]);', requests.get(url).text).group(1))

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

for exercise in data:
    print(*exercise['Texts'], sep='\n')
    print('-' * 80)

印刷：

...

Strength - knee - flexors - wall mounted pulleys - prone.
Knee flexor strengthening in prone using pulleys
To strengthen the knee flexors.
To strengthen the muscles at the back of your thigh.
Position the patient in prone facing away from the pulleys. Adjust the pulley system so that the direction of pull opposes knee flexion. Instruct the patient to flex their knee.
Position yourself lying on your stomach. Adjust the pulley system so that the direction of pull is towards the top of the bed. Start with your knee straight and leg resting on the bed. Finish with your knee bent.
More advanced: 1. Progress using strength training principles.

--------------------------------------------------------------------------------
Strength - knee - flexors - theraband - prone.
Knee flexor strengthening in prone using theraband
To strengthen the knee flexors.
To strengthen the muscles at the back of your thigh.
Position the patient in prone with their knee extended. Adjust the theraband so that the direction of pull opposes knee flexion. Instruct the patient to flex their knee.
Position yourself lying on your stomach. Adjust the theraband so that the direction of pull is towards the top of the bed. Start with your knee straight and leg resting on the bed. Finish with your knee bent.
Less advanced: 1. Downgrade the colour of the theraband. More advanced: 1. Upgrade the colour of the theraband.

... and so on.