python - 用 BeautifulSoup 隔离脚本

Question

我已经在 BeautifulSoup 中加载了整个 HTML 页面。有没有办法可以隔离这个字典集合？

• 类型（黄色）在页面中只出现一次，没有重复

这是我用来导入 HTML 文件的代码（不能使用 urllib）：from bs4 import BeautifulSoup

with open('/content/drive/My Drive/Colab Notebooks/Projects/20200710_StreetEasy_WebScraping/1.html') as f:
  contents = f.read()
  soup = BeautifulSoup(contents, 'lxml')
print(soup)

搜索 a 标签返回输出

a = soup.find_all('a')
a
[<a class="html-attribute-value html-resource-link" href="https://cdn-assets-s3.streeteasy.com/assets/manifest-c93475b02bd2409b4a52e21af023e5d5f489f19500d234a3660fe4d35069bbac.json" rel="noreferrer noopener" target="_blank">//cdn-assets-s3.streeteasy.com/assets/manifest-c93475b02bd2409b4a52e21af023e5d5f489f19500d234a3660fe4d35069bbac.json</a>,
 <a class="html-attribute-value html-resource-link" href="https://browser.sentry-cdn.com/5.19.0/bundle.min.js" rel="noreferrer noopener" target="_blank">https://browser.sentry-cdn.com/5.19.0/bundle.min.js</a>,
 <a class="html-attribute-value html-resource-link" href="https://cdn-assets-s3.streeteasy.com/assets/jquery-fe1be651ec56a9cc875a437f09db5b175cc6acf4b911bed0ef265955a099db55.js" rel="noreferrer noopener" target="_blank">//cdn-assets-s3.streeteasy.com/assets/jquery-fe1be651ec56a9cc875a437f09db5b175cc6acf4b911bed0ef265955a099db55.js</a>,
...

搜索脚本标签不返回任何输出

import re
scripts = soup.find_all("script")
scripts
[]

也许我在导入文档时做错了什么？

标签： pythonhtmlbeautifulsoup

Answer 1

您可以使用find_all中的字符串参数来过滤包含JSON的脚本标签@context

scripts = soup.find_all("script", string=re.compile("@context"))

然后遍历你的scripts并在删除后加载 JSON//<![CDATA[和//]]