时间戳

首页 > 解决方案 > 使用 python 从图形中抓取工具提示

python - 使用 python 从图形中抓取工具提示

问题描述

我试图弄清楚如何从这个 url中的图表中刮取工具提示值

我无法通过在 chrome 中检查任何图形元素来弄清楚它们是如何呈现的。此外，以下语句返回的 html 似乎与我在 chrome 中看到的不同：

r = requests.get("https://coronavirus.iowa.gov")

任何帮助表示赞赏。

标签： pythonweb-scrapinggraphbeautifulsoup

解决方案

数据是通过 JavaScript 动态加载的，因此解析有点复杂。

您可以使用此示例：

import re
import requests
import json

url = 'https://public.domo.com/embed/pages/dPRol'
url2 = 'https://public.domo.com/embed/pages/dPRol/stack?parts=metadata,datasources,drillPathURNs,domoapp'
url3 = 'https://public.domo.com/embed/pages/dPRol/cards/{urn}/render?parts=image,summary'
json_data = json.loads(json.loads(r'"{\"queryOverrides\":{\"filters\":[]},\"chartState\":{\"overrides\":{\"hide_annotation_list\":true}},\"transparent\":true,\"textColor\":\"#54585A\",\"scaleLineColor\":\"#D3D3D2\",\"imageMap\":true,\"pageLayout\":true,\"width\":793,\"height\":182,\"scale\":1,\"cardLoadContext\":{\"context\":\"page\",\"sessionId\":\"bb2fb992-d6cd-4a92-b21d-6e14a660b5e4\",\"visibilityState\":\"visible\",\"contextId\":\"dPRol\",\"trigger\":\"initial_load\"}}"'))
token = re.search(r"'x-domo-embed-token': '(.*?)'", requests.get(url).text).group(1)
data = requests.get(url2, headers={'x-domo-embed-token':token}).json()

# set search_for to one of following:
#   Individuals Tested
#   Individuals Positive
#   Total Recovered
#   Total Deaths
search_for = 'Individuals Tested'

for card in data['cards']:
    if card['title'] == search_for and \
       card['metadata']['chartType'] in ('badge_line_bar', 'badge_line_stackedbar'):
        urn = card['urn']
        break

final_data = requests.put(url3.format(urn=urn), json=json_data, headers={'X-DOMO-Embed-Token': token, 'X-Requested-With': 'XMLHttpRequest'} ).json()

for image in final_data['image']['jsonImageMap']:
    print(image['t'])

印刷：

Individuals Tested on 2020-05-24 : 52,189
Individuals Tested on 2020-05-23 : 51,765
Individuals Tested on 2020-05-22 : 48,200
Individuals Tested on 2020-05-21 : 47,410
Individuals Tested on 2020-05-20 : 45,411
Individuals Tested on 2020-05-19 : 44,929
Individuals Tested on 2020-05-18 : 44,106
Individuals Tested on 2020-05-17 : 43,951
Individuals Tested on 2020-05-16 : 43,089
Individuals Tested on 2020-05-15 : 43,708
Individuals Tested on 2020-05-14 : 42,137
Individuals Tested on 2020-05-13 : 41,523
Individuals Tested on 2020-05-12 : 38,791
Individuals Tested on 2020-05-11 : 37,522

... and so on.

推荐阅读