时间戳

首页 > 解决方案 > 使用python在Barchart网站上抓取表格但只出现一个信号数据

python - 使用python在Barchart网站上抓取表格但只出现一个信号数据

问题描述

我使用上面链接中的脚本在 Barchart 网站上获取了一个表格，它以某种方式只抓取了单日数据，而不是出现在整个页面上的数据。我猜有些填充我错了，我不知道如何解决它。


import requests
from urllib.parse import unquote


geturl=r'https://www.barchart.com/stocks/quotes/AAPL%7C20210423%7C126.00C/price-history/'
apiurl=r'https://www.barchart.com/proxies/core-api/v1/quotes/get'


getheaders={

    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9',
    'cache-control': 'max-age=0',
    'upgrade-insecure-requests': '1',
    "referer":"https://www.barchart.com/stocks/quotes/AAPL%7C20210423%7C126.00C/price-history/historical",
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36'
    }



s=requests.Session()
r=s.get(geturl, headers=getheaders)


headers={
    'accept': 'application/json',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9',

    'user-agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36",
    'x-xsrf-token': unquote(unquote(s.cookies.get_dict()['XSRF-TOKEN']))


}



payload={

    "symbol":"AAPL|20210423|126.00C",
    "fields":"tradeTime.format(m\/d\/Y),openPrice,highPrice,lowPrice,lastPrice,priceChange,percentChange,volume,openInterest,impliedVolatility,symbolCode,symbolType",
    "type":"eod",
    "orderBy":"tradeTime",
    "orderDir":"desc",
    "limit":65,
    #"meta":"field.shortName,field.type,field.description",
    'raw': '1'


}

r=s.get(apiurl,params=payload,headers=headers)
j=r.json()
print(j)

输出：

{'count': 1, 'total': 1, 'data': [{'tradeTime': '04/21/2021', 'openPrice': '6.65', 'highPrice': '7.65', 'lowPrice': '5.56', 'lastPrice': '7.58', 'priceChange': '+0.45', 'percentChange': '+6.31%', 'volume': '1,213', 'openInterest': '3,951', 'impliedVolatility': 'N/A', 'symbolCode': 'STKOPT', 'symbolType': 34, 'raw': {'tradeTime': 1619036084, 'openPrice': 6.65, 'highPrice': 7.65, 'lowPrice': 5.56, 'lastPrice': 7.58, 'priceChange': '0.45', 'percentChange': 0.0631, 'volume': 1213, 'openInterest': 3951, 'impliedVolatility': None, 'symbolCode': 'STKOPT', 'symbolType': 34}}]}

我预计它可以抓取3 个月的数据，有人能找出问题所在吗？谢谢

标签： pythonweb-scrapingpython-requests

解决方案

这对我有用：

将 API 'https://www.barchart.com/proxies/core-api/v1/quotes/get' 替换为 'https://www.barchart.com/proxies/core-api/v1/historical/get'

输出 {'count': 32, 'total': 4, 'data': [{'tradeTime': '04/23/2021', 'openPrice': '6.16', 'highPrice': '9.09', '. ..

推荐阅读