python - Converting list of JSON pages to one object
问题描述
I want to create program which uses api to generate some data/statistics.
Api is presented on 32 pages, so I have to iterate, add page to list and after getting all of them I want to convert elements in list to one huge object and operate on him.
So, to get data from API and store in the list I have:
response = []
for i in range(1,32):
url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
response.append(requests.get(url).json)
I was trying using json.loads(), json.dumps() but nothing worked. Any solution idea hot to solve this?
EDIT: when I collect my response and use:
valid = json.dumps(response)
print(valid)
I got error: Object of type 'method' is not JSON serializable
解决方案
尝试:
import json
def collect_responses(num_of_pages):
response = []
for i in range(1, num_of_pages):
url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
response.append(requests.get(url).json())
return response
valid = json.dumps({"responseList": collect_responses(32)}
或者以更一般的方式:
def paginate_while_200(url_template_missing_pagenum):
list_of_json_response = []
i = 1
while True:
response = requests.get(url % i)
if response.status == 200:
try:
list_of_json_response.append(response.json())
i += 1
finally:
return list_of_json_response
else:
return list_of_json_response
url = "https://api.dane.gov.pl/resources/17201/data?page=%d"
json.dumps({"responseList": paginate_while_200(url)})
推荐阅读
- sql - 如何最有效地探索一列 url 查询字符串的内容
- playframework-2.5 - java.lang.NoClassDefFoundError : com/google/gson/JsonElement on playframework 在启动服务器时不起作用
- javascript - 使用 XPath 选择包含特定文本的 LI 时遇到问题
- node.js - Nodejs Transform - 在推送之前调用回调可以被认为是好的做法吗?
- swift - Swift 将控制器推回导航堆栈
- xamarin - Xamarin iOS 图像在 Grid 内重叠
- tensorflow - 如何为自定义算子自动实现 NCHW 格式?
- docker - 如何选择性地忽略用于同一项目中不同 Docker 文件的文件夹
- python - Selenium 不可见异常
- javascript - 如何在 eruby 代码中实现 JS 标记以在点击时触发?