python - 使用 python 将 JSON 发送到带有时间戳字段的 ElasticSearch 索引
问题描述
我有一个脚本可以调用我的 API 并提取库存数据。我想将来自 API 的响应保存为 JSON 文件,然后使用响应中的“t”字段作为日期字段/时间戳创建 ES 索引。
我可以在我的 ES 集群中看到数据,但它没有被索引,并且“t”字段显示为错误的类型。长而不是日期。
我不确定我能如何最好地索引它,results
因为这对我来说很关键,因为我真的想把它展示为一个时间序列。
收集器.py
def collect():
import json
key = ""
url = "https://api"
payload={}
headers = {}
response = requests.request("GET", url, headers=headers, data=payload)
j = response.json()
print (j['results'][1])
data = response.text
for i in j['results']:
print (i)
with open('data.json', 'w', encoding='utf-8') as f:
json.dump(j, f, ensure_ascii=False, indent=4)
import os, sys
from elasticsearch import Elasticsearch
directory = '/home/'
res = requests.get('http://localhost:9200')
print (res.content)
es = Elasticsearch([{'host': 'localhost', 'port': '9200'}])
i = 1
for filename in os.listdir(directory):
if filename.endswith(".json"):
f = open(filename)
docket_content = f.read()
# Send the data into es
es.index(index='core', ignore=400, doc_type='docket',
id=i, body=json.loads(docket_content))
i = i + 1
return render_template('show.html', data=data)
JSON 响应和 data.json 的内容
{
"ticker": "AAPL",
"queryCount": 10,
"resultsCount": 10,
"adjusted": true,
"results": [
{
"v": 1668,
"vw": 151.8826,
"a": 151.8826,
"o": 152,
"c": 151.92,
"h": 152,
"l": 151.75,
"t": 1636012800000,
"n": 70
},
{
"v": 1467,
"vw": 151.9323,
"a": 151.9059,
"o": 151.95,
"c": 151.96,
"h": 151.96,
"l": 151.89,
"t": 1636012860000,
"n": 60
},
{
"v": 1096,
"vw": 151.9585,
"a": 151.9195,
"o": 151.96,
"c": 151.94,
"h": 151.96,
"l": 151.94,
"t": 1636012920000,
"n": 64
},
{
"v": 1303,
"vw": 151.7871,
"a": 151.8889,
"o": 151.77,
"c": 151.73,
"h": 151.77,
"l": 151.73,
"t": 1636013040000,
"n": 70
},
{
"v": 847,
"vw": 151.8279,
"a": 151.8811,
"o": 151.87,
"c": 151.8,
"h": 151.87,
"l": 151.8,
"t": 1636013100000,
"n": 37
},
{
"v": 451,
"vw": 151.8722,
"a": 151.8737,
"o": 151.87,
"c": 151.87,
"h": 151.87,
"l": 151.87,
"t": 1636013280000,
"n": 17
},
{
"v": 5347,
"vw": 151.8021,
"a": 151.8446,
"o": 151.82,
"c": 151.8,
"h": 151.82,
"l": 151.8,
"t": 1636013400000,
"n": 55
},
{
"v": 2834,
"vw": 151.7431,
"a": 151.8266,
"o": 151.74,
"c": 151.73,
"h": 151.74,
"l": 151.73,
"t": 1636013460000,
"n": 58
},
{
"v": 615,
"vw": 151.7193,
"a": 151.8226,
"o": 151.73,
"c": 151.68,
"h": 151.73,
"l": 151.68,
"t": 1636013520000,
"n": 22
},
{
"v": 876,
"vw": 151.717,
"a": 151.8173,
"o": 151.71,
"c": 151.73,
"h": 151.73,
"l": 151.71,
"t": 1636013580000,
"n": 33
}
],
"status": "OK",
"request_id": "2354523452356236",
"count": 10
}
弹性指数
core
mappings
properties
a
type "float"
c
type "float"
h
type "float"
l
type "float"
n
type "long"
o
type "float"
t
type "long"
v
type "long"
vw
type "float"
解决方案
你真的应该在索引之前定义你的映射,要么使用模板,要么使用集合映射发布索引。否则 Elasticsearch 将尝试为每个字段选择最佳数据类型,但并不总是正确,如您所见
推荐阅读
- maxima - 更改最大值乘法的标准tex输出
- javascript - 当为 webpack 输入命令“npm run dev”时,它会创建一个以 NaN 结尾的文件夹
- r - 仅在某些组中的回归线(ggplot2)
- html - 在同一行或同一行显示两个 div
- python - 在右侧对齐图标并将文本在 QPushButton 中居中
- wso2 - 配置 Api Analytics 后尝试登录 Api Manager 时出现错误 Wso2“问题访问:/。原因:未找到”
- c# - 如何在通用函数中使用 if/else?C#
- java - Java:确保 Files.list() 仅返回“完整”文件
- xamarin - 是否可以为 Xamarin Frame 提供不同的边框颜色?
- python - 试图从多个链接中抓取