python - 如何将弹性搜索索引的响应保存到 s3 存储桶中
问题描述
如何在 my-index 下面生成
在本地创建一个环境
使用 pip 安装必要的库(elasticsearch、requests、requests_aws4auth、boto3)
使用 lambda_function.py在里面创建文件
env\Lib\site-packages\
并添加以下代码压缩上述文件夹并将其命名为 lambda_function.zip 并上传到 lambda 函数,您可以在其中创建具有必要 IAM 角色的函数
import boto3 from requests_aws4auth import AWS4Auth from elasticsearch import Elasticsearch, RequestsHttpConnection session = boto3.session.Session() credentials = session.get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, session.region_name, 'es', session_token=credentials.token) es = Elasticsearch( ['https://search-testelastic-2276kyz2u4l3basec63onfq73a.us-east-1.es.amazonaws.com'], http_auth=awsauth, use_ssl=True, verify_certs=True, connection_class=RequestsHttpConnection ) def lambda_handler(event, context): es.cluster.health() es.indices.create(index='my-index', ignore=400) r = [{'Name': 'Dr. Christopher DeSimone', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Tajwar Aamir (Aamir)', 'Specialised and Location': 'Health'}] for e in enumerate(r): es.index(index="my-index", body=e[1])
回复如下
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":3,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"my-index_1","_type":"_doc","_id":"elqrJHMB10jKFvejVaNM","_score":1.0,"_source":{"Name":"Dr. Christopher DeSimone","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"e1qrJHMB10jKFvejVqMK","_score":1.0,"_source":{"Name":"Dr. Tajwar Aamir (Aamir)","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"fFqrJHMB10jKFvejVqMR","_score":1.0,"_source":{"Name":"Dr. Bernard M. Aaron","Specialised and Location":"Health"}}]}}
- 如何将上述响应保存为 s3 存储桶中文件夹中的 json
存储桶名称 = test20220elastic
解决方案
您可以重用您的session
对象来创建 S3 资源:
es = ...
s3 = session.resource('s3')
bucket = s3.Bucket('test20220elastic')
def lambda_handler(event, context):
...
for e in enumerate(r):
result = es.index(index="my-index", body=e[1])
bucket.put_object(Body=json.dumps(result), Key="my_folder/my_result.json",
ContentType="application/json")
不过,您可能希望为每个结果建立一个不同的键名。
推荐阅读
- python-3.x - 如何使用 sklearn 对具有多个特征的时间序列数据进行归一化?
- html - 在 html 中使用节点搜索 mysql 数据库
- asp.net - 如何在.net core web api中路由控制器类?
- c++ - 使用未命名管道的 IPC
- python - Python TypeError:“NoneType”对象不可迭代,但定义了每个变量
- r - Logistic 回归的 ROC 异常
- c# - 无法获取 web html 代码:System.Net.WebException:'远程服务器返回错误:(403)禁止。
- python - 使用 lxml etree 直接访问元素和属性
- c++ - OpenGl glutIdleFunc Animation
- c - 可调变量 Simulink 和实时研讨会 (RSIM)