时间戳

首页 > 解决方案 > Amazon Athena 在地理 json 上返回“HIVE_CURSOR_ERROR: HIVE_CURSOR_ERROR”

hive - Amazon Athena 在地理 json 上返回“HIVE_CURSOR_ERROR: HIVE_CURSOR_ERROR”

问题描述

我正在尝试使用 Amazon Athena 读取 geojson 文件。

我的输入数据的头部如下所示：

{
"type": "FeatureCollection",
"name": "sql_statement",
"features": [
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22482 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981739.267883020918816, 30855.609566356935829 ], [ -981739.267883020918816, 31355.354737498135364 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -981241.022986860014498, 30855.609566356935829 ], [ -981739.267883020918816, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22483 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981241.022986860014498, 30855.609566356935829 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980742.778090699226595, 30855.609566356935829 ], [ -981241.022986860014498, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22484 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980742.778090699226595, 30855.609566356935829 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -980244.533194538322277, 30855.609566356935829 ], [ -980742.778090699226595, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22485 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980244.533194538322277, 30855.609566356935829 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979746.288298377417959, 30855.609566356935829 ], [ -980244.533194538322277, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22486 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979746.288298377417959, 30855.609566356935829 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -979248.043402216513641, 30855.609566356935829 ], [ -979746.288298377417959, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22487 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979248.043402216513641, 30855.609566356935829 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -978749.798506055609323, 31355.354737498135364 ], [ -978749.798506055609323, 30855.609566356935829 ], [ -979248.043402216513641, 30855.609566356935829 ] ] ] } },

我使用地理空间文档作为模板定义了表格：

CREATE external TABLE IF NOT EXISTS testdb.grid_500
 (
 gridsize double,
 big_gid int, 
 little_gid int, 
 geometry binary
 )
ROW FORMAT SERDE 'com.esri.hadoop.hive.serde.JsonSerde'
STORED AS INPUTFORMAT 'com.esri.json.hadoop.EnclosedJsonInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://jdl-athena/grid/'
;

然而，我对表运行的任何查询都会返回相同的、相当无用的错误：

Your query has the following error(s):

HIVE_CURSOR_ERROR: HIVE_CURSOR_ERROR

This query ran against the "testdb" database, unless qualified by the query.
Please post the error message on our forum or contact customer support with 
Query Id: 25d3da93-5cfd-46bb-9f77-4eb014679ba6.

任何想法为什么我可能会收到此错误或如何调试它？我什至无法弄清楚如何获得足够的信息来诊断可能发生的事情。

标签： hiveamazon-athena

解决方案

因此，经过多次 Google Fu 之后，我发现（当然）ESRI 已经为地理数据创建了自己的 JSON 格式。我使用的是开放标准GeoJson，ESRI 格式称为ESRI Enclosed JSON. 对于几何 JSON，Athena 仅支持 ESRI 格式，不支持 GEOJSON。

我在这里发现了这种区别： https ://github.com/Esri/gis-tools-for-hadoop/issues/29

所以看起来我需要不同格式的输入数据。

推荐阅读