java - SQL Polybase 可以从 Azure datalake gen2 读取数据吗？

问题描述

我想将 Polybase 设置为从 Azure Data Lake Gen2 读取/写入数据。

到目前为止，通过 Polybase 将数据写入 Azure 没有任何问题。

使用的代码：

CREATE DATABASE SCOPED CREDENTIAL AzureStorageKey
WITH IDENTITY = 'Some_Any_String', Secret = 'theazureKey00102037etcetc';
GO



CREATE EXTERNAL DATA SOURCE AzureStorage 
with (  
      TYPE = HADOOP,
      LOCATION ='wasbs://mycontainer@mylake.blob.core.windows.net',  
      CREDENTIAL = AzureStorageKey  
);  
GO


CREATE EXTERNAL FILE FORMAT DelimitedFormat WITH (  
      FORMAT_TYPE = DELIMITEDTEXT,
      FORMAT_OPTIONS (FIELD_TERMINATOR =';',
            USE_TYPE_DEFAULT = TRUE)); 
GO


CREATE EXTERNAL TABLE [dbo].[TestTable] (  
      [ID]          INT         NOT NULL,
      [Name]        VARCHAR(10) NOT NULL,
      [SomeValue]   INT             NULL
)  
WITH (LOCATION='/2021/01/10',
      DATA_SOURCE = AzureStorage,  
      FILE_FORMAT = DelimitedFormatCom
); 



INSERT INTO [dbo].[TestTable] (ID, Name, SomeValue) 
SELECT 1 as ID, 'AAA' as Name, 100 as SomeValue
union
SELECT 2 as ID, 'BBB' as Name, 200 as SomeValue
union
SELECT 3 as ID, 'CCC' as Name, 300 as SomeValue

插入结果

(3 rows affected)

Completion time: 2021-01-11T22:08:53.0447043+00:00

我可以看到数据湖中的数据

如果我想从上面读取数据

SELECT * FROM [dbo].TestTable

结果是一个错误

Cannot execute the query "Remote Query" against OLE DB provider "MSOLEDBSQL" for linked server "(null)". HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;

我正在使用本地 SQL Server 2019 - 开发人员版。

已安装 Java

谢谢你。

标签： javasql-serverazurepolybase

java - SQL Polybase 可以从 Azure datalake gen2 读取数据吗？

问题描述

解决方案

推荐阅读