azure - 通过 ARM 模板在 Azure 数据湖存储帐户中创建文件系统
问题描述
我使用 ARM 模板创建了 Azure Data Lake gen 2。但现在我试图弄清楚如何在 ARM 中创建数据湖文件系统,但似乎找不到执行此操作的 API。这是不可用的,这可能通过另一种方式吗?
尝试手动创建文件系统并在门户中导出模板,但似乎没有看到文件系统资源。只有这个警告。
因为这表明文件系统可能在 API 中作为“blobservices/containers”进行侦听,所以我尝试将此资源添加到 ARM 模板
{
"name": "[concat( parameters('DataLakeName'), '/default/input')]",
"type": "Microsoft.Storage/storageAccounts/blobServices/containers",
"dependsOn": [
"[concat('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
],
"apiVersion": "2018-07-01",
"properties": {
"publicAccess": "None",
"metadata": {}
},
"resources": []
}
但不幸的是,这不起作用并提供了以下错误消息:
Blob API is not yet supported for hierarchical namespace accounts.
这让我认为这甚至可能无法通过 ARM 部署。有人已经尝试过吗?
我的数据湖存储帐户的 ARM 模板资源块作为上下文完成:
{
"name": "[parameters('DataLakeName')]",
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2018-07-01",
"location": "[parameters('location')]",
"tags": {},
"properties": {
"accessTier": "[parameters('accessTier')]",
"supportsHttpsTrafficOnly": "[parameters('supportsHttpsTrafficOnly')]",
"isHnsEnabled": true
},
"resources": [
{
"type": "providers/advancedThreatProtectionSettings",
"name": "Microsoft.Security/current",
"apiVersion": "2017-08-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts/', parameters('DataLakeName'))]"
],
"properties": {
"isEnabled": true
}
}
],
"dependsOn": [],
"sku": {
"name": "[parameters('accountType')]"
},
"kind": "StorageV2"
}
解决方案
我在这个github 问题答案上找到了这个解决方案。从一个发现还没有任何解决方案的人那里,所以这里要做的基本事情是手动调用一个rest api来这样做。解释和博客可以在这个链接中找到: http: //sql.pawlikowski.pro/2019/03/10/connecting-to-azure-data-lake-storage-gen2-from-powershell-using-rest-api -一步一步的指南/
这是用于创建文件系统的 Powershell 脚本,以防不推荐使用该链接:
所有归功于 Michał Pawlikowski,感谢您创建这个脚本就像一个魅力。
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true,Position=1)] [string] $StorageAccountName,
[Parameter(Mandatory=$True,Position=2)] [string] $FilesystemName,
[Parameter(Mandatory=$True,Position=3)] [string] $AccessKey
)
# Rest documentation:
# https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/create
$date = [System.DateTime]::UtcNow.ToString("R") # ex: Sun, 10 Mar 2019 11:50:10 GMT
$n = "`n"
$method = "PUT"
$stringToSign = "$method$n" #VERB
$stringToSign += "$n" # Content-Encoding + "\n" +
$stringToSign += "$n" # Content-Language + "\n" +
$stringToSign += "$n" # Content-Length + "\n" +
$stringToSign += "$n" # Content-MD5 + "\n" +
$stringToSign += "$n" # Content-Type + "\n" +
$stringToSign += "$n" # Date + "\n" +
$stringToSign += "$n" # If-Modified-Since + "\n" +
$stringToSign += "$n" # If-Match + "\n" +
$stringToSign += "$n" # If-None-Match + "\n" +
$stringToSign += "$n" # If-Unmodified-Since + "\n" +
$stringToSign += "$n" # Range + "\n" +
$stringToSign +=
<# SECTION: CanonicalizedHeaders + "\n" #>
"x-ms-date:$date" + $n +
"x-ms-version:2018-11-09" + $n #
<# SECTION: CanonicalizedHeaders + "\n" #>
$stringToSign +=
<# SECTION: CanonicalizedResource + "\n" #>
"/$StorageAccountName/$FilesystemName" + $n +
"resource:filesystem"#
<# SECTION: CanonicalizedResource + "\n" #>
$sharedKey = [System.Convert]::FromBase64String($AccessKey)
$hasher = New-Object System.Security.Cryptography.HMACSHA256
$hasher.Key = $sharedKey
$signedSignature = [System.Convert]::ToBase64String($hasher.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($stringToSign)))
$authHeader = "SharedKey ${StorageAccountName}:$signedSignature"
$headers = @{"x-ms-date"=$date}
$headers.Add("x-ms-version","2018-11-09")
$headers.Add("Authorization",$authHeader)
$URI = "https://$StorageAccountName.dfs.core.windows.net/" + $FilesystemName + "?resource=filesystem"
Try {
Invoke-RestMethod -method $method -Uri $URI -Headers $headers # returns empty response
}
catch {
$ErrorMessage = $_.Exception.Message
$StatusDescription = $_.Exception.Response.StatusDescription
$false
Throw $ErrorMessage + " " + $StatusDescription
}
推荐阅读
- c - 如何知道我的下一次调用 printf() 是否会超出屏幕大小
- python - 如何开始为应用程序构建 API?
- php - $this 的 PHP 自动绑定,是复制还是扩展?
- shopify - Shopify:在新订单电子邮件模板中显示折扣代码
- powershell - 过滤多个 CSV 文本并创建新文件
- django - 如何根据 html 模板中的条件呈现按钮 | 姜戈
- python - 为什么我们不能在集合中添加列表、字典或元组
- botframework - 无法发布活动。未经授权。401 bot框架模拟器
- java - 带有 ssl 的 Spring kafka,生产者错误“重复键”
- android - 是否可以在 Android 导航组件中为事务设置默认动画主题?