azure-data-factory - 是否可以使用 powershell 为长时间运行的作业设置 ADF 触发器超时

Question

我需要使用 PowerShell 为 ADF 中长时间运行的作业设置 Azure ADF 触发器超时。如果触发器激活超过脚本中定义的特定时间，则应停止脚本执行。

标签： azure-data-factoryazure-powershellazure-data-factory-2

Answer 1

您可以尝试以下脚本：在云 CLI 中运行或使用安装在本地计算机上的 Azure Powershell 模块。

这等待用户输入

$rg = Read-Host "Enter ResourceGroupName"
$df = Read-Host "Enter DataFactoryName"
$tgr = Read-Host "Enter TriggerName"
$TriggerRunStartedAfter = Read-Host "Enter TriggerRunStartedAfter eg: 2021-07-01"
$TriggerRunStartedBefore = Read-Host "Enter TriggerRunStartedBefore eg: 2021-07-04"
$TimeSpan = Read-Host "TimeLimit - enter in minutes"

$ts = New-TimeSpan -Minutes $TimeSpan

$triggerDetails = Get-AzureRmDataFactoryV2TriggerRun -ResourceGroupName $rg -DataFactoryName $df -TriggerName $tgr -TriggerRunStartedAfter $TriggerRunStartedAfter -TriggerRunStartedBefore $TriggerRunStartedBefore
$status = $triggerDetails.status

#Get last/latest pipelines in record

$startTime = $triggerDetails.TriggerRunTimestamp[-1]
$pipedict = $triggerDetails.TriggeredPipelines[-1]

write-host "Last run pipelines and IDs:" -foregroundcolor yellow -NoNewline; write-Host $pipedict -foregroundcolor white
$pipekey = $pipedict.keys

write-Host ""
write-Host "Pipeline names:" -foregroundcolor Yellow -NoNewline; write-Host $pipekey -foregroundcolor white

$userchoice = Read-Host "Type in a specific pipeline name or All to show details for each pipeline"

function GetStatus ($choice){


$pipeid = $pipedict[$choice]
write-Host "PipelineID:" -foregroundcolor Yellow -NoNewline; write-Host $pipeid -foregroundcolor white
$pipedetails = Get-AzureRmDataFactoryV2PipelineRun -ResourceGroupName $rg -DataFactoryName $df -PipelineRunId $pipeid
write-Host "Status:" -foregroundcolor Yellow -NoNewline; write-Host $pipedetails.Status -foregroundcolor white
$pstatus = $pipedetails.Status
$pipelinename = $pipedetails.PipelineName
$DurationInMins = $pipedetails.DurationInMs
$lup = $pipedetails.lastUpdated
$maxTime = $startTime + $ts


if($pstatus -eq 'InProgress'){

if($lup -ge $maxTime)
{

$exceeded = NEW-TIMESPAN –Start $lup –End $maxTime
Write-Host "Pipeline running past limit. Minutes exceeded: " -NoNewline; write-Host $exceeded -foregroundcolor Red -NoNewline; write-Host "Total time: " -NoNewline; write-Host $DurationInMins -foregroundcolor Red
}
else {
write-Host "Pipeline is under limits" -foregroundcolor Green
}
}
else {
write-Host "Pipeline not Active" -foregroundcolor Green
}
}

if($userchoice -eq "all")
{
write-Host "You selected all" -foregroundcolor green
foreach ($h in $pipekey){
write-Host ""
write-Host "for  " -NoNewline; write-Host $h -foregroundcolor red
GetStatus -choice $h
}

}
else{
write-Host "choice is:" $userchoice
GetStatus -choice $userchoice
}
 
function Stop ($cstopc){

$stopdetails = Stop-AzureRmDataFactoryV2PipelineRun -ResourceGroupName $rg -DataFactoryName $df -PipelineRunId $pipeid
write-Host $stopdetails
write-Host "Pipeline stopped"
$cstoptrigger = Read-Host "do you want to stop the trigger as well? Y or N"
if($cstoptrigger -eq "y"){
$stoptriggerdetails = Stop-AzureRmDataFactoryV2Trigger -ResourceGroupName $rg -DataFactoryName $df -TriggerName $tgr
write-Host $stoptriggerdetails
write-Host "Trigger stopped"
}
}
 if($exceeded){
 $cstop = Read-Host "Stop pipelines? Y or N or all"
if($cstop -eq "y"){
Stop -cstopc $cstop
}
 }
 else{
write-Host ""
write-Host "bye :)"
 }

或键入所有细节并让脚本完成工作（替换上述脚本开头的这一位代替 Read-Host 行）

例子：

#Setresource details
$rg = "v-kabhyr"
$df = "ktestdf"
$tgr = "trigger1"
$TriggerRunStartedAfter = "2021-07-01"
$TriggerRunStartedBefore = "2021-07-04"
$TimeSpan = 5

命令详情：

获取所有数据工厂

PS /home/karthik> Get-AzureRmDataFactoryV2 -ResourceGroupName "v-kabhyr" -Name "ktestdf"

Invoke-AzureRmDataFactoryV2Pipeline命令在指定管道上启动运行并返回该运行的 ID。

PS /home/karthik> Invoke-AzureRmDataFactoryV2Pipeline -ResourceGroupName "v-kabhyr" -DataFactoryName "ktestdf" -PipelineName "pipeline2"
a6219afe-766d-43a6-9b40-f3f3990196db

可以将此 GUID 传递给Get-AzureRmDataFactoryV2PipelineRun或Get-AzureRmDataFactoryV2ActivityRun以获取有关此运行的更多详细信息。

PS /home/karthik> Get-AzureRmDataFactoryV2PipelineRun -ResourceGroupName "v-kabhyr" -DataFactoryName "ktestdf" -PipelineRunId "a6219afe-766d-43a6-9b40-f3f3990196db"

Get-AzureRmDataFactoryV2PipelineRun 命令返回有关指定管道运行的信息。 如果指定了 PipelineRunId，它将显示具有该 ID 的运行的详细信息。如果未指定 PipelineRunId，则它显示有关在 LastUpdatedAfter 和 LastUpdatedBefore 的值之间发生的指定管道的所有运行的信息。

Start-AzureRmDataFactoryV2Trigger cmdlet 在数据工厂中启动触发器。如果触发器处于“已停止”状态，则 cmdlet 将启动触发器并最终根据其定义调用管道。如果触发器已处于“已启动”状态，则此 cmdlet 无效。如果指定了 Force 参数，则 cmdlet 在启动触发器之前不会提示。

Get-AzureRmDataFactoryV2TriggerRun命令返回有关在给定时间范围内指定触发器的触发器运行的详细信息。

PS /home/karthik> Get-AzureRmDataFactoryV2TriggerRun -ResourceGroupName "v-kabhyr" -DataFactoryName "ktestdf" -TriggerName "trigger1" -TriggerRunStartedAfter "2021-07-01" -TriggerRunStartedBefore "2021-07-04"

Stop-AzureRmDataFactoryV2PipelineRun cmdlet 停止在使用 pieline 运行 ID 指定的数据工厂中运行的管道。

Stop-AzureRmDataFactoryV2PipelineRun -ResourceGroupName "ADF" -DataFactoryName "WikiADF" -PipelineRunId b9730a13-aa12-4926-a8b3-8e3a974ab0bd

Confirm Are you sure you want to stop pipeline run 'b9730a13-aa12-4926-a8b3-8e3a974ab0bd' in data factory 'WikiADF'? [Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): y

true

Stop-AzureRmDataFactoryV2Trigger cmdlet 停止数据工厂中的触发器。如果触发器处于“已启动”状态，则 cmdlet 将停止触发器并且不再调用管道。如果触发器已处于“已停止”状态，则此 cmdlet 无效。如果指定了 Force 参数，则 cmdlet 在停止触发器之前不会提示。

Stop-AzureRmDataFactoryV2Trigger -ResourceGroupName "ADF" -DataFactoryName "WikiADF" -TriggerName "ScheduledTrigger"

Confirm
Are you sure you want to stop trigger 'ScheduledTrigger' in data factory 'TestFactory'?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): y
True