powershell - Get Unique records from large files quickly

Question

I have big files (at least 20 MB each) where i need to look for string M(\d{10})

Below is the script I am using:

Get-Content -Path Test.log | %{ [Regex]::Matches($_, "M(\d{10})") } | %{ $_.Value } | select -Unique

This is taking good time and more CPU, please suggest how to get the results with lower CPU usage/quicker.

标签： powershell

Answer 1

只需测量自己（以尽量减少缓存效果差异，第一个重复）：

Measure-Command {Get-Content -Path Test.log | %{ [Regex]::Matches($_, "M(\d{10})") } | %{ $_.Value } | select -Unique}

Measure-Command {Get-Content -Path Test.log | %{ [Regex]::Matches($_, "M(\d{10})") } | %{ $_.Value } | select -Unique}

Measure-Command {sls -Path Test.log  "M(\d{10})"  | %{ $_.Matches.Groups[1].Value } | select -Unique}