Powershell:从大文件获取内容(服务器列表)
Powershell:从大文件获取内容(服务器列表)
提问于 2016-08-12 09:51:15
我有100,000个来自文本文件(serverlist.txt)的服务器列表
当我在一个镜头中运行，它将打破我的内存和cpu和时间花更长的时间(约3天)完成扫描的DNSlookup。
我试图拆分包含以下20k服务器列表的文件，并且可以完成对每个文件的10分钟扫描。
serverlist1.txt
serverlist2.txt
serverlist3.txt
serverlist4.txt
serverlist5.txt$objContainer = @()
$values = @()
$domains = Get-Content -path "serverlist1.txt"
$named = 0
$timestamp= get-date
$domains | ForEach-Object {
$domain = $_
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
$obj = New-Object -TypeName PSObject
#$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
$objContainer += $obj
$values = @()
$named = 0
}
Write-Output $objContainer
$objContainer | Export-csv "dnslog_$((Get-Date).ToString('MM-dd-yyyy_hh-mm-ss')).csv" -NoTypeInformation我的问题是，如何在生成dnslog(datetime).csv后立即执行并循环文本文件中的输入
例如：
- 运行powershell脚本.\filename.ps1
- 从serverlist1.txt输入
- 输出dnslog(日期时间).csv
- 从serverlist2.txt输入
- 输出dnslog(日期时间).csv
- 从serverlist3.txt输入
- 输出dnslog(日期时间).csv
- 从serverlist4.txt输入
- 输出dnslog(日期时间).csv
- 从serverlist5.txt输入
- 输出dnslog(日期时间).csv
完成！
如果我有超过5个文本文件列表，它将继续循环从输入文件，直到完成。
回答 2
Stack Overflow用户
回答已采纳
发布于 2016-08-12 13:36:55
在Chris的回答中，我还会在Get内容中添加一个ReadCount标志，如下所示：
Get-Content -path "serverlist1.txt" -ReadCount 1 | % {这样就省去了把整个文件读入内存的麻烦。
Stack Overflow用户
发布于 2016-08-12 10:05:33
您应该考虑运行这个并行批处理作业。你试过这样做了吗？
您可以通过删除对内存的所有提交(变量赋值和用+=重写数组)来处理内存中断问题。
$timestamp = get-date
Get-Content -path "serverlist1.txt" | ForEach-Object {
$domain = $_
# You can clear this here.
$values = @()
$named = 0
# There are potentially better options than nslookup.
# Needs a bit of care to understand what's an alias here though.
# [System.Net.Dns]::GetHostEntry($domain)
# And if you don't like that, quite a few of us have written equivalent tools in PowerShell.
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
# Leave the output object in the output pipeline
# If you're running PowerShell 3 or better:
[PSCustomObject]@{
Domain = $domain
Name = $values[0]
'IP Address' = $values[1]
Alias = $values[2]
TimeStamp = $timestamp
}
# PowerShell 2 is less flexible. This or Select-Object.
#$obj = New-Object -TypeName PSObject
##$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
#$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
#$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
#$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
#$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
# To leave this in the output pipeline, uncomment this
# $obj
# No version of PowerShell needs you to do this. It's a good way to ramp up memory usage
# for large data sets.
# $objContainer += $obj
} | Export-Csv "dnslog_$(Get-Date -Format 'MM-dd-yyyy_hh-mm-ss').csv" -NoTypeInformation页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/38914999
复制相关文章
相似问题
腾讯云开发者
