More Common Variables That Aren't Common
In a recent article, I began exploring a few common parameters. These are parameters that every cmdlet includes, although I suspect their usage is anything but common. This is unfortunate because incorporating these parameters can enable very creative pipelined expressions or add depth to your PowerShell scripts. In the previous article in this series, I demonstrated how to use the common OutVariable parameter. While this can be useful, the parameter has limitations. Fortunately, some of those limitations can be mitigated using another common parameter, PipelineVariable. This parameter can be tricky to wrap your head around so let’s dig into it.
Pipeline Execution
The OutVariable and PipelineVariable parameters are designed based on pipeline execution. As I demonstrated in the last article, the variable you specify with the OutVariable parameter isn’t finalized until the associated command completes. This is a critical point. Except for a few exceptions, when you run a pipelined expression, PowerShell isn’t waiting for each command to complete.
CommandA -outvariable a | CommandB -ob b | CommandC | CommandD CommandA immediately begins piping to CommandB, which immediately begins piping to CommandC. In other words, the first three commands could be running simultaneously, passing objects down the pipeline. This is a topic I’ll have to dive into further in a future article.PS
This means that $a isn’t finalized until CommandA completes. The implication is that you can’t expect to use the variable later in the same pipelined expression. Here’s an example that tries to do just that but fails to give the desired result.
PS C:\> Get-CimInstance -ClassName Win32_PhysicalMemory -ComputerName dom1, srv1, srv2 -OutVariable v |
Measure-Object -Property capacity -Sum |
Select-Object @{Name = "Computername"; Expression = { $v.pscomputername.toupper() } },
@{Name = "MemoryGB"; Expression = { $_.sum / 1GB } }
Computername MemoryGB
------------ --------
{DOM1, SRV1, SRV2} 6The variable $V contains the results of the Get-CimInstance command. This command is trying to use the PSComputername property of each object in $V.
The command is structured this way because the Win32_PhysicalMemory class will return an instance for each memory bank, and a computer may have more than one. Of course, there are other ways to structure the command, including breaking it up into several steps instead of a single pipelined expression.
PipelineVariable
This leads us to the PipelineVariable parameter, which has an alias of pv. You set this parameter per command, and you can use the variable later in your pipeline expression while it is running.
However, you can’t simply replace one common parameter with the other.
PS C:\> Get-CimInstance -ClassName Win32_PhysicalMemory -ComputerName dom1, srv1, srv2 -PipelineVariable v |
Measure-Object -Property capacity -Sum |
Select-Object @{Name = "Computername"; Expression = { $v.pscomputername.toupper() } },
@{Name = "MemoryGB"; Expression = { $_.sum / 1GB } }
Computername MemoryGB
------------ --------
SRV2 6When using the PipelineVariable parameter, you typically need to structure your expression to use ForEach-Object.
PS C:\> Get-CimInstance -ClassName Win32_PhysicalMemory -ComputerName dom1, srv1, srv2 -PipelineVariable v |
ForEach-Object {
$_ | Measure-Object -Property capacity -Sum |
Select-Object @{Name = "Computername"; Expression = { $v.pscomputername.toupper() } },
@{Name = "MemoryGB"; Expression = { $_.sum / 1GB } }
}
Computername MemoryGB
------------ --------
DOM1 2
SRV1 2
SRV2 2This is an excellent example of the importance of visualization and understanding what is happening in the pipeline. Each CimInstance is getting piped to ForEach-Object which processes each instance separately. The $_ getting piped to Measure-Object is the CimInstance. At the same time, each CimInstance is also getting saved to the pipeline variable $v.
The Measure-Object command, which knows nothing about the CimInstance, is piped to Select-Object. Here I can use the pipeline variable and reference the PSComputername property. Remember, $v is each CimInstance as it is processed in the pipeline. The $_ in the MemoryGB expression is the measurement object.
I have to point out that this example is a bit artificial as each computer only has a single memory bank. In fact, I expect you’ll often find you won’t need to use this common parameter. Here’s a revised version of my code with my laptop, which has two memory banks.
PS C:\> Get-CimInstance -ClassName Win32_PhysicalMemory -ComputerName dom1, srv1, srv2, $env:computername | Group-Object -property PSComputername | Select-Object @{Name = "Computername"; Expression = { $_.name.toUpper()}},@{Name = "MemoryGB"; Expression = { ($_.group | Measure-Object -Property capacity -Sum).sum / 1GB } },
@{Name= "MemoryBanks";Expression = {$_.count}}
Computername MemoryGB MemoryBanks
------------ -------- -----------
DOM1 2 1
SRV1 2 1
SRV2 2 1
THINKX1-JH 32 2Combining Common Parameters
That isn’t to say there isn’t a use case. Here’s code that you might have in a script to report the size and number of files in a top-level folder.
$path = 'c:\work'
Get-ChildItem -Path $Path -Directory -OutVariable d |
ForEach-Object -Begin { $data = @() } -Process {
Write-Host "Processing $($d.fullname)" -ForegroundColor green
$data += Get-ChildItem -Path $d.fullname -File -Recurse | Measure-Object -Property length -Sum -ov m |
Select-Object @{Name = "Path"; Expression = { $d.Fullname } },
@{Name = "LastModified"; Expression = { $d.lastWriteTime } },
@{Name = "Name"; Expression = { $d.Name } },
@{Name = "Files"; Expression = { $m.count } },
@{Name = "SizeKB"; Expression = { [math]::Round($m.sum / 1kb, 2) } }
} -End {
Write-Host "Processed $($d.count) top level folders in $Path" -ForegroundColor Cyan
$data
}Read through the code and try to visualize what it is doing. Notice that I am storing the top-level directories using OutVariable. Measure-Object is storing its results in variable $m. But here’s what happens.
I am using Write-Host to illustrate how the value of $d is changing. Now, look at this version which uses a combination of OutVariable and PipelineVariable.
$path = 'c:\work'
Get-ChildItem -Path $Path -Directory -PipelineVariable d -OutVariable dirs | ForEach-Object -Begin { $data = @() } -Process {
Write-Host "Processing $($d.fullname)" -ForegroundColor green
$data += Get-ChildItem -Path $d.fullname -File -Recurse | Measure-Object -Property length -Sum -pv m |
Select-Object @{Name = "Path"; Expression = { $d.Fullname } },
@{Name = "LastModified"; Expression = { $d.lastWriteTime } },
@{Name = "Name"; Expression = { $d.Name } },
@{Name = "Files"; Expression = { $m.count } },
@{Name = "SizeKB"; Expression = { [math]::Round($m.sum / 1kb, 2) } }
} -End {
Write-Host "Processed $($dirs.count) top level folders in $Path" -ForegroundColor Cyan
$data
}The Get-ChildItem command is getting the top-level directories. When the command finishes, all directory objects will be stored in $dirs using OutVariable. Each directory object is also stored in $d using the PipelineVariable common parameter. This variable can be used later in the pipeline. I’m also storing the results of Measure-Object in a pipeline variable, $m. I can use these variables in the Select-Object expression to define custom properties using values derived from earlier in the pipeline expression. Remember, all the code starting with Get-Childitem is a single pipelined expression.
There is one final distinction between these variables. The $dirs variable defined with OutVariable persists after the pipeline expression completes.
PS C:\> > $dirs
Directory: C:\work
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 3/15/2022 7:23 PM DSCModule
d---- 4/24/2022 4:47 PM foobar
d---- 4/29/2022 3:22 PM ironscripter
d---- 7/7/2022 1:30 PM samples
d---- 7/8/2022 12:18 PM stuffBut the pipeline variable does not.
PS C:\> $d
PS C:\>Pipeline Trade-Offs
However, just because I can write this as a monolithic pipeline expression doesn’t mean I should. When I run this code against my scripts directory, it takes over 3 seconds to process 182 top-level folders. Or I could break it down into several separate steps that don’t require using the common parameters.
$path = 'c:\scripts'
$dirs = Get-ChildItem -Path $Path -Directory
foreach ($d in $dirs) {
#Write-Host "Processing $($d.fullname)" -ForegroundColor green
$m = Get-ChildItem -Path $d.fullname -File -Recurse | Measure-Object -Property length -Sum
[pscustomobject]@{
Path = $d.fullname
Name = $d.Name
LastModified = $d.lastWriteTime
Files = $m.count
SizeKB = [math]::Round($m.sum / 1kb, 2)
}
} #foreach $d
Write-Host "Processed $($dirs.count) top level folders in $Path" -ForegroundColor CyanThis shaved a couple of hundred milliseconds off with the bonus that I think this code is easier to read and troubleshoot.
Summary
There is a need for OutVariable and PipelineVariable with the latter being more of a special use case. Depending on how you structure your PowerShell expression, you might not even need them. But I wanted to ensure you knew about these parameters so you could pull them out when necessary.