PowerShell Pipeline Processing
In this issue:
Overview
In this issue we'll dive into how PowerShell processes a pipeline expression. Understanding this process is critical for writing effective and efficient PowerShell expressions, especially where the pipeline might be blocked or truncated.
Introduction
Perhaps the most compelling PowerShell feature, and one that is used often in the pipeline. The concept of a pipeline isn't unique to PowerShell. However, passing an object from one command to another is another story. Instead of parsing text, we often use cmdlet parameters which are often designed with object properties in mind.
The tricky part of the pipeline is that you can't observe it directly. But understanding how it works is critical to building an effective pipeline expression. Or trying to troubleshoot a problem. I thought we should dig into this and take a closer look at how objects move through the pipeline.
Processing the Pipeline
I expect many of you already know this, but PowerShell cmdlets are divided into three phases or steps. You can write an advanced PowerShell function using [cmdletbinding()] and manually adding named script blocks.
Begin
The first phase or script block is Begin. If your command is designed to accept pipeline input, any code in this section executes once before any pipeline processing. This is where you would put setup and initialization code. Be careful of referencing parameters that might get their value via the pipeline in this section.
Process
The second phase or script block is Process. Code in this section runs once for each object in the pipeline. You will typically reference parameter variables that accept pipeline input in this script block.
End
After everything has been processed from the pipeline, code in the third phase is executed once. This is the End script block.
Sample Function
Here's a sample function that illustrates these concepts.
function Invoke-StepA {
[cmdletbinding()]
[OutputType('int32')]
param(
[Parameter(
Position = 0,
ValueFromPipeline
)]
[ValidateNotNullOrEmpty()]
[int32[]]$InputObject
)
begin {
Write-Verbose "[$((Get-Date).TimeOfDay) BEGIN $($MyInvocation.MyCommand)] Starting $($MyInvocation.MyCommand)"
Write-Verbose "[$((Get-Date).TimeOfDay) BEGIN $($MyInvocation.MyCommand)] Pipeline position $($MyInvocation.PipelinePosition)"
Write-Verbose "[$((Get-Date).TimeOfDay) BEGIN $($MyInvocation.MyCommand)] Running under PowerShell version $($PSVersionTable.PSVersion)"
} #begin
process {
Write-Verbose "[$((Get-Date).TimeOfDay) PROCESS $($MyInvocation.MyCommand)] Processing..."
#$InputObject can accept pipeline input or be passed as an array
foreach ($item in $InputObject) {
$item * 2
}
} #process
end {
Write-Verbose "[$((Get-Date).TimeOfDay) END $($MyInvocation.MyCommand)] Ending $($MyInvocation.MyCommand)"
} #end
} #close Invoke-StepA
The Begin and End blocks are optional, although if you don't intend to use them I recommend using code like this:
Begin {
#this is intentionally empty
}
This way you don't have to wonder if something was accidentally deleted.
PS C:\> $r = 1..5 | Invoke-StepA -Verbose
VERBOSE: [14:56:31.5476616 BEGIN Invoke-StepA] Starting Invoke-StepA
VERBOSE: [14:56:31.5479350 BEGIN Invoke-StepA] Pipeline position 1
VERBOSE: [14:56:31.5483902 BEGIN Invoke-StepA] Running under PowerShell version 7.6.9
VERBOSE: [14:56:31.5487267 PROCESS Invoke-StepA] Processing...
VERBOSE: [14:56:31.5490977 PROCESS Invoke-StepA] Processing...
VERBOSE: [14:56:31.5493928 PROCESS Invoke-StepA] Processing...
VERBOSE: [14:56:31.5496220 PROCESS Invoke-StepA] Processing...
VERBOSE: [14:56:31.5499459 PROCESS Invoke-StepA] Processing...
VERBOSE: [14:56:31.5502366 END Invoke-StepA] Ending Invoke-StepA
The verbose messaging clearly demonstrates the three phases, although I get slightly different results when passing parameter values directly.
PS C:\> $r = Invoke-StepA -Verbose -InputObject (1..5)
VERBOSE: [15:00:16.3654877 BEGIN Invoke-StepA] Starting Invoke-StepA
VERBOSE: [15:00:16.3659816 BEGIN Invoke-StepA] Pipeline position 1
VERBOSE: [15:00:16.3663559 BEGIN Invoke-StepA] Running under PowerShell version 7.6.9
VERBOSE: [15:00:16.3666912 PROCESS Invoke-StepA] Processing...
VERBOSE: [15:00:16.3671087 END Invoke-StepA] Ending Invoke-StepA
In this situation, the code in the Process script block is only executed once.
Multiple Commands
Next, let's combine functions in a pipelined expression. I'll be using these functions.