Going Out on a Limb on the Abstract Syntax Tree
I thought I'd start out the new year by revisiting a topic I haven't covered in quite awhile. This is a topic that definitely fits the theme of this newsletter which is to dive deep into PowerShell topics and get a peek under the hood. Today I want to start with a short introduction to the Abstract Syntax Tree, also referred to as the AST.
PowerShell, and related tools, can use the AST to analyze a PowerShell command or script file and break it down to its individual components. This is a powerful tool that can be used to analyze PowerShell code. You can use the AST to see how PowerShell interprets your code.
Parsing with the AST
The AST is a .NET class referenced as [System.Management.Automation.Language.Parser]. I can use my Get-TypeMember function from the PSScriptTools module to explore the class.
PS C:\> Get-TypeMember System.Management.Automation.Language.Parser
Type: System.Management.Automation.Language.Parser
Name MemberType ResultType IsStatic IsEnum
---- ---------- ---------- -------- ------
GetType Method Type
ParseFile Method ScriptBlockAst True
ParseInput Method ScriptBlockAst True
To demonstrate, I'm going to parse this demo PowerShell script file.
#requires -version 5.1
#requires -RunAsAdministrator
<#
Sample-Script.ps1
Get event log data from a specified computer.
This is a sample script intended to demonstrate
a variety of PowerShell scripting techniques
and concepts. This script is for educational
purposes and not intended for production use.
#>
Param(
[string]$Computername = $Env:COMPUTERNAME,
[PSCredential]$Credential
)
# define a list of event logs to query with Get-WinEvent
$logs = 'System', 'Application', 'Windows PowerShell'
# create a hashtable for log results
$logHash = [ordered]@{
Computername = $Computername
Date = Get-Date
}
# define the filter hashtable for Get-WinEvent
$filter = @{
LogName = $null
Level = 2, 3
}
Write-Host "Querying logs on $Computername" -ForegroundColor Cyan
foreach ($log in $logs) {
<#
I am splatting any bound PSParameters
to the Get-WinEvent cmdlet. Otherwise,
Get-WinEvent will use default values.
#>
$filter.LogName = $log
$entries = Get-WinEvent @PSBoundParameters -FilterHashtable $filter
Write-Host "Found $($entries.Count) entries in $log" -ForegroundColor Cyan
$logHash.Add($log, $entries)
} #close foreach
#write results to the pipeline as a custom object
[PSCustomObject]$logHash
I'll let you run the script to see what it does. The script works, which is the important part.
To parse the file, I will need to pass a few parameters.
PS C:\> [System.Management.Automation.Language.Parser]::ParseFile.OverloadDefinitions
static System.Management.Automation.Language.ScriptBlockAst ParseFile(string fileName, [ref] System.Management.Automation.Language.Token[] tokens, [ref] System.Management.Automation.Language.ParseError[] errors)
This critical step here is recognizing that some of the parameters are passed by reference ([ref]). This means I need to create an empty array to hold the results. I need a variable for tokens and a variable for errors. Because these are defined a [ref] type, you need to define them before you can use them.
New-Variable astTokens -Force
New-Variable astErr -Force
I might as well define the path to the script file as well.
$path = 'c:\scripts\sample-script.ps1'
Now, I can parse the file.
$AST = [System.Management.Automation.Language.Parser]::ParseFile($Path, [ref]$astTokens, [ref]$astErr)
If there were any errors, they would be stored in the $astErr variable. I'll come back to $astTokens in a little bit.
The AST Object
The value of $AST is a ScriptBlockAst object. This is a rich object.
PS C:\> Get-TypeMember System.Management.Automation.Language.ScriptBlockAst
Type: System.Management.Automation.Language.ScriptBlockAst
Name MemberType ResultType IsStatic IsEnum
---- ---------- ---------- -------- ------
Find Method Ast
FindAll Method IEnumerable`1
GetHelpContent Method CommentHelpInfo
GetScriptBlock Method ScriptBlock
GetType Method Type
SafeGetValue Method Object
Visit Method Object
Visit Method Void
Attributes Property ReadOnlyCollection`1
BeginBlock Property NamedBlockAst
CleanBlock Property NamedBlockAst
DynamicParamBlock Property NamedBlockAst
EndBlock Property NamedBlockAst
Extent Property IScriptExtent
ParamBlock Property ParamBlockAst
Parent Property Ast
ProcessBlock Property NamedBlockAst
ScriptRequirements Property ScriptRequirements
UsingStatements Property ReadOnlyCollection`1
I'm not going to cover all of these members today. Many of them aren't applicable unless processing an advanced PowerShell function. I'll save that for another day.
Requirements
One property that you might find of interest is ScriptRequirements. This property returns a ScriptRequirements object.
PS C:\> $ast.ScriptRequirements
RequiredApplicationId :
RequiredPSVersion : 5.1
RequiredPSEditions : {}
RequiredModules : {}
RequiredAssemblies : {}
IsElevationRequired : True
This shows the script requirements as defined by the #requires statements in the script. This is a handy way to programmatically determine what a script needs to run. In my example you can see the required PowerShell version, and the fact that the script must run as an administrator.
It might be fun to create a requirements object for the script file.
#create a hash table with default values
$hash = [ordered]@{
Path = $Path
RequiredVersion = $Null
RequiresAdmin = $False
IsCore = $False
LastWriteTime = (Get-Item $path).LastWriteTime
}
If ($AST.ScriptRequirements) {
$hash.RequiredVersion = $AST.ScriptRequirements.RequiredPSVersion
$hash.RequiresAdmin = $AST.ScriptRequirements.IsElevationRequired
if ($ast.ScriptRequirements.RequiredPSVersion -gt '5.1') {
$hash.IsCore = $True
}
}
It is trivial to create an object from the hash table.
PS C:\> [PSCustomObject]$hash
Path : c:\scripts\sample-script.ps1
RequiredVersion : 5.1
RequiresAdmin : True
IsCore : False
LastWriteTime : 1/6/2025 2:54:40 PM
I don't know about you, but I have a directory full of PowerShell script files and having this kind of information could be very handy. I took my code snippet and wrote a PowerShell function around it.
#requires -version 5.1
Function Get-RequiredMetadata {
[cmdletbinding()]
[OutputType('ScriptRequirementMetadata')]
Param(
[Parameter(
Position = 0,
Mandatory,
ValueFromPipeline,
HelpMessage = 'The path to the .PS1 script file'
)]
[ValidatePattern('.*\.ps1$')]
[ValidateScript({ Test-Path $_ })]
[string]$Path
)
Begin {
Write-Verbose "Starting $($MyInvocation.MyCommand)"
New-Variable astTokens -Force
New-Variable astErr -Force
$progSplat = @{
Activity = $MyInvocation.MyCommand
Status = 'Starting'
}
Write-Progress @progSplat
}
Process {
Write-Verbose "Processing $Path"
$progSplat.Status = "Processing $Path"
Write-Progress @progSplat
$AST = [System.Management.Automation.Language.Parser]::ParseFile($Path, [ref]$astTokens, [ref]$astErr)
$hash = [ordered]@{
PSTypeName = 'ScriptRequirementMetadata'
Path = $Path
RequiredVersion = $Null
RequiresAdmin = $False
IsCore = $False
LastWriteTime = (Get-Item $Path).LastWriteTime
}
If ($AST.ScriptRequirements) {
#save requiredVersion as a string if found
if ($AST.ScriptRequirements.RequiredPSVersion -match '\d') {
$hash.RequiredVersion = $AST.ScriptRequirements.RequiredPSVersion -as [string]
}
$hash.RequiresAdmin = $AST.ScriptRequirements.IsElevationRequired
if ($ast.ScriptRequirements.RequiredPSVersion -gt '5.1') {
$hash.IsCore = $True
}
}
#write a custom object to the pipeline
[PSCustomObject]$hash
}
End {
$progSplat['Completed'] = $True
Write-Progress @progSplat
Write-Verbose "Ending $($MyInvocation.MyCommand)"
}
}
The function uses Write-Progress to provide some feedback. I also added a PSTypeName property to the hash table so that the custom object will have a type name. This is a good practice when creating custom objects because it gives you the option of defining a custom format file.
I'll use this function to process the root of my script directory.
$r = dir c:\scripts\*.ps1 | Get-RequiredMetadata
This took 15.6 seconds to process 3400 files.
PS C:\> $r[0]
Path : C:\scripts\12Days.ps1
RequiredVersion :
RequiresAdmin : False
IsCore : False
LastWriteTime : 8/28/2020 5:00:51 PM
This data let's me discover old files that I might want to update, or even delete.
PS C:\> $r | Group-Object RequiredVersion -NoElement
Count Name
----- ----
2030
1 1.0
399 2.0
337 3.0
207 4.0
90 5.0
1 5.0.9814.0
257 5.1
3 6.0
1 6.1.2
1 6.2
19 7.0
9 7.1
12 7.2
11 7.3
26 7.4
The function saves the required version as a string, which makes it easier to store in a CSV or JSON file.
$r | Export-Csv c:\scripts\scriptmetadata.csv
$r | ConvertTo-Json -compress | Out-File c:\scripts\scriptmetadata.json -Encoding utf8
The JSON file is compressed, but still about twice the size of the CSV file.
PS C:\> dir c:\scripts\scriptmetadata*
Directory: C:\scripts
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1/6/2025 3:41 PM 259399 scriptmetadata.csv
-a--- 1/6/2025 3:44 PM 509118 scriptmetadata.json
But the data can be re-imported just as easily.
PS C:\> $n = Import-Csv C:\scripts\scriptmetadata.csv | Select-Object -first 5
PS C:\> $n | Format-Table
Path RequiredVersion RequiresAdmin IsCore LastWriteTime
---- --------------- ------------- ------ -------------
C:\scripts\12Days.ps1 False False 8/28/2020 5:00:51 PM
C:\scripts\13Expressions.ps1 3.0 False False 6/2/2020 10:00:29 AM
C:\scripts\13More.ps1 4.0 False False 2/13/2015 12:40:52 PM
C:\scripts\13ScriptBlocks-v2.ps1 2.0 False False 7/31/2013 4:47:28 PM
C:\scripts\13ScriptBlocks.ps1 2.0 False False 4/13/2012 9:27:21 AM
Remember that all the CSV imported properties will be treated as strings. I can get the same results importing the JSON data.
$n = Get-Content C:\scripts\scriptmetadata.json | ConvertFrom-Json | Select-Object -First 5
Even the JSON file is larger, it is a little simpler to bring back into PowerShell with minimal effort to preserve type. Although I would probably build an import function based on this code:
$n = Get-Content C:\scripts\scriptmetadata.json | ConvertFrom-Json |
Select-Object -First 5 |
ForEach-Object {
Write-Host $_.path -fore yellow
#create a structured custom object from an ordered hash table
$obj = [ordered]@{
PSTypeName = 'ScriptRequirementMetadata'
}
$obj.Path = $_.Path
$obj.RequiredVersion = $_.RequiredVersion -As [Version]
$obj.RequiresAdmin = $_.RequiresAdmin
$obj.IsCore = $_.IsCore
$obj.LastWriteTime = $_.LastWriteTime
#convert the hashtable into an object
[PSCustomObject]$obj
}
This lets me restore the typename and convert the version property to a [Version] object.
Summary
All of this was made possible by using the AST to parse a PowerShell script file. This is a powerful tool that can be used to analyze PowerShell code. I hope this has given you some ideas on how you might use the AST in your own scripts, and just wait until I show you the AST tokens. Next time.