Climbing Higher in the Abstract Syntax Tree
We've started an exploration of using the Abstract Syntax Tree` (AST) to analyze PowerShell code. You can use the AST to dissect PowerShell code to its fundamental elements. You might use this to build troubleshooting or debugging tools. I like to use the AST to build tools to help me manage my code. I showed you an example last time and there will be more before we are through.
In the previous article, we looked in general at the AST object. I'm going to continue with the sample script I showed last time. Let's get the AST again.
$Path = 'c:\scripts\sample-script.ps1'
New-Variable astTokens -Force
New-Variable astErr -Force
$AST = [System.Management.Automation.Language.Parser]::ParseFile($Path, [ref]$astTokens, [ref]$astErr)
Tokens
The astTokens variable is a collection of PowerShell scripting elements. These are the building blocks that comprise your PowerShell code. My simple sample script can be broken down to 131 tokens.
PS C:\> $astTokens.count
131
There are different token types stored in $astTokens.
PS C:\> $astTokens | Get-Member | Select-Object TypeName -Unique
TypeName
--------
System.Management.Automation.Language.Token
System.Management.Automation.Language.VariableToken
System.Management.Automation.Language.StringLiteralToken
System.Management.Automation.Language.NumberToken
System.Management.Automation.Language.StringExpandableToken
System.Management.Automation.Language.ParameterToken
You can use Get-TypeMember from the PSScriptTools module to view the properties of each token type.
PS C:\> Get-TypeMember System.Management.Automation.Language.Token
Type: System.Management.Automation.Language.Token
Name MemberType ResultType IsStatic IsEnum
---- ---------- ---------- -------- ------
GetType Method Type
Extent Property IScriptExtent
HasError Property Boolean
Kind Property TokenKind True
Text Property String
TokenFlags Property TokenFlags True
PS C:\> Get-TypeMember System.Management.Automation.Language.ParameterToken
Type: System.Management.Automation.Language.ParameterToken
Name MemberType ResultType IsStatic IsEnum
---- ---------- ---------- -------- ------
GetType Method Type
Extent Property IScriptExtent
HasError Property Boolean
Kind Property TokenKind True
ParameterName Property String
Text Property String
TokenFlags Property TokenFlags True
UsedColon Property Boolean
We're going to dive into this in a moment, but this is how the token types derived from my code.
PS C:\> $astTokens | Group-Object {$_.GetType().name}
Count Name Group
----- ---- -----
2 NumberToken {2, 3}
3 ParameterToken {-ForegroundColor, -FilterHashtable, -ForegroundColor}
2 StringExpandableToken {"Querying logs on $Computername", "Found $($entries.Count) entries in $log"}
7 StringLiteralToken {'System', 'Application', 'Windows PowerShell', Get-Date…}
98 Token {#requires -version 5.1, …
19 VariableToken {$Computername, $Env:COMPUTERNAME, $Credential, $logs…}
Each token type has different properties so you can't easily display all of $astTokens in a single command. Although, you could use a grouped hash table.
PS C:\> $h = $astTokens | Group-Object {$_.GetType().name} -AsHashTable
PS C:\> $h.NumberToken | Format-Table
Value Text TokenFlags Kind HasError Extent
----- ---- ---------- ---- -------- ------
2 2 None Number False 2
3 3 None Number False 3
PS C:\> $h.StringLiteralToken
Value : System
Text : 'System'
TokenFlags : ParseModeInvariant
Kind : StringLiteral
HasError : False
Extent : 'System'
Value : Application
Text : 'Application'
TokenFlags : ParseModeInvariant
Kind : StringLiteral
HasError : False
Extent : 'Application'
Value : Windows PowerShell
Text : 'Windows PowerShell'
TokenFlags : ParseModeInvariant
Kind : StringLiteral
HasError : False
Extent : 'Windows PowerShell'
Value : Get-Date
Text : Get-Date
TokenFlags : CommandName
Kind : Generic
HasError : False
Extent : Get-Date
Value : Write-Host
Text : Write-Host
TokenFlags : CommandName
Kind : Generic
HasError : False
Extent : Write-Host
Value : Get-WinEvent
Text : Get-WinEvent
TokenFlags : CommandName
Kind : Generic
HasError : False
Extent : Get-WinEvent
Value : Write-Host
Text : Write-Host
TokenFlags : CommandName
Kind : Generic
HasError : False
Extent : Write-Host