Serialization Situation

git

                March 26, 2024

            Serialization Situation

            One of PowerShell's greatest strengths, and one of its greatest challenges, is its object-oriented nature. Once you get your head wrapped around this concept, using PowerShell in the console or scripted automation begins to feel like magic. Where this becomes challenging is when you need to take output and save it to disk for future re-use. This process is know as serialization. In text-based situations this is easy.

git log | Out-File glog.txt

The output of the *git* command is text, so saving it to a file is simple.

But what about PowerShell?

PS C:\> Get-WinEvent -LogName System -MaxEvents 3 | Out-File systemlog.txt

The file gets created.

   ProviderName: Microsoft-Windows-Time-Service

TimeCreated                     Id LevelDisplayName Message

-----------                     -- ---------------- -------

3/25/2024 9:59:16 AM           158 Information      The time provider '?

   ProviderName: Netwtw10

TimeCreated                     Id LevelDisplayName Message

-----------                     -- ---------------- -------

3/25/2024 9:51:13 AM          7003 Information      7003 - Roam Complete

3/25/2024 9:47:04 AM          6062 Warning          6062 - Lso was trig?

But the saved output is not very useful. And a lot of detail has been lost.

Let's take a look at how to serialize PowerShell objects to disk in a way that preserves the object's structure and detail.

Get a premium subscription for full article and archive access

One of PowerShell's greatest strengths, and one of its greatest challenges, is its object-oriented nature. Once you get your head wrapped around this concept, using PowerShell in the console or scripted automation begins to feel like magic. Where this becomes challenging is when you need to take output and save it to disk for future re-use. This process is know as serialization. In text-based situations this is easy.
git log | Out-File glog.txt

The output of the git command is text, so saving it to a file is simple.
But what about PowerShell?
Get-WinEvent -LogName System -MaxEvents 3 | Out-File systemlog.txt

The file gets created.
   ProviderName: Microsoft-Windows-Time-Service

TimeCreated                     Id LevelDisplayName Message
-----------                     -- ---------------- -------
3/25/2024 9:59:16 AM           158 Information      The time provider '?

   ProviderName: Netwtw10

TimeCreated                     Id LevelDisplayName Message
-----------                     -- ---------------- -------
3/25/2024 9:51:13 AM          7003 Information      7003 - Roam Complete
3/25/2024 9:47:04 AM          6062 Warning          6062 - Lso was trig?

But the saved output is not very useful. And a lot of detail has been lost.
You could always do something like this:
Get-WinEvent -LogName System -MaxEvents 3 |
Select-Object -Property * |
Out-File systemlog-all.txt

But what can you really do with this?
Message              : The time provider 'VMICTimeProvider' has indicated that the current hardware and operating environment is not supported and has stopped. This behavior is expected for VMICTimeProvider on non-HyperV-guest environments. This may be the expected behavior for the current provider in the current operating environment as well.
Id                   : 158
Version              : 0
Qualifiers           :
Level                : 4
Task                 : 0
Opcode               : 0
Keywords             : -9223372036854775808
RecordId             : 135070
ProviderName         : Microsoft-Windows-Time-Service
ProviderId           : 06edcfeb-0fd0-4e53-acca-a6f8bbf81bcb
LogName              : System
ProcessId            : 57844
ThreadId             : 57564
MachineName          : Prospero
UserId               : S-1-5-19
...

You could bring this back into PowerShell with a complicated script using regular expressions and text parsing.  However, your time is valuable and you have better things to do.
Deserialization
The typical serialization format in PowerShell is XML. This is used in PowerShell remoting to return data from PSSession.
PS C:\> $r  Invoke-Command { Get-Service winrm } -Computername SRV1
PS C:\> $r | Get-Member | Select-Object TypeName -Unique

TypeName
--------
Deserialized.System.ServiceProcess.ServiceController

The Deserialized prefix indicates that the object is a deserialized object. This is a special type of object that is created when data is passed between sessions. The object is serialized into XML and then deserialized back into an PS custom object.
PS C:\>< $r.GetType().FullName
System.Management.Automation.PSObject

One very important distinction between a native object and its deserialized version is the lack of methods. Using Get-TypeMember I can see the native object's methods.
PS C:\> Get-TypeMember System.ServiceProcess.ServiceController -MemberType Method

   Type: System.ServiceProcess.ServiceController

Name                      MemberType ResultType          IsStatic IsEnum
----                      ---------- ----------          -------- ------
Close                     Method     Void
Continue                  Method     Void
Dispose                   Method     Void
Equals                    Method     Boolean
ExecuteCommand            Method     Void
GetDevices                Method     ServiceController[]     True
GetHashCode               Method     Int32
GetLifetimeService        Method     Object
GetServices               Method     ServiceController[]     True
GetType                   Method     Type
InitializeLifetimeService Method     Object
Pause                     Method     Void
Refresh                   Method     Void
Start                     Method     Void
Stop                      Method     Void
ToString                  Method     String
WaitForStatus             Method     Void

Compare this to the serialized object.
PS C:\> $r | Get-Member -MemberType Method

   TypeName: Deserialized.System.ServiceProcess.ServiceController

Name     MemberType Definition
----     ---------- ----------
GetType  Method     type GetType()
ToString Method     string ToString(), string ToString(string format, System.IFormatProvider formatProvider), string IFormattable.ToStrin…

Get-TypeMember can't be used with deserialized objects because it can't convert the type name to a valid type.

The methods you see are ones PowerShell adds to all objects. They are not related to the original, native object. However, PowerShell can easily construct an object with the original property names and value, which is probably what you want anyway.
cliXML
You can manually serialize an object to XML using the Export-Clixml cmdlet. This is the best method for saving objects to disk for later use.
Get-WinEvent -LogName System -MaxEvents 10 | Export-Clixml systemlog.xml

The output is technically an XML file.
 Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
   RefId="0">
     RefId="0">
      System.Diagnostics.Eventing.Reader.EventLogRecord
      System.Diagnostics.Eventing.Reader.EventRecord
      System.Object

    System.Diagnostics.Eventing.Reader.EventLogRecord

       N="Id">158
       N="Version">0
       N="Qualifiers" />
       N="Level">4
       N="Task">0
       N="Opcode">0
       N="Keywords">-9223372036854775808
       N="RecordId">135075
       N="ProviderName">Microsoft-Windows-Time-Service
       N="ProviderId">06edcfeb-0fd0-4e53-acca-a6f8bbf81bcb
...

But it is XML that is specific to PowerShell. You could use the file in a non-PowerShell XML application, but it would be difficult to work with. Rather, the XML file is structured to best re-create the original objects in PowerShell when imported.
$in = Import-clixml systemlog.xml

This is still a deserialized object
PS C:\> $in | Get-Member | Select-Object TypeName -Unique

TypeName
--------
Deserialized.System.Diagnostics.Eventing.Reader.EventLogRecord

But, you can use the output just as you would have with the original output.
PS C:\> $in | Group-Object -Property ProviderName -NoElement | Sort Count,Name | Select Name,Count

Name                             Count
----                             -----
Microsoft-Windows-Time-Service       2
Microsoft-Windows-Kernel-General     4
Netwtw10                             4

If you need to save output for later use in PowerShell, the cliXML cmdlets are the best way to go, especially if you want to save as much detail as possible.

You might want to take advantage of the NoClobber parameter of Export-Clixml to prevent overwriting an existing XML file.

You should always be thinking about how you intend to re-use the serialized data. PowerShell objects can be quite complex with nested object properties. You might need to use the -Depth parameter to control how deep the serialization goes. The default is 2 levels. But you might not need that much detail.
PS C:\> get-process -id $pid | export-clixml d:\temp\pid-1.xml -Depth 1

Or you might want more.
PS C:\> get-process -id $pid | export-clixml d:\temp\pid-3.xml -Depth 3

As you might expect, the deeper you go, the bigger the file becomes.
PS C:\> dir d:\temp\pid*.xml

    Directory: D:\temp

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         3/25/2024  11:24 AM          26018 pid-1.xml
-a----         3/25/2024  11:23 AM         441304 pid-2.xml
-a----         3/25/2024  11:24 AM        1051868 pid-3.xml

And this is for a single process!
Here's the result on importing.
PS C:\> (import-clixml D:\temp\pid-1.xml).mainmodule
System.Diagnostics.ProcessModule (powershell.exe)
PS C:\> (import-clixml D:\temp\pid-2.xml).mainmodule

   Size(K) ModuleName                  FileName
   ------- ----------                  --------
       440 powershell.exe              C:\Windows\System32\Windowspowershel...

If you don't need that much detail, you can improve performance and reduce the file size by reducing the depth.
There's no simple way to calculate the maximum depth you might need. You'll have to determine that on your own.
PS C:\> $here = Get-Process -id $pid
PS C:\> $here.mainmodule.FileVersionInfo.FileVersionRaw.Major
10
PS C:\> (import-clixml D:\temp\pid-3.xml).mainmodule.fileversioninfo.fileversionraw.major
10

Zip It
Because your XML files can get quite large, you might want to consider compressing them. We've looked at compressing files before using the .NET Framework. But I put together a simple function to create a serialized cliXML object and compress it to a zip file. The function will delete the original XML file.
Function Export-ClixmlArchive {
    [CmdletBinding(SupportsShouldProcess)]
    [alias("exmlzip")]
    [OutputType("None","System.IO.FileInfo")]
    Param (
        [Parameter(Position = 0, Mandatory)]
        [ValidatePattern('\.zip$')]
        [string]$Path,
        [int32]$Depth = 2,
        [Parameter(Mandatory, ValueFromPipeline)]
        [ValidateNotNullOrEmpty()]
        [Object]$InputObject,
        [switch]$NoClobber,
        [switch]$Passthru
    )

    Begin {
        Write-Verbose "[$((Get-Date).TimeOfDay) BEGIN  ] Starting $($MyInvocation.MyCommand)"
        $data = [System.Collections.Generic.List[Object]]::new()
        #define the xml file path
        $xmlPath = $Path -replace '\.zip$', '.xml'
        Write-Verbose "[$((Get-Date).TimeOfDay) BEGIN  ] XML file path: $xmlPath"
        $processing = $False
    } #begin
    Process {
        If (-Not $Processing) {
            #Display the progress message once
            Write-Verbose "[$((Get-Date).TimeOfDay) PROCESS] Processing $($InputObject.GetType().Name) objects"
            $Processing = $True
        }
        #add each input object to the list
        $data.Add($InputObject)
    } #process
    End {
        #create the cliXML file
        if ($data.Count -gt 0) {
            Write-Verbose "[$((Get-Date).TimeOfDay) END    ] Exporting $($data.count) objects"
            if ($NoClobber -and (Test-Path -Path $xmlPath)) {
                Write-Warning "File $xmlPath already exists."
            }
            else {
                Write-Verbose "[$((Get-Date).TimeOfDay) END    ] Exporting data to $xmlPath"
                $data | Export-Clixml -Path $xmlPath -Depth $Depth

                #create the zip file
                if ($NoClobber -and (Test-Path -Path $Path)) {
                    Write-Warning "File $Path already exists."
                }
                elseif (Test-Path $xmlPath) {
                    Try {
                        Write-Verbose "[$((Get-Date).TimeOfDay) END    ] Compressing $xmlPath to $Path"
                        Compress-Archive -Path $xmlPath -DestinationPath $Path -CompressionLevel Optimal -Force -ErrorAction Stop
                    }
                    Catch {
                        Throw $_
                    }
                    #delete the xml file
                    Write-Verbose "[$((Get-Date).TimeOfDay) END    ] Deleting $xmlPath"
                    Remove-Item -Path $xmlPath -Force -ErrorAction Stop
                    if ($Passthru) {
                        Get-Item $Path
                    } #if
                }
            } #if
        }
        else {
            Write-Warning 'No data to export.'
        } #if

        Write-Verbose "[$((Get-Date).TimeOfDay) END    ] Ending $($MyInvocation.MyCommand)"
    } #end
}

Because Export-Clixml supports -WhatIf, I added it to my function. Here's what it looks like using the defined alias.
PS C:\> get-process | where ws -gt 100mb | exmlzip d:\temp\proc100.zip -WhatIf
What if: Performing the operation "Export-Clixml" on target "d:\temp\proc100.xml".
PS C:\> get-process | where ws -gt 500mb | exmlzip d:\temp\proc500.zip -Verbose -Passthru
VERBOSE: [13:54:51.0494083 BEGIN  ] Starting Export-ClixmlArchive
VERBOSE: [13:54:51.0514100 BEGIN  ] XML file path: d:\temp\proc500.xml
VERBOSE: [13:54:51.0874415 PROCESS] Processing Process objects
VERBOSE: [13:54:51.0937561 END    ] Exporting 4 objects
VERBOSE: [13:54:51.0947561 END    ] Exporting data to d:\temp\proc500.xml
VERBOSE: [13:54:54.8457910 END    ] Compressing d:\temp\proc500.xml to d:\temp\proc100.zip
VERBOSE: [13:54:54.9478040 END    ] Deleting d:\temp\proc500.xml

    Directory: D:\temp

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         3/25/2024   1:54 PM         116238 proc500.zip
VERBOSE: [13:54:54.9590349 END    ] Ending Export-ClixmlArchive

Summary
The best technique for serializing data that you intend to re-use in PowerShell is to use the Export-Clixml cmdlet. This will save the data in a format that can be easily imported back into PowerShell. You can control the depth of the serialization to manage the size of the file. You can also compress the file to save space. The function I provided is a simple way to automate this process.
In a way, Export-Clixml let's you save an object to a file with the highest "resolution." This is especially true if you increase the depth. If you don't need that level of detail there are other options such as JSON. We'll look at serialization options and challenges with that format in an upcoming article.

            (c) 2022-2025 JDH Information Technology Solutions, Inc. - all rights reserved

Don't miss what's next. Subscribe to Behind the PowerShell Pipeline: