PowerShell Pipeline Buffering

Came across an interesting PowerShell question on Stack Overflow today, Powershell piping causes explosive memory usage. The post had a series of external executable commands being piped to each other, like a.exe | b.exe | c.exe, and observed that the executables were not being run simultaneously with each process’s STDOUT going straight to the next process’s to STDIN, but rather everything was being buffered and the executables were being run sequentially.

I posted an answer with some code to demonstrate that PowerShell is buffering STDIN for external processes until STDIN is closed; STDOUT is not buffered, nor is STDIN when native PowerShell commands / scripts are on the right side of the pipeline.

It’s an interesting side-effect of mixing PowerShell with non-PowerShell commands.


2 thoughts on “PowerShell Pipeline Buffering

  1. I’m not sure if there’s a better way to work around the powershell pipeline buffering behavior, but here’s one way:

    C:\> & cmd.exe “/C program1.exe -option1 -option2 -etc | program2.exe -option1 -etc”


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s