PowerShell Pipeline Buffering

Came across an interesting PowerShell question on Stack Overflow today, Powershell piping causes explosive memory usage. The post had a series of external executable commands being piped to each other, like a.exe | b.exe | c.exe, and observed that the executables were not being run simultaneously with each process’s STDOUT going straight to the next process’s to STDIN, but rather everything was being buffered and the executables were being run sequentially.

I posted an answer with some code to demonstrate that PowerShell is buffering STDIN for external processes until STDIN is closed; STDOUT is not buffered, nor is STDIN when native PowerShell commands / scripts are on the right side of the pipeline.

It’s an interesting side-effect of mixing PowerShell with non-PowerShell commands.

2 thoughts on “PowerShell Pipeline Buffering

  1. I’m not sure if there’s a better way to work around the powershell pipeline buffering behavior, but here’s one way:

    C:\> & cmd.exe “/C program1.exe -option1 -option2 -etc | program2.exe -option1 -etc”

    Like

Leave a comment