Introduction

My previous article showed how to run PowerShell scripts from C#. That implementation was limited in the sense that it would run scripts synchronously, blocking until the script finished what it was doing. That is fine for short-running scripts, but if you have long-running or even never-ending scripts, you will need asynchronous execution. This article shows just how to do that.

Basic Steps

Here are the basic steps to run a PowerShell script asynchronously:

Create a Pipeline instance by calling Runspace.CreatePipeline()

Pass the script to the Pipeline instance using pipeline.Commands.AddScript()

Feed the Pipeline its input objects using pipeline.Input.Write()

Close the input by calling pipeline.Input.Close()

Call pipeline.InvokeAsync(); this will cause the Pipeline to create a worker thread that will execute the script in the background

Start reading pipeline.Output for the results of the script, until pipeline.Output.EndOfPipeline becomes true

There are two ways in which you can be notified of the availability of new output data:

First, there is a property pipeline.Output.WaitHandle of type System.Threading.WaitHandle. This handle can be used with the various static Wait***() methods of System.Threading.WaitHandle to wait for new data to arrive.

Second, pipeline.Output has an event called DataReady. By subscribing to this event, the PowerShell background thread will call you every time new data becomes available.

The Trouble with Output.WaitHandle

At first hand, Output.WaitHandle seems like a nice option to choose for retrieving the script output data; it provides complete separation between the producer (the PowerShell thread) and the consumer (the output reading thread), unlike the DataReady event which is called directly from the PowerShell thread. But there's a problem: if the consumer isn't fast enough in dealing with the output, the producer will happily continue producing at full speed until it consumes all your memory, or when the script ends. While it seems as though there are provisions in the output queue to limit the maximum amount of memory used (there is a MaxCapacity readonly property), I haven't found a way to actually set this limit.

You may wonder why the consumer would be too slow to process the output. Well, if you're using PowerShell scripting to automate some aspects of your C# program, you'll probably want to display the script output in some way on your user interface. The output of a PowerShell script will easily outrun the refreshing capabilities of any GUI.

Ready for DataReady

Since the DataReady event is called directly from the PowerShell thread, it allows us to throttle back the PowerShell processing speed to the point where it exactly matches the throughput of the consumer. Of course, this means it will reduce the speed of the PowerShell script, but I feel that is the lesser of two evils, in this case.

PipelineExecutor

All of the above steps have been wrapped in a single helper class called PipelineExecutor. This class helps you to easily run a PowerShell script in the background, and also provides you with events to receive the output data of the script. It also 'Invokes' the data to the correct thread, so when the events arrive, you no longer have to perform the dreaded 'InvokeRequired' routine just to display the data. Here's the public interface of the class:

Using the Code

The following code shows how to create and asynchronously run a PowerShell script using PipelineExecutor:

...
using System.Collections.ObjectModel;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
using Codeproject.PowerShell
...
// create Powershell runspace
Runspace runSpace = RunspaceFactory.CreateRunspace();
// open it
runSpace.Open();
// create a new PipelineExecutor instance
// 'this' is the form that will show the output of the script.
// it is needed to marshal the script output data from the
// powershell thread to the UI thread
PipelineExecutor pipelineExecutor =
new PipelineExecutor(runSpace, this, textBoxScript.Text);
// listen for new data
pipelineExecutor.OnDataReady +=
new PipelineExecutor.DataReadyDelegate(pipelineExecutor_OnDataReady);
// listen for end of data
pipelineExecutor.OnDataEnd +=
new PipelineExecutor.DataEndDelegate(pipelineExecutor_OnDataEnd);
// listen for errors
pipelineExecutor.OnErrorReady +=
new PipelineExecutor.ErrorReadyDelegate(pipelineExecutor_OnErrorReady);
// launch the script
pipelineExecutor.Start();

Please note that to compile the example project, you'll first have to install PowerShell and the Windows Server 2008 SDK. For details, see my previous article.

Error Handling

There are two kinds of errors that you can encounter during execution of a powershell script:

Errors that occur during execution of a powershell script, but that don't result in the termination of the script.

Fatal errors that are the result of an invalid powershell syntax

Errors of the first kind will be pushed into the error pipeline. You can listen for those by adding an event handler to PipelineExecutor.OnErrorReady.

To detect and display errors of the second kind (fatal syntax errors) you need to inspect the property Pipeline.PipelineStateInfo.State inside your OnDataEnd event handler. If this value is set to PipelineState.Failed then the property Pipeline.PipelineStateInfo.Reason will contain an exception object with detailed information on the cause of the error. The following code snippet shows how this is done in the example project:

If you want to see the error handling in action, please execute the "Error handling demonstration" script of the example project.

Points of Interest

For people interested in the inner workings of PipelineExecutor, I'd like to point out the private StoppableInvoke method. It waits for the target thread to process the data, thus causing the throttling effect on the PowerShell script. It also avoids potential deadlock problems that would happen if I had simply used ISynchronizeInvoke.Invoke because it is interruptible by way of a ManualResetEvent. This pattern could be useful in other workerthread-to-UI notification situations.

I've also worked on improving the performance of the script execution by separating the 'BeginInvoke' and 'EndInvoke' stages of StoppableInvoke into subsequent DataReady cycles, and this works like a charm... but the resulting output performance is just too close to the consumer performance, causing lag in the user interface. My theory is that .NET likes to give priority to Invoke messages before handling the UI update messages. So, if you get close to 100% of the message loop performance, it will starve the UI updates. Solving this problem probably warrants a different article.

Comments and Discussions

i don't see well where the script are send async.
let say i have a PSscript that copy file to 6000 machine(stored in a datatable).
using your code i try do all the copy in same time but i have problem.

ether if i put the script in a foreach loop it say pipeline can't run simultaneous.
so i'm confused how can i can copy file to all those computer and having my 6000 return async ??

in powershell i would do something like that
../copy-file.ps1 MachineName -asjob

The runtime version it uses is inherited from the process, so I guess you have to target a higher platform (.net 3.5 or 4.0) when building the Asynch Powershell program (right click on the project, properties, application tab, target framework dropdown list).

Define "unable". Is the collection empty, or doesn't it compile? Your pasted code contains lots of comments with all your tries.. please narrow it down to the smallest example you can manage, without the dead code.

Error The best overloaded method match for 'PipelineExecutor.PipelineExecutor(System.Management.Automation.Runspaces.Runspace, System.ComponentModel.ISynchronizeInvoke, string)' has some invalid arguments

First the easy part: the IDisposable implementation on the form is not needed, in fact I don't know why this is in there (I must have accidentally added it with the latest update).

The other part of your question is a bit more difficult. For the PipelineExecutor to work it needs a way to push (or marshal) the results of the script to the UI thread. The ISynchronizeInvoke interface does this (a WinForms Form implements this interface).

To make it work in WPF however, you have to implement your own ISynchronizeInvoke object which uses the WPF Dispatcher to marshal the results to the WPF UI thread. Luckily someone has done this already: http://doitonthedesktop.blogspot.com/2010/04/hosting-openspan-complete.html[^]. If you use that one, make sure to also include the modification that is mentioned in the comments, it looks important

I should probably update the article to include a WPF compatible PipelineExecutor since WPF is the way of the future

jpmik, Thanks for the replies. I went ahead and implemented the ISynch with the dispatcher from that link you provided. I have gotten further but still coming up with an error. I was hoping you might point me in the right direction. When I execute "dir" for instance I get a an ObjectDisposedException after the first returned child item.

If you look at the source of the HowToRunPowerShell article you'll see the following line:

pipeline.Commands.Add("Out-String");

So that causes the 'Out-String' command to be appended to your scripts. This lets powershell format the results to a string, according to the powershell defaults. If you remove that line the results will be identical. In comparison, the asynchronous project simply uses an object.ToString() to format the results.

If you use the following script in the asynchronous host, you'll see the first result again:

Firstly I thank you for the excellent work.
Now, I wanted to ask whether it is possible to implement a Windows Service, with the class PowerShellExecute. I tried but could not. I always have an error in:

In the examples I pass 'this' to the pipelineexecutor constructor because 'this' is a Window which implements ISynchronizeInvoke. In a service you normally don't have a userinterface, so you will probably have to create your own implementation of ISynchronizeInvoke and pass that to PipelineExecutor.

Alternatively (and probably EASIER) is to modify PipelineExecutor.cs so it calls the OnDataReady, OnDataEnd, OnErrorReady events directly, without using the ISynchronizeInvoke. Here's how you do that.. replace the StoppableInvoke() method in the sources by this:

I have a pipe that creates a session in Outlook Live with a PowerShell.
I have only one session, I can't create multiple sessions.
Now, the problem is that I have multiple clients to execute commands in this session, through sockets.
My problem, is return the right result of the PowerShell Command invoked by a client (socket), to the right invoker client (sokect).

I've been working on a web version of your script.
That is, a page where one can add some powershell
scripts and it be executed and the output displayed.

Initially I had the same problem that Paulo and I fixed it
with your tip. But I am still having problem to get everthing working. Right now, it executes perfectly when I am debugging but it doesn't show anything when there are no breakpoints.

I thought it could be a problem with this "fix" we're using here. So, I changed your original code to do not call BeginInvoke and EndInvoke but DynamicInvoke(as you said) and it starts to throw an exception about thread invalid access[1].

So, can the same error going in the modified version of your script be affecting my implementation in some hidden way and causing this strange "only work in debug" behavior?

Although I am too noob in C# it looks to me that another option would be implement ISsynchronizeInvoke interface inside my page class. But it seems hard for my current C# skills. Do you have some kind of general advice to do this job in a softer way?

Thank you very much!
Very nice article!

Caio

[1]- "Cross-thread operation not valid: Control 'listBoxOutput' accessed from a thread other than the thread it was created on."

- replace all the StoppableInvoke calls by direct calls to the event handlers OnDataReady, OnDataEnd, OnErrorReady
- so, now your application will get all the data, but it's on the wrong thread
- in the application event handlers, put all the data in a Queue, and make sure you lock() the queue when you do
- now on the userinterface thread you need a timer or something that will retrieve the data from that queue (now it's on the correct thread)
- display the data
- done!

Thank you so much for the article, This is what I was looking for last three days.

I've read your article, Currently I'm trying to have web-based application to get the result of Powershell from single client as a first phase (or multiple possibly). But the original source doesn't work with my model due to WPF. I'm a beginner for this area(C#), it would be really helpful to learn C# and Powershell.