PowerShell output redirection: Unicode or ASCII?

What does the redirection operator > in PowerShell do to text: leave it as Unicode or convert it to ASCII? The answer depends on whether the thing to the right of the > operator is a file or a program.

Strings inside PowerShell are 16-bit Unicode, instances of .NET’s System.String class. When you redirect the output to a file, the file receives Unicode text. As Bruce Payette says in his book Windows PowerShell in Action,

and out-file defaults to Unicode. The advantage of explicitly using out-file is that you can then specify the output format using the -encoding parameter. Possible encoding values include Unicode, UTF8, ASCII, and others.

If the thing on the right side of the redirection operator is a program rather than a file, the encoding is determined by the variable $OutputEncoding. This variable defaults to ASCII encoding because most existing applications do not handle Unicode correctly. However, you can set this variable so PowerShell sends applications Unicode. See Jeffrey Snover’s blog post OuputEncoding to the rescue for details.

Of course if you’re passing strings between pieces of PowerShell code, everything says in Unicode.

Very helpful thanks. Got caught out recently trying to process some files from Mapinfo without losing the little Copyright symbol and while still allowing the Application to read them. Tried UT8 first, as ASCII seemed to lose the symbol. Now trying Unicode. It might turn out that the default was the one I needed!

Take note that powershell cmdlet out-file with utf8 encoding includes the byte order mark and there is no option for turning that off with this cmdlet. I had to use [System.IO.File]::WriteAllBytes(string, string) which, by default, writes utf8 without byte order mark.