Sunday, 7 June 2015

When two Seemingly Identical CSV Files are Not!

I’m working on a PowerShell tool that needs to create CSV
output, so I thought “if I create a standard text output with commas in, it
will work.” It didn’t work first time and here’s why!

The code I was using to create my CSV file was like this:

$OutFile
| Out-File $SavePath -Force

The problem is that Out-File defaults to Unicode. An easy
way to demonstrate this is:

Out-File
test.file

Open the file in bog-standard Notepad, go to File > Save As... and you’ll notice
the current encoding is Unicode.

Image: Notepad
showing Unicode file Encoding

The solution is a very simple change to the code:

$OutFile | Out-File $SavePath -Encoding Default -Force

If you’re wondering what default is, right-click the desktop
and create a new text file, then open up in Notepad and go to File > Save As... again, and you’ll
notice the encoding is ANSI!

Image: Notepad
showing default ANSI file Encoding

And this leads on to the title. When I copy and pasted
the output generated by my script into a freshly created text file, it worked
as a CSV, but the original didn’t work as a CSV (everything was in one column);
using -Encoding Default we now have
success!