When I try to run this it seems that it errors out on the -skip. Could that maybe be from a different version?
–
percent20Jan 15 '10 at 20:41

1

-Skip is new to Select-Object in PowerShell 2.0. Also, if the files are all ascii then you might want to use set-content -enc ascii. If the encodings are mixed, then it gets trickier unless you don't care about the file encoding.
–
Keith HillJan 15 '10 at 20:51

I installed powershell 2.0 and it is working now.
–
percent20Jan 15 '10 at 21:06

While I really admire the answer from @hoge both for a very concise technique and a wrapper function to generalize it and I encourage upvotes for it, I am compelled to comment on the other two answers that use temp files (it gnaws at me like fingernails on a chalkboard!).

Assuming the file is not huge, you can force the pipeline to operate in discrete sections--thereby obviating the need for a temp file--with judicious use of parentheses:

I realize this is quite an old question, but I just had to do the same task, and gc | select ... | sc took over 4GB of RAM on my machine while reading a 1.6GB file. It didn't finish for at least 20 minutes after reading the whole file in (as reported by Read Bytes in process explorer), at which point I had to kill it.

IIRC this is because the parentheses around the gc|select means it reads the entire file into memory before piping it through. Otherwise the open stream causes set-content to fail. For big files I think your approach is probably best
–
AlexMar 15 '13 at 15:58

Thank you, @AASoft, for your great solution! I've allowed myself to improve it slightly by dropping the comparison operation in every loop speeding up the process by something like 25% - see my answer for details.
–
OliverJul 14 '14 at 21:20