Hi David - can you please send the original file and the TextPipe filter? The filter is a simple byte-by-byte state machine, expecting <decimal digits>; or <hex digits>; or It handles 2-byte unicode characters properly. I don't understand why *any* changes are being made to your file as the non...

What about this: https://en.wikipedia.org/wiki/Perl_Compatible_Regular_Expressions Unicode character properties Unicode defines several properties for each character. Patterns in PCRE can match these properties. e.g. \p{Ps}.*?\p{Pe} would match a string beginning with any "opening punctuation" and e...

Hi there, One approach you could use with REDSHIFT import is to REMOVE lines that do not have 850 fields in them, then treat these exceptions differently. The file size of 600GB is not an issue for TextPipe. Use Filter Library\Extract\Extract lines not matching (inverse grep) With an EasyPattern of:...