I have 60,075 .txt files(TAB delimited) that are created from ImageJ that contains the XY coordinates and RGB tuples for each frame(60075 frames) of a video. (Macro took me days to make, 16 hours to run, and ate up 360GB)

I need the RGB tuple information from one coordinate x=384, y=377 for each frame. (this information is not always found on the same line#, which is why I need a piece of logic to find the data)

I need this data in a new file that can be loaded into a spreadsheet. So it needs to have either TAB or comma separated values. The file being created will have 60076 lines of data when completed.

Each RGB tuple file is named 00000.txt - 60075.txt (G:\Frames as bitmap\00000.txt) <-where it sits for me

If you have several coordinates to check for each file, then it will be far more efficient to read each file only once and check all the coordinates during this unique reading of each file.

I would assume that the coordinates that you ar elooking for are usually coming always within a certain line number ranges. It would probably faster to discard immediately the lines that cannot give a match because they are outside the required range.

The use of the index function would probably be faster than the use of this regex:

Code

if ($_ =~ /^390\t379/)

For monitoring the progress, you could print a message for each file being checked, or for every ten files (checking whether the file number can be divided by 10), or whatever experience will give you a satisfactory monitoring, you need to make some tries.