Complete noob. Trying to extract some positions from a text file (easting and northing), then do some basic trigonometry. I have multiple issues, the first being the extraction. The below is what I am using for a fixed string:

Ideally what I would like to do is read an input file with an external lookup search list, and output each to the same file name. In the above example it is looking for R 1 (for some reason this file does not pad with leading zeros), and output to R001.txt

The second thing I want to do is subtract the eastings and northings from each line to get delta eastings & delta northings, and then calculate some other attributes (azimuth etc).

Now, I am not sure if I should first get these into readable files, so that we are not reading the same input over and over again. What I am suggesting, is that maybe the output file should be:

easting (1), northing (1), easting (2), northing (2)

So that the next parse of the file can do the trigonometry in one line step (hope that makes sense)?

I think for now I should just concentrate on getting the output correct, rather than the math, so if anyone can help it would be much appreciated.

I've attached a sample file. Ideally I would like to extract the following:

Code

R 1 R 564 R 565 R1128 R1129

and so on, but like I say, maybe an external lookup file would be the best option (because then I can just edit it to suit as I need).

Many thanks for the reply. I should have checked the sample file properly before I posted (was just trying to keep it small), and can see it only had one set of data... this format actually repeats over and over again, and the filesize is realistically ~1.5 Gb in total!

I was intending to extract each of the matched records to a unique file (of the same ID) in an attempt to keep the file sizes down when moving on to the math, and then graphing etc.

So for my new sample attached (using my original code - which should have had \n), I would get the below for ID 1:

If you only need to traverse the file once, then Bill's posted solution is probably what you should use. If you need to traverse the file several times, then the best method would be to put all the data into a database.

This would be necessary with a large (1.5 GB) file as all the items in a data structure would most likely exceed your RAM memory.

Loading the database for a file this size would probably take as much time as reading through it once. But then lookups should be relatively speedy.

Below are two scripts. One loads the data into the database and the second retrieves the records. I used the SQLite database as it suffices for this problem. You would need to install DBI and DBD::SQLite. The SQLite database is included when you download DBD::SQLite.

Sorry for the late response, I couldn't log in (said account was not verified).... seems OK now though.

I'm exceeding the limits of my shell at the moment. I was using mobaxterm, then tried the Ubuntu bash shell in Windows 10. A lot of the extra modules I cannot download (at the moment as I'm on a ship). Looks like this is a project I'll need to continue when I get home...

As for the subtraction of eastings/nothings. Once all of the unique instances of one ID are in the database. If you subtract the first easting, from the second easting (for say ID 001) then we would have the delta Easting. Same for the northings. Once we have these values we can calculate the azimuth between the two points and the hypotenuse, which would be the delta along.

Hope that makes sense. Could really do with a picture - will do that next..

The attached image hopefully makes sense. Once the actual hypotenuse is calculated, the nominal hypotenuse value is subtracted (one assuming that distance between points and azimuth were constant), and we would get the delta distance along.

I am also trying to calculate the delta distance across. To do this I need to calculate the azimuth between the two points to do the trig to find the dAcross value. Then, as there is a nominal value for this based on constant distance between points and azimuth, this can be calculated and subtracted to get the delta distance across.

Once the delta distance along and delta distance across is calculated for each coordinate pairs down the dataset, I would like to plot them.