This is a topic to teach and discuss the creation of real maps based on the real world, with the real cities on their correct geographic positions and correct approximate ingame population (scaled to the real world, that is). All of this in a way that prevents sticking to any newGRFs during the creation of the map.

The principle

The problem with making a scenario of a country and saving it in .scn is you burn some newGRFs into the scenario; if you want to change newGRFs, you have to do the scenario all over again.

Well, the only alternative to prepare agnostic maps would be the heightmap system.

So far, heightmaps have been used as a foundation to create realistic (or not) maps, by making everything except terrain shaping by hand. Some people have tried to create "heightmap bumps" or marks for the city locations. I've decided to take this one step further.

In the beginning of the efforts,wWhat I have done is collect the coordinates and population figures of 152 of the most important Portuguese cities & towns, one by one, searching in google. I then created a .txt file ready to be imported into microDEM via GPS->WayPoints; Read From File. (for those of you who don't know, MicroDEM is a program that opens and edits heightmaps). MicroDEM would import the city coordinates as waypoints, which could then be "painted" (in the correct place, of course) into the heightmap with a symbol of your choice. This way, one could create a heightmap WITH terrain symbols on the places the cities should be.

Regarding the populations, I collected the populations of all the 152 towns & cities (again, one by one). Next, I categorized the cities according to population ranges (5000 to 10000 people, 10000 to 15000, etc). This way, I could easily scale the Portuguese towns. All I had to do was choose which OpenTTD population the biggest city would have (in my case, Lisbon). I did a "template conversion" so that Lisbon had 10,000 ingame population. It was a good number to scale up/down mentally; if I wanted a smaller Lisbon, say 5,000, I'd just divide all the other pre-calculated population ranges by 2.

A better method

Moki found an even better way to do this. He found a method that no longer requires the search of city locations & populations one by one. Here he explains it step by step:

Moki wrote:

That's how it's done:

1. Download and merge the DEMs for the whole area2. Download the proper location files from http://earth-info.nga.mil/gns/html/namefiles.htm3. Import the txt into Open Office Calc. This is done by open, changing the file type to Text CSV and opening the file. The settings should be changed to "Character Set = UTF-8" and "separated by Tab". This and the following steps can probably be done with Microsoft Excel too, but I have no idea, how that works.4. Data -> Filter -> Standard Filter ... filter by the PC column (that's the importance value/size). For my map, I used PC <= 3, so all cities with values 1-3 are used. This works well for whole country maps. For a regional map, you might want to use PC <= 4 or 5 and for an international map PC <= 2. If you want to use all towns (even the smallest ones without any rating), filter the FC column for P instead. This leaves all "populated places" and removes all the rivers, forests, etc. that you probably don't want to label on your map.5. Now delete all columns that aren't needed. You need LAT (latitude) LONG (longitude) and FULL_NAME_RG (name). The PC column can also be deleted now as we don't need it anymore. 6. Rearrange the columns in the following order: LAT, LONG, NAME. MicroDEM needs this format or it won't work.7. Press ctrl-a -> ctrl-v (copy everything)8. Open the text editor of your choice (notepad or similar), make a new document and press ctrl-v (paste)9. Save that txt file somewhere. Doesn't really matter where, just remember the directory.10 Go back into microDEM, and select GPS -> Waypoints from the Menu. A Waypoints window will pop up.11. In that window click Read from file and select your previously created txt file. All the locations should appear in the waypoints list. The additional data like MGRS, Easting, Northing and Elevation will automatically be calculated.12. Click on WPTs and define how you want your points to look13. Check Waypoint names on map near the bottom of the window, if you want the names to be displayed14. Click Add on map and Plot on map15. If the points aren't displayed on the map yet, click Force redraw (that blue arrow thing) in the map window.16. Now you can zoom and move around your map as usual and of course save the map image. All further steps are done with Photoshop/Gimp and of course in OpenTTD.

Zydeco kindly uploaded a patch he had made some time ago which allows importing the cities into the scenario editor. This way, you no longer have to use heightmap bump references to place cities manually.

Zydeco wrote:

When I was making a few maps from heightmaps a few years ago, I made a patch that would read towns and coordinates from a CSV file, and place them automatically, or place a sign if the towns failed to found.I've updated it for the current version, and tried with your data (although I set all the town sizes to small, it would require manually growing them.It also fails to place a few, because my edge coordinates are a bit off, but it saves lots of work finding where to place towns.

I've uploaded the patch, heightmap and the towns file I made by rearranging yours into the format needed by my patch, and the heightmap resulting from running the import_towns path/to/towns-pt.txt command in the editor, without doing any further adjustments

So, using Moki's method and Zydeco's patch, we can directly fetch a geographic map and city locations & populations from the net, and then import this data into OpenTTD. By other words, we would take steps 1 to 4 of Moki's method, and then use the import town patch. As Moki said:

Moki wrote:

With a tiny bit of reformatting (which can be automated), the data should also be usable with Zydeco's patch. It's definitely possible to automatically translate the PC ratings directly into town sizes and the coordinates are already in the right format.

To-do

We can now fine-tune the patch so that it can take input from Moki's method directly, whilst preventing the coordinate offset he mentioned. Next, all that's left is taking the input of the cities' population and scaling them directly to ingame population, maybe according to some user-defined parameters like "minimum city population" or "maximum city population"... We're really close to having the whole process complete

Next is the town import patch provided by Zydeco, along with a Win32 Build compiled by Moki with the import_towns patch applied.

The patch adds a command to OpenTTD console: import_towns. So, in order to use it, you open the scenario editor, open the console (pressing the \ button over the tab keyboard button) and write import_towns <file_name>. At least this is the principle of it, not sure if the very specific details are correct (a more solid guidance for this shall be written here later). Stay tuned!

Attachments:

File comment: The town import patch kindly provided by Zydeco, who is the true author of the patch.import_towns.patch [6 KiB]
Downloaded 166 times

Oh my, that's genius! And you tell me about this after I just finished my current map?!

I'm a bit confused about the coordinates text file... The first 2 numbers (38.713800 -9.139400) are obviously decimal coordinates, followed by the name, but what does the rest mean? Did you write the files yourself or did you use some tool to compile it?

What I did was input the decimal coordinates on the waypoint dialog of microDEM. After you insert the decimals, microDEM calculates those other values and exports them on the txt file too. And I used microDEM itself to create the .txt, btw . It was VERY lengthy, took me like 6 hours to add the coordinates one by one... Must be a better way, though!

Ah, ok. It's one of those methods that doesn't make the task faster or easier, but more precise.

I'm not an expert with microDEM, but I'll definitely have a try these days. It's one powerful tool, so there's probably an easier way that's not as obvious or well documented.

As I mentioned in the other thread, I usually just place the biggest (or most important) cities by exact map data and estimate the rest. It works like "There are those 2 big cities and about 2/3 of the way between them there's a smaller town"... I just place markers for the cities and then guess where the small town should be.

[edit] There were a few decent sites that sounded promising for coordinates in bulk... mostly unavailable now, thanks to the US government

When I was making a few maps from heightmaps a few years ago, I made a patch that would read towns and coordinates from a CSV file, and place them automatically, or place a sign if the towns failed to found.I've updated it for the current version, and tried with your data (although I set all the town sizes to small, it would require manually growing them.It also fails to place a few, because my edge coordinates are a bit off, but it saves lots of work finding where to place towns.

I've uploaded the patch, heightmap and the towns file I made by rearranging yours into the format needed by my patch, and the heightmap resulting from running the import_towns path/to/towns-pt.txt command in the editor, without doing any further adjustments

@Zydeco:Could you provide a windows binary for that patch (no need for updating it to current trunk)? It looks very useful, but I never got the hang of patching and compiling...

I found a nice source for coordinates. You can download the "GeoLite city" CSV from here and import it into Open Office Calc. The database is rather extensive (about 440000 locations worldwide and even towns with less than 5000 inhabitants are listed) and already gives you proper decimal coordinates. It's not perfect, but easier than finding every single location by google maps or similar tools.Maybe it's even possible to export that data from Open Office in a format that's already microDEM-friendly after removing the cities you don't want to include.

Sadly, no luck for decent population data so far. For Germany the data is pretty easy to find (we count and document everything here ), but a comprehensive list like the one for the coordinates is hard to find.

[EDIT]Woah, forget that last link... go directly to http://earth-info.nga.mil/gns/html/namefiles.htm and be stunned. They got regularly updated data on a huge number of cities and geographic features for the whole world sorted by countries. I just downloaded the data for Germany and there are about 83000 "populated places" (that's everything but the very smallest villages) and about 87000 other locations like administrative regions, vegetation features (forests, basically) and a few others.There's no exact population data, but a rather useful "populated place classification" that sorts the towns by importance from 1 to 5. I have no idea how they do this classification, but the biggest towns are usually the most important, so it's a good starting point for town sizes in OpenTTD.They got no data on the US, though... just a link to the United States Geological Survey which just got (you guessed it) shut down

[EDIT 2]The file from the previous link can be imported into microDEM and it should be possible to generate a map overlay from it. I have no idea, how to do that, though... even if nobody finds out how to do that, it's still good data and a convenient format, though.

Sooo... I found a way to make microDEM do almost exactly what I want See the attached image for the result. This is not the finished heightmap, but the perfect basis for further work. Of course, the names can be turned off and the color and size of the points can be changed as needed.

That's how it's done:

1. Download and merge the DEMs for the whole area2. Download the proper location files from http://earth-info.nga.mil/gns/html/namefiles.htm3. Import the txt into Open Office Calc. This is done by open, changing the file type to Text CSV and opening the file. The settings should be changed to "Character Set = UTF-8" and "separated by Tab". This and the following steps can probably be done with Microsoft Excel too, but I have no idea, how that works.4. Data -> Filter -> Standard Filter ... filter by the PC column (that's the importance value/size). For my map, I used PC <= 3, so all cities with values 1-3 are used. This works well for whole country maps. For a regional map, you might want to use PC <= 4 or 5 and for an international map PC <= 2. If you want to use all towns (even the smallest ones without any rating), filter the FC column for P instead. This leaves all "populated places" and removes all the rivers, forests, etc. that you probably don't want to label on your map.5. Now delete all columns that aren't needed. You need LAT (latitude) LONG (longitude) and FULL_NAME_RG (name). The PC column can also be deleted now as we don't need it anymore. 6. Rearrange the columns in the following order: LAT, LONG, NAME. MicroDEM needs this format or it won't work.7. Press ctrl-a -> ctrl-v (copy everything)8. Open the text editor of your choice (notepad or similar), make a new document and press ctrl-v (paste)9. Save that txt file somewhere. Doesn't really matter where, just remember the directory.10 Go back into microDEM, and select GPS -> Waypoints from the Menu. A Waypoints window will pop up.11. In that window click Read from file and select your previously created txt file. All the locations should appear in the waypoints list. The additional data like MGRS, Easting, Northing and Elevation will automatically be calculated.12. Click on WPTs and define how you want your points to look13. Check Waypoint names on map near the bottom of the window, if you want the names to be displayed14. Click Add on map and Plot on map15. If the points aren't displayed on the map yet, click Force redraw (that blue arrow thing) in the map window.16. Now you can zoom and move around your map as usual and of course save the map image. All further steps are done with Photoshop/Gimp and of course in OpenTTD.

With a tiny bit of reformatting (which can be automated), the data should also be usable with Zydeco's patch. Since I didn't find the time to work out compiling, I didn't look into that any further. It's definitely possible to automatically translate the PC ratings directly into town sizes and the coordinates are already in the right format. Almost automatic real world map generation, here I come

When I was making a few maps from heightmaps a few years ago, I made a patch that would read towns and coordinates from a CSV file, and place them automatically, or place a sign if the towns failed to found.I've updated it for the current version, and tried with your data (although I set all the town sizes to small, it would require manually growing them.It also fails to place a few, because my edge coordinates are a bit off, but it saves lots of work finding where to place towns.

I've uploaded the patch, heightmap and the towns file I made by rearranging yours into the format needed by my patch, and the heightmap resulting from running the import_towns path/to/towns-pt.txt command in the editor, without doing any further adjustments

I tried applying Zydeco's patch but didn't succeed. I tried r25866, but make gives an error (screenshot attached) and doesn't create openttd.exe. Compiling the clean trunk with MinGW (Windows 7 64bit) works fine, so it must be something about the patch... as I said, I've got no experience with programming, patching or compiling and have no idea, where I could have gone wrong.

I tried applying Zydeco's patch but didn't succeed. I tried r25866, but make gives an error (screenshot attached) and doesn't create openttd.exe. Compiling the clean trunk with MinGW (Windows 7 64bit) works fine, so it must be something about the patch... as I said, I've got no experience with programming, patching or compiling and have no idea, where I could have gone wrong.

Thanks you, Zydeco! After several tries it finally worked. I attached the windows build for those who want to experiment with the patch but aren't willing to learn about patching and compiling under Windows. It took me almost a whole day to get everything working, so I can fully understand that Just start the game as usual, go to the scenario editor, import your heightmap and type "import_towns" (without the ") into the console and go from there. I'm sure you'll figure it out.

Now I'm off to find the fastest and most effective way to use the combined DEM and NGA data for this patch. I hope to find the time for writing a comprehensive tutorial (from heightmap download to finished scenario) tomorrow. Almost automatic scenario generation for the ambitious mapmaker (AASGAM, maybe?) is only a step away.

@Zanman777: Could you please link this in the first post? People might find it useful.

@Zanman777No, I used MinGW/Msys and followed the tutorial step by step. Took me a while, but in the end it worked remarkably well.And please do choose whatever name you like. Silly acronyms seem to be a tradition around here, so I won't object at all

@ZydecoI may have found a rather obvious bug in your patch. I just tried importing towns into a testing map and get the "Error reading edge coordinates..." every time when East-Coordinates are involved. For example 50N,20W,40N,30W works fine while 50N,20E,40N,10E or 50N,10E,30N,10W give me the error. The same thing happens for towns. If a town has East-coordinates, the console says "Coordinates for [Town] are wrong...", no matter if the town's in the selected area or not. South and North work fine, though - even with an area like 10N,20W,10S,40W. Everything's fine as long as the eastern hemisphere isn't involved. So the patch is great for Portugal but problematic for Germany I just had a look at the code (love it - very clean and well-commented), but it's 4:45am and my head doesn't want to learn about programming anymore.

@ZydecoI may have found a rather obvious bug in your patch. I just tried importing towns into a testing map and get the "Error reading edge coordinates..." every time when East-Coordinates are involved. For example 50N,20W,40N,30W works fine while 50N,20E,40N,10E or 50N,10E,30N,10W give me the error. The same thing happens for towns. If a town has East-coordinates, the console says "Coordinates for [Town] are wrong..."

I've tried myself (on Mac OS X) and I haven't had problems with edges like 50N,20E,40N,10E or 50N,10E,30N,10W, and towns in it.I think the problem may be your locale expecting comma instead of period as decimal separator, I'm not sure how that works on windows, on unix I'd run the program with LC_ALL=CI can probably update the patch later today to parse decimal numbers in a locale-independent way

Who is online

Users browsing this forum: No registered users and 3 guests

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum