Last weekend OpenTTD migrated its source (Subversion) and bugs (FlySpray) to GitHub (https://github.com/OpenTTD/OpenTTD). This change was long overdue, and after months of slow-chatting ideas back and forward, last month we finally reached a decision: we are migrating to GitHub.

This migration took a few weeks of preparation; and we are still in the process of dotting the 'i's and crossing the 't's, but the result is very pleasing and we hope you will like it too.

So what has changed?

We ditched Subversion, and now use git. git has been in widespread use for several years now, so although we understand it can take a bit of time getting used to, though there are plenty of manuals around to get you started. We strongly advise you to read the Guides of GitHub (https://guides.github.com). It is really good for the basics. The benefits for git over subversion are ample; to name a few: local repositories, your own branches, decentralized, local history, ..

Next we ditched FlySpray, and are now using GitHub Issue tracker. FlySpray was on the list for replacement for years. It hasn't been maintained upstream (well, recently there are some commits; but nothing that changes the issues we are having). A replacement was hard to find; but GitHub's Issue Tracker is sufficient for what we want.

But more importantly, GitHub has Pull Requests. This replaces "patches" on FlySpray. The way OpenTTD was handling patches (adding attachments to FlySpray issues) was very old .. and badly needed replacement. The inability to leave in-line comments and see what changed was really hurting reviews, up to the point where they didn't happen.

The other big change here is that Pull Requests are now validated by Continuous Integration before they are merged; meaning the main branch (master, or what used to be trunk) is a lot more stable, and reviewers of Pull Requests have fewer things to worry about (they can see it compiled on a various of compilers/platforms, instead of finding that out themselves).

In other words: the workflow for patches has now drastically improved; and we have been experiencing that over the last few days already.

The migration to GitHub has also another reasoning: it is much more open and reachable for other people. GitHub is very well known for a lot of developers; this means that we reach more developers that "just make a PR to fix this tiny thing". And because reviewing is easier, it is also much more likely to be accepted. Win-win-win-win.

The last few weeks we have been working on plans, scripts, and scenarios on how to get here. We imported all FlySpray issues (closed or not) to GitHub, keeping the number as it was on FlySpray the same - making #1 on FlySpray also #1 on GitHub. Next, last week, we have been cleaning up all the issues on GitHub. We looked at what could be closed, what needs attention, etc. We are down from over 400 issues, to only 235 issues at the time of writing, and we plan to prune it down even further.

We also use a new policy in regards to closing issues; keeping issues around with "a good idea" or "not really a bug but we should maybe fix it" turns out to have the reversed effect: nobody looks at it anymore. Although we really appreciate feedback and ideas, we will close enhancement requests that we don't expect to fulfill in the next year. Also bugs that are not really bugs but feature requests, are much more likely to be closed with the message: "we are going to pass on this one; thank you though". Many of those ideas etc do have a place on the forums; and if enough people like it, someone will stand up and make it We just don't want the Issue Tracker to become a huge sinkhole of endless ideas nobody is going to do anything with.

Some random facts: during this cleanup we found 35 tickets that went under the radar and still need triage; we invite anyone to help us with that. We are down to 65 enhancements, 46 tickets we consider "good first issues", 136 real bugs (well, most likely anyway), and still have to process 69 patches that were imported from FlySpray.

The latter is a bit difficult, as we would much rather prefer that the original author make the Pull Request than we. But slowly we will work on the list, and make a Pull Request ourselves. This will take some time.

We also have been working on making a better more robust Compile Farm. The old one wasn't really maintained and most targets didn't support C++11, let alone C++14 or C++17. This was hurting development.

So what we did over the last few months, is dockerize (https://www.docker.com/) the Compile Farm, and put the source of it up (on GitHub of course): https://github.com/OpenTTD/OpenTTD-CF/. This is still being worked on, and is one of the priorities in the next few weeks. These Docker containers allow you to locally build the exact same binaries as we do for releases, but also to validate your code under other compilers and platforms. Every Pull Request is now validated by these validation dockers and the results are reported back to the Pull Request. This comes back to the earlier point: developers see issues faster, meaning they need less time doing "housekeeping", meaning they have more time doing to good stuff: accepting Pull Requests

The other thing we are working towards, is making it possible for anyone to easily get release binaries for their fork. This is mostly for Patchpacks, but also for big new features which needs good user-testing (NRT, for example). We are attempting to allow with some simple administrative actions the Compile Farm to create those binaries, and make them publicly available. This of course means that "official" OpenTTD gets into competition with Patchpacks; and we approve of this. We are fully aware it could lead to a Patchpack being more popular than "official" OpenTTD; and we will be upfront about that.

This does need some drastic code changes; for example savegames are currently not compatible between Patchpacks and "official" OpenTTD, and merging from one to the other can lead to nightmares in savegame compatibility. There are already solutions in Patchpacks available, and we will actively work towards getting them in the "official" OpenTTD.

So what more is there to come? Last week we closed off our local mirrors of Git and Mercurial in favour of GitHub. This means https://git.openttd.org/ and https://hg.openttd.org/ now redirect to GitHub. All the repositories hosted on there are no longer available. If you depended on them, we do apologize. The reason for this is simple: if we keep them alive, it might be that you, as user of such repository, will never notice it is not being updated anymore. And we don't really have another way to make you aware, other than shutting it down. A bit drastic, but sometimes you need to break eggs to ..

svn://svn.openttd.org/ is currently still available, but by the end of this weekend will also be unavailable. It can still be reached at svn://svn-archive.openttd.org/. Here too, we want people to know we moved to GitHub. Not seeing any updates is worse than not being able to reach the address, going to the website, and figuring out what is going on. Again, apologies for any inconvenience this might give.

Last in this list, the git on GitHub has been recreated. This means that if you have some branches of your own based on our old git mirror, you have to do some action before you can work with the version on GitHub. Mostly, this involves: git remote add upstream https://github.com/OpenTTD/OpenTTD; git rebase --onto upstream/master origin/master origin/<your branch>. This should fix most issues for most people. If not, drop by on IRC and let us help you!

In closing, OpenTTD has been doing everything "on its own" since 2004. For the longest time this was needed as nobody could supply what we were looking for. But the world has caught up; GitHub supplies us with everything we were looking for. It is a change, a huge change, but we truly believe it will lead to a better place; both for OpenTTD and all its fans, but also for all the developers out there.

If you are interesting in helping out finishing this migration, please step forward (get to IRC, and start talking). Mainly if you like Docker, if you are interested in cross-compiling, know how to use MSBuild from PowerShell, like writing documentation (e.g. CONTRIBUTING.md or README.md), enjoy cleaning up issues, know how to triage issues, etc.

And interested in helping out with OpenTTD? Go to our Issue Tracker, click on the "good first issue" label, and get going! Drop by on IRC any time for additional help and/or hints.
TL;DR: Put up or shut up. Things are changing. Get moving. Or to leave it in the words of andythenorth:

We have moved the OpenTTD source to GitHub.

It was probably overdue.

Things will be better.

We tried not to kill any kittens in the process. Promise.

The only thing necessary for the triumph of evil is for good men to do nothing.

I will have to catch up with the recent changes, but I really like the roadmap you have decided on. I miss the numbered revisions (r00001...) though.

Thumbs up for giving developers a way to ease the creation of binaries for distribution. That must be still under development, but where they will be stored? Which users and how will they trigger the creation of such binaries?

About the saveload versions with different branches, I am interested about how you decide to solve it, because that is an annoying issue.

Finally! It will be more familiar to me to reporting bugs.
Additionally, should I pull request when I have to change my language's translation via github (like OpenRCT2), or keep using WebTranslator?
(Of course I think there is rarely additional stuff to translate.)

Telk wrote:Additionally, should I pull request when I have to change my language's translation via github (like OpenRCT2), or keep using WebTranslator?

We just reenabled the WebTranslator/Eints, so translators can continue as before.
Eints validates the translations and then pushes them every day.

PRs are the way for adding completely new languages, but we do not want to handle PRs for updates to a few strings in languages we do not understand

Juanjo wrote:About the saveload versions with different branches, I am interested about how you decide to solve it, because that is an annoying issue.

I am not sure whether my link refers to the newest version, but something in the direction of:https://github.com/JGRennison/OpenTTD-p ... ver_sl.cpp
But one could also go towards self-describing structures and map array. (read: store descriptions of the binary structures in the savegame, including string-identifiers for the fields, types and sizes).

Baldy's Boss wrote:I'm a FreeBSD user who fairly recently migrated TO Subversion for tracking ports,and there it's considered the new thing compared to CVSup.
Not the same in application development world?

So if I can make a prediction, in 14 years FreeBSD moves to git They are a bit lagging behind it seems

CVS .. that is a long long long long time ago

But so yeah, the same, just delayed with a few years Most likely for very valid and good reasons. svn2git tools are very good too, so meh

PS: 2012 is no longer "recent" ..
Okay, I take my comment back. They migrated in 2012 to SVN. Seems they are not lagging behind that much. Maybe in 2 years to git?

The only thing necessary for the triumph of evil is for good men to do nothing.

Baldy's Boss wrote:I'm a FreeBSD user who fairly recently migrated TO Subversion for tracking ports,and there it's considered the new thing compared to CVSup.
Not the same in application development world?

So if I can make a prediction, in 14 years FreeBSD moves to git They are a bit lagging behind it seems

CVS .. that is a long long long long time ago

But so yeah, the same, just delayed with a few years Most likely for very valid and good reasons. svn2git tools are very good too, so meh

PS: 2012 is no longer "recent" ..
Okay, I take my comment back. They migrated in 2012 to SVN. Seems they are not lagging behind that much. Maybe in 2 years to git?

Turns out there IS in fact a https://github.com/freebsd for those so inclined but I can't see them outsourcing the central functions.

With that compile farm in place I MIGHT actually try to make my patch pack more compatible with other platforms. Having infrastructure in place that I can simply use will help a lot with that. Making the savegames compatible however will be a nightmare I think. Maybe JGR can help me integrate his system somehow, but it still won't be super easy. Sometimes we have the same feature but they use flags in different enum locations and such or the feature may have been introduce slightly differently so it's actually not the same feature at all.

I personally am not super optimistic about achieving savegame compatibility between patch packs but we'll see.

It's quite possible, my patchpack can load several different variants of SpringPP savegames.
It is however a non-trivial amount of effort to get right and requires having near-complete feature coverage. Many key features cannot be gracefully ignored on load if the version doing the loading doesn't identically support a feature used in the savegame.

KeldorKatarn wrote:...but they use flags in different enum locations and such or the feature may have been introduce slightly differently so it's actually not the same feature at all.

afterload.cpp is full of these sorts of fixups and bit shuffling, both in trunk and in some of my additions.
I wouldn't worry too much about those.

Eddi wrote:it's not called "trunk" anymore, it's now called "master" and can be found on https://github.openttd.org
also, there's no revision numbers anymore.
nightly versions are currently unavailable, until the compile infrastructure is updated.