I decided to write this to inform the community of the specifics of the situation that took the entirety of Wesnoth.org down for the past two days. I’m doing this of my own volition and intentionally without consulting our Community Manager on the matter.

The hosting company that runs Wesnoth.org’s server took it offline because of a payment issue resulting from an expired credit card issued to our financing entity, Wesnoth Inc. Dave received an email warning him that the credit card info on file was no longer valid and that the site would be taken offline on the third week of February if he did not address this soon. For some reason the hosting company decided to do it on the second week instead.

So there’s an issue of negligence both on the hosting company’s part, and Wesnoth Inc’s. For reasons beyond my comprehension, it appears that despite the board elections held last year there still isn’t a definite appointed treasurer and that there is still a single inactive person in charge of running our finances, and by extension, the legal aspect of hosting Wesnoth.org. It also appears that after one of the originally-elected board members quit, doofus-01 was appointed to the board in a heretofore unspecified role. I’d like to give him my congratulations.

Right now the development team, Wesnoth.org staff, and Wesnoth Inc board appear to have no idea what each other are up to, and as a result more incidents of this nature can’t be ruled out as a possibility in the near future. I’d like to ask you to bear with them while they sort out their organizational structure and decide which communication platform to use for their day-to-day discussions. After 8 years of similar issues taking place on a relatively frequent basis, I’m sure things will go back to ‘normal’ soon — after all, Wesnoth is slated to be published on Steam’s distribution platform as soon as version 1.14.0 is released at some point this year.

But for the time being and for the sake of my own health, I’m going to wait it all out, and see if our Community Manager will eventually remember that there’s more to Wesnoth than a select few members of the mainline development team and the people running its finances.

shadowm wrote:So there’s an issue of negligence both on the hosting company’s part, and Wesnoth Inc’s. For reasons beyond my comprehension, it appears that despite the board elections held last year there still isn’t a definite appointed treasurer and that there is still a single inactive person in charge of running our finances, and by extension, the legal aspect of hosting Wesnoth.org. It also appears that after one of the originally-elected board members quit, doofus-01 was appointed to the board in a heretofore unspecified role. I’d like to give him my congratulations.

Awe shucks... Thank you, shadowm!

I am a little unsure of my role too, but there is some effort by the others in trying to find someone for the phone version.

shadowm wrote:Right now the development team, Wesnoth.org staff, and Wesnoth Inc board appear to have no idea what each other are up to, and as a result more incidents of this nature can’t be ruled out as a possibility in the near future. I’d like to ask you to bear with them while they sort out their organizational structure and decide which communication platform to use for their day-to-day discussions

nuorc wrote:Spoiler:

There's long been an IRC/forum divide, and now we have discord. Twitter and Facebook might fit in somehow, but I don't think it was as much of an issue. We'll see how it goes, I guess. But I think the communications platform issue is easier to survive than the organizational issue. That looks really difficult to solve, as it requires some trust and dedication that are hard to demand of volunteers (to a game). I hope greater minds than mine continue to ponder.

One thing that I'd like to have clarified: What is meant by "Wesnoth.org staff"? I assume it includes the Admin group, but what about Moderators/MP Moderators? Would any/all of the Developer group count towards that? There's been a rather frustrating lack of any communication, with the only real information seemingly coming out as a result of iceiceice resigning. It just seems like very little has been actually happening - even when I wrote up a summary of the Board's IRC logs (which I am more than willing to keep doing, if taking the time to reread the logs and write up a summary is a blocker for the current Board members), there was nearly a month of no discussion at all, and this treasurer/credit card issue doesn't exactly inspire confidence either.

99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code

Pentarctagon wrote:One thing that I'd like to have clarified: What is meant by "Wesnoth.org staff"?

The people who run Wesnoth.org itself. All great people who usually don’t receive the recognition they deserve because from Q3 2009 until last year I’d been serving as their mouthpiece, and before that, it was primarily Ivanovic (our Release Manager from circa 2006 until January 2015) and his predecessors who were in charge of announcing everything involving server downtimes and such.

The current line-up:

Rhonda, who runs the main Wesnoth.org host infrastructure. She also usually manages to fix really complicated stuff while the rest of us run in circles like headless chickens (like that time we ended up with an unbootable system after a botched RAID1 rebuild).

Soliton, same as above. Also does some wesnothd stuff from time to time, mostly when I’m not around (see below) and also because he was the wesnothd expert before I even joined.

shadowm (who is still not shadowmaster no matter how much you insist on it), who did pretty much all of the server migration work back in Q4 2013 and inadvertently (because I didn’t read the fine print) ended up responsible for everything that isn’t listed above or below; that is to say, the web server, the database server, and most of the stuff that runs on the web server (including forums and wiki). Note that I’m by no means irreplaceable, and both Rhonda and Soliton have the skills to take over should something happen to me, that is, if given enough spare time; and have done so as needed on several occasions. I guess you could say I’m the first line of defense during an incident, then Soliton, then Rhonda, in descending order of spare time and ascending order of expertise. That’s right, I’m just your tech support grunt crossing my fingers that I can guide you through a cookie-cutter troubleshooting recipe without having to transfer your call to my seniors, who are more than capable of stir-frying me alive if I make the slightest mistake. Reportedly, I taste like chicken.

loonycyborg, who maintains the jenkins CI suite, and as of the past year has also partially replaced me as campaignd (add-ons server) and wesnothd (MP server) VM maintainer since the Git master/1.13 versions of both software now run with his Asio-based network code instead of the legacy network code, and they’ve required a bit of extra testing and patching after his code was first merged into master last year. I still find myself doing most of the user-facing campaignd maintenance work, though, such as tending to support requests regarding lost passphrases and unmaintained add-ons.

AI/AI0867, who used to run the mailer setup but is currently completely inactive. Nowadays Rhonda or Soliton or me are in charge of that depending on which one of us gets to it first (i.e. time of the day, day of the week and so on).

Elias/allefant, who developed and runs the units.wesnoth.org build infrastructure. He would also really like somebody to replace him and make units.wesnoth.org great and awesome.

Kinda sorta zookeeper, who doesn’t have access to anything at all because he’s a Windows scrub.[/sarcasm]

The fact that we are all members of the Developers group is mostly coincidental and only two of us (and zookeeper) are forum admins, and only one of us has actually done forum admin work on a regular basis for the past 8 years (guess who), other than purging spammers and other mundane stuff. The other forum admins, moderators, community, etc. all rely on us keeping Wesnoth.org running. And in turn, we rely on Wesnoth Inc paying for the hosting so that we can have a trusted server to work with for the community’s sake.

Also, I’d like to clarify that the reason that the list of Wesnoth.org staff has remained mostly secret over the years is to keep people from pestering some of us with off-topic stuff. For example, I can deal with someone requesting a username change on the forums, but Rhonda won’t tend to that because it’s not her task and she has far more pressing things to focus on and less time; Soliton and I might run the MP servers, but we don’t deal with moderation incidents because that’s the job of the MP moderators we have appointed. It’s for these reasons that we routinely encourage people to use our official communication channels instead — that is to say, group PMs on the forums, IRC, public posts in the Website section, the forum support email address that will appear any time the forum literally can’t even, and so on.

I'd like to apologize for not writing a post like this immediately after the restoration of services. I did tweet, but should have thought to make an immediate announcement on the forums.

But after I learned, for example, that new mainline campaigns are announced elsewhere but not here, I already lowered my expectations...

It would have been announced with the release of 1.13.7. However, you may have a point that it should be done here too.

And to address Chewan, no, I have not been as good a community manager as I could be. I apologize. I have tried to keep the community informed of any major change or event that happens, but I will certainly try to interact with people here on the forums more.

Creator of Shadows of Deception (for 1.12) and co-creator of the Era of Chaos (for 1.12/1.13).SurvivalXtreme rocks!!!
What happens when you get scared half to death...twice?