root log: November 2014

It's November, and we have a lot to be thankful for. It's been a grand 15 years since we added the 2 in Everything2. It's been so long that the original Everything is but a glint in history's eye. We paved the way for a lot of modern websites, and it's been an incredible 15 years. Safe to say that this website hasn't exactly kept up with the times in terms of functionality or looks, but we still love it. This log is going to talk a bit about how we're modernizing things from here, and what this all means.

Last month, in root log: October 2014, I said that I was going to be working on writeup feedback, and I still am. I am taking a deep dive into the display code for how the writeup pages are generated. This is advancing a bit more of the modernization work I am pursuing, and it's all in all, a very good thing. I'm hoping to deliver on it this month, after I unsnarl some of the stuff in show content.

I haven't been root logging as much, but work is still continuing. A lot of work is happening in the lab, but I'm going to try to be more up front about what's changing. We've got 15 years of cruft to undo, but I'm making steady pace on it day by day. The major push right now is to refactor the display items into controllers. This is going to use the basis of the emulation work that I've put forward before. All of the moving things into Everything::Application and the work pushing htmlcodes and opcodes into Everything::Delegation are going to help immensely in how this goes down. Some of the layers can't be fixed until after we have a full controller stack working for us. For instance, maintenances are too entangled into the Nodebase objects to pull out until after we have those update handlers nailed. My original method for doing this was to make it so that we had strong objects (the M in the MVC paradigm), and we could refactor a lot of the code into that. It is pretty clear from here however that we need to separate display and controller logic first.

This is going to accelerate the Mason (templating system) work, and the more extensive push to use Moose is going to come after that.

Time to shave some yaks. I'll start adding updates from here on down

Nov 1-15:

My day job was a bit crazy in here, but the gist is that I experimented with various Controller schemes and dug into how writeup feedback could be added to the codebase without causing an immediate run-up in technical debt. Instead, I refactored everything that could leave Everything::HTML over to Everything::Application, and that leaves only the page routing and generation code inside of it. This leaves us with a good point to put a fork in to move things into a template scheme, nodetype by nodetype. The rest of the time I have spent assembling the starter template divisions from scratch. This means moving a lot of the code that is in nodelets and htmlcodes over to a controller/sub-controller setup. We can emulate execution if we need to, but generating pages without using eval() is the end goal of this exercise.

Nov 17:

Finished removing the legacy setting of 'settings_useEpicenterZen', which is now on by default. 72 accounts weren't converted, so I did them by hand, and removed all mention of the setting.

Deleted the nodecache nodelet. There are other, more secure ways to look at the nodecache in production. Cache dump being the real way to do it. Having something that would / could run every pageload doesn't make any sense.

Nov 19:

Template work in development now includes the entirety of the zen container chain. We can fully render the page out of Mason properly, including a lot of display variants. We do not properly include some of the mobile stuff, as that is like instantly going to be thrown out.

Moved browser logging from the Epicenter (no kidding) over to Everything::HTML, where it will eventually be put into the request class, after we don't need to use the display fork in development.

Simplified zen stdcontainer in a few places by using the $APP functions instead of inline stuff.

Nov 20:

Epicenter now works in the template environment (which is off by default).

Nov 21:

Starting to do another round of cleanup on the configuration system. Need to de-nest it to make it it's own Moose object. For that reason, starting to move stuff out of $CONF->{system} to just $CONF. The guest_user piece is done. It's a bit of a snarl since it's literally a 6 step process to move it over.

Nov 28:

All of the CONF hashref stuff is out of the codebase, replaced with a really convenient Moose object. Now working on getting some nodelet stats, so I can work to thin out the ones that aren't going to be supported anymore. We need to push them into templates/controllers now so that I can start to get the site to be moved over.

I've done a survey of how many of each nodelet are in active use by every profile on the site:

Nuked Tick Tock, as only 7 users were using it, and it is one of the old ecore nodelets

Nuked Server Statistics, as only 7 users were using it, and it wasn't actually representative of anything. It would get updated from the bastion host where all of the cron jobs run, so it'd always show artificially high load (as it would get evaluated in the middle of a cron job batch).

Nuked %$NODE because it was only used by a few retired coder gods, and I don't think we need to port it.

Nov 29:

Working through the codebase to wind down all the areas in which New Writeups - Zen existed. A transition layer was enacted inside of nodelet meta-container, but it hasn't gotten to everybody, so time to do some data manicuring.

Did some initial investigation work into data stash stuff. This is going to replace the way that the HTML gets stashed into the database inside of the nodelet code. I've long thought that the JSON parser (implemented in C) is probably considerably faster than my more performant, but still lacking Everything vars parser, which is stored as a CGI string. Running a quick benchmark, it's pretty damn obvious:

Put in place the start of the datastash code, which is a straight up replacement of nodelets, and eventually settings. newwriteups is the first such stash, now powered by a new cronjob which will execute all of the Everything::DataStash objects it finds in @INC. This will allow me to star to clean up these old jobs.