Monday, October 31, 2011

Please Excuse the Mess

It’s update time. The season is over and a slower news cycle is upon us. It’s time for me to roll up my sleeves and work on the site. First up is the transition of our Content Management System (CMS) from Expression Engine‘s 1.x series to its 2.x series. I had hoped to accomplish this upgrade last March but, unfortunately, some of the third-party scripts the site uses had not yet been updated to the new version, my personal life was much too busy and, frankly, I did not have sufficient funds to pay for a server upgrade and all the script upgrades. Although money is still tight, things aren’t as busy and all the scripts I need are now available. It’s time to get cracking.

The CMS software was almost completely rewritten in this update. As a result, many of their native and third party scripts, and quite a few of my own scripts, either need to be updated or replaced with something new. I’ve already rewritten my homebrewed scripts. Unfortunately a few third-party scripts have no direct upgrade path, so I will need to pay for a replacement or I will need to write a replacement myself. Luckily many of the replacements are actually superior, so in a lot of cases we’ll end up with better functionality as a result.

Getting BBTF updated is a rather large, time-consuming undertaking. I’ve done some trial runs on my development server and I must say the upgrades are not trivial and the process is often problematic. I’ve learned that the upgrade is better done in steps rather than trying to do it all at once by stringing together a series of upgrade scripts. It is also far easier to troubleshoot problems when the process is done piecemeal. This does cause a few problems because, with my schedule, some portions of the updates will need to be done over a number of days. Although I will try to make the transition as inconspicuous to you as I can, my tests suggest some things may not be available (like bookmarking, categories) for a longer time than I’d prefer.

As in most anything, planning makes perfect. In preparation for the conversion to 2.x, a few items (polling, for one example) will need to be disabled. To keep you apprised of problems, I am creating this thread specially for the upgrade process. Although most of the upgrades will be done in the wee hours, there may be times I will need to take the site offline for short periods during the day. I will try to post a notice at least 24 hours in advance of any planned outage.

With the size of the site’s database I have found less problems crop up during the upgrade when I reduce the number of fields in a few of the tables. Since changing the structures of some of the underlying databases also will make the upgrade of the archives and search features easier, I have decided to make most of these changes prior to the conversion. Again, from my testing, this changeover will, for the most part, be transparent. Nevertheless, I warn you that a few of the older/lesser trafficked blogs may go offline for short periods. I also warn you that the site will most likely be offline for most of the day I run the vendor’s conversion script. I will try to schedule this downtime in late November well before the Winter Meetings. Once again, I will try to give you sufficient notice to lessen the shock (especially for our most hardcore Primates).

Once the conversion to 2.x is completed I will incrementally update the site. Unlike past upgrades I will try to keep the site’s architecture about the same and will simply incorporate new features into the existing site design. My plan also includes upgrading the site’s servers after the holidays. As I mentioned above, I had hoped to upgrade the servers last March but unfortunately with a wedding on horizon, I didn’t have the time or money to get it done. In the interim I was able to tweak the server configurations to get the site’s performance up to a passable level. I fear, however, with some of the features I have planned, our current servers will not be up to the task. I will do my best to get our servers upgraded this January.

Anyway, with less stuff going on in my personal life this winter, I hope to use my time wisely to make the site more useful for next season. I hope you will find my additions worthwhile. Thanks for your patience through the process.

Jim

UPDATE: 12/9/2012 - The updates are underway. Due to requests from the Hall of Merit regulars, some of the updates were postponed until after balloting

Reader Comments and Retorts

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

Speaking of late November, I believe the voting for the Hall of Merit election closes on November 28 (assuming it doesn't get pushed back to try and draw more voters). So that would not be a good day to shut the site down.

* Customizable hot topics bar (automatically include/exclude threads by topic, for example).
* Mobile features (possibly dedicated mobile apps to handle certain use cases)
* Improved performance (caching, etc.)
* An API to “open up” the site’s data to third-party developers (i.e., the user base). This might be the best way to get mobile apps, by the way.
* Improved lounge interface
* Cleanup of generated HTML so that it is much more modifiable/customizable/scriptable on the client side

As usual, I’m very tempted to offer my time, but can’t really pull the trigger, as I know I’ll under-deliver. Maybe if a few of us build some critical mass to volunteer, that would have a better chance at success.

I still believe there’s a great opportunity for growth by expanding into the other major sports - BBTF is a fantastic one-stop shop for baseball fans, and it’s a format I don’t see anywhere for basketball and football. FBTF and BKTF, anyone?

Jim, I think the number one issue with the site are these constantly arising periods where performance goes all to hell and page loads take forever. It seems to me that the top priority should be moving away from you personally maintaining a server to virtualized hosting.

Letting third-party developers get in and add features seems like a great way to add the bells and whistles.

Your upgrade path sound very good and well thought-out. I agree with the sequence of changes you have in mind and your reasoning. Letting us all know that this upcoming month may have the occasional squirrelly day or feature is a BIG help - I won't get frustrated; I'll just wait a day. Oh, and I am willing to offer help, but I may be too rusty for you. The last time I wrote code for a living was in 2000. If you would like help, let me know what tools you are using, and I'll let you know if I recognize any of them. I wouldn't have offered at all, but you mentioned tables and not relational DBMS. I've never written relational, but I have LOTS of experience with table structures. - Brock Hanke

My biggest annoyance is the cluttering up of hot topics with off-topic discussions. I do appreciate that there's a catchall soccer and basketball thread, but they seem to be continuously on the sidebar. Plus there's the occasional (ha) digression thread that I am not interested in. I would like an ignore function for topics, right up by the Bookmark link. I don't think an "ignore by topic" would work since people don't take the topics seriously when they submit the links.

1. BBTF tends to be inaccessible for about 30 minutes each night starting around 1:00 AM. If that can't be eliminated altogether, is there any way to schedule it for later, e.g., 4:00 AM, long after most people are offline for the night?

2. A longer Hot Topics sidebar and/or a sidebar that watches topics in which one has commented would be great. Hot Topics tends to be dominated by that day's threads, while some older threads remain active but fall out of sight due to less-frequent comments. I know the Bookmarks feature offers the capability to "watch" a thread, but bookmarking every topic in which one posts seems redundant.

Again, thanks for your time and effort here. Good luck with the upgrades.

Count me among those who would most like to see either a longer Hot Topics or to keep hot topics free from the OT stuff. I think it would help facilitate more discussion of baseball topics if more of those threads were readily visible and accessible.

Count me among those who would most like to see either a longer Hot Topics or to keep hot topics free from the OT stuff. I think it would help facilitate more discussion of baseball topics if more of those threads were readily visible and accessible.

A longer or customized Hot Topics bar would be great, but since half the "baseball" threads are either pinata posts ("Roger Maris should be in the Hall of Fame"; "Sabermetrics? What's That?"; "Cliff Lee Has No Heart"; "Murray Chass sez....", etc.) or the same topics over and over ("The Wild Card Is An Outrage"; "They're Screwing Ron Santo!"; "Jeff Francoeur Spotting!"), I'm not so sure that the overall quality of the site would improve. The best baseball articles are the ones linked to Hardball Times and similar blogs, and they usually wind up dying after a handful of comments, whereas the Jeff Francoeur fan club apparently can't get enough of him.

bbc, years ago I vowed to put my money where my mouth is. When change is in sight, I'm good for my word.

My biggest annoyance is the cluttering up of hot topics with off-topic discussions. I do appreciate that there's a catchall soccer and basketball thread, but they seem to be continuously on the sidebar.

As to this and the other comments about OT threads - this is why I'm lobbying for an expansion of the Think Factory universe. BBTF has an accessible format I believe would work quite well with other sports. When that happens, then people can go argue NBA on BKTF, which helps BBTF stay baseball-focused.

My biggest annoyance is the cluttering up of hot topics with off-topic discussions. I do appreciate that there's a catchall soccer and basketball thread, but they seem to be continuously on the sidebar.

I'm a regular in the soccer thread but I'll second this. Now that there is an "OT" designation on those I think that should be a separate sidebar to itself. In other words you'd have "Hot Topics" then below it (or wherever) "Off Topic".

I don't think a STF or BKTF is necessarily going to start here. One of the things that is great about this place is that we have created a community here. I am genuinely interested in what people have to say and I find I often learn something in off topic discussions. The soccer thread for example generates about 700-800 comments a month so we're talking 25 a day, not a lot and it's more a function of "hey, within this group of people I share a baseball fandom with there is a subset that enjoys soccer" and that creates a nice diversion. I don't really want to go to an "all soccer all the time" site because my interest is not that high.

I don't see why there is a backlash against stuff people are not interested in. At any given time three of the last ten baseball-related posts are not going to be interesting to me. I don't think the soccer/football/basketball/hockey threads really take up that much space. This time of year obviously they get a bit more attention but the fact is there is not that much baseball related to discuss right now.

I just took a thorough look at my BTF page, and the current Hall of Merit discussion thread isn't there, anywhere. I'm pretty sure that's not intentional, so I'm noting it here so that whoever is in charge of such stuff can fix that. If we want to do things like recruit new HoM participants, that thread needs to stay visible throughout the process. - Brock

Is it possible for you to control the ads that are shown on the page? In particular, the ads for some game or something called Palisade Guardian cause my mouse cursor to flicker on the screen and it is a tremendous annoyance.

Is it possible for you to control the ads that are shown on the page? In particular, the ads for some game or something called Palisade Guardian cause my mouse cursor to flicker on the screen and it is a tremendous annoyance.

I came to this thread to mention the same thing. The cursor flickers not only on this site, but on all other open Firefox tabs as well. I usually have to refresh 4-5 times to get an ad that doesn't cause that.

I came to this thread to mention the same thing. The cursor flickers not only on this site, but on all other open Firefox tabs as well. I usually have to refresh 4-5 times to get an ad that doesn't cause that.

Firefox's plugin container is an abomination. It's still leaps and bounds better than MSIE, but I'm starting to lean more towards Chrome because I'm tired of manually killing the damn thing.

Oh - and just in case any web ad folks happen to be reading this - you should know that I and many others continue to lobby for making usage of Flash a capital offense... I'm neither an Apple zombie nor an Apple basher, but Steve Jobs' pathological distaste for supporting Flash, I consider his greatest legacy.

I don't see why there is a backlash against stuff people are not interested in. At any given time three of the last ten baseball-related posts are not going to be interesting to me. I don't think the soccer/football/basketball/hockey threads really take up that much space. This time of year obviously they get a bit more attention but the fact is there is not that much baseball related to discuss right now.

I'll emphasize that I don't have a problem with the off topic threads. I just don't want them cluttering up the Hot Topics. Having an Ignore similar to Bookmark would work fine for me since many baseball threads that are properly tagged will turn into off topic posts. So unless the tags can be changed if the conversations veer off, it wouldn't help. And I don't think that kind of maintenance is practical anyway.

#10 I'd bet serious money that this is the backup. Databases and backups are at minimum tricky to manage, and I suspect that whoever is doing it wants to be able to keep an eye on the site (since it's probably shut down the database, back up anything that's changed, re-start the database. And sometimes those restarts fail)

There are products that can back up live databases but they're pricey. I'm a late nighter too but I can live with the way it works now.

The backup of the database (specifically the comment database) is what causes the problem around 2AM. The issue will be minimized somewhat by a database server with more memory and faster processors. I will look into alternative backup solutions if that doesn't make the problem manageable.

On my To-Do list is customization of the Hot Topics menu. Logged in members will be able to select from a few different default snapshots (no off-topic, personalized, etc.). I will also try out some Ajax powered/iframe solutions. (A customized Hot Topics would remain on the page and would automatically refresh itself every few minutes.)

I will also be improving the bookmark system. Bookmarked threads would automatically be added to the personalized Hot Topics. Members will be able to update their Hot Topics preference using Ajax right within Hot Topics, which means you'll be able to switch between different snapshots at any time.

A rating system will also be added. Members will be able to rate the linked article and comment thread in a number of ways. You'll then be able to display the top rated articles/comment threads. It's important to me to make the rating system as easy to use (and unobtrusive) as possible so that the people who are interested in rating content (or seeing rated content) can do it easily, while anyone who is not interested can either ignore it or, if it's your preference, hide it altogether.

In any event, I am open to making site improvements.

I've been doing some of the backend work that needs to get done before the EE upgrade. So far, I don't believe what I've been doing has even been noticed. Hopefully that will remain the case.

I came to this thread to mention the same thing. The cursor flickers not only on this site, but on all other open Firefox tabs as well. I usually have to refresh 4-5 times to get an ad that doesn't cause that.
Firefox's plugin container is an abomination. It's still leaps and bounds better than MSIE, but I'm starting to lean more towards Chrome because I'm tired of manually killing the damn thing.

Just to be clear, I am using Chrome and still get the stupid cursor flickering from that ad and it also affects all other open tabs in all Chrome windows. Thankfully as of today, that ad doesn't seem to be around.

I will also try out some Ajax powered/iframe solutions. (A customized Hot Topics would remain on the page and would automatically refresh itself every few minutes.)

Thanks for the great work, Jim. If I could make one small, technical suggestion regarding bookmarks:

I will also be improving the bookmark system. Bookmarked threads would automatically be added to the personalized Hot Topics. Member will be able to update their Hot Topics preference using Ajax right within Hot Topics, which means you'll be able to switch between different snapshots at any time.

Is it possible to move the location of the "DEL" link used to delete a thread bookmark? It's currently aligned directly below the "last thread page" link, and the close proximity of the two has lead to accidental deletions on more than one occasion. Even shifting the "DEL" link so that it is aligned flush-right on the same line would be a great improvement from a usability standpoint.

The more integrated HTML5 framework is the future -- I just hope browsers keep pace so you can easily disable intrusive or resource intensive scripting that people will inevitably be popping directly into pages. Otherwise -- in a year or 3 -- we're going to back in the old GeoCities world, but with more multimedia rather than just ugly aesthetics and CSS problems for a temporary bit.

Zonk, thanks for the HTML5 tip. One part of the site which certainly needs an overhaul is the design. It's the one part of the process that I dread, however as it's incredibly tedious and acts like a black hole for my time. This site's design is, at least, five years behind in usability.

One part of the site which certainly needs an overhaul is the design. It's the one part of the process that I dread, however as it's incredibly tedious and acts like a black hole for my time. This site's design is, at least, five years behind in usability.

Gold Stars, an errant system process, that could not be killed, was running on the web server and using 100% of the processors. To fix this problem, I had to immediately restart the server. Unfortunately, the server did not come back online immediately and the data center was not prompt in fixing the problem which caused the server to not respond.

"Code" tab: preserve format, like for a table or something. Usually supposed to represent actual programming code that isn't to be executed, but the table format thing is what it's used for mostly around here. For example:

a b c1 23 4567 8 9

"@" allows you to put in a link to an e-mail address. .(JavaScript must be enabled to view this email address) Rarely called for here.

But if you put "weddingtop" on ignore, all you'll see is "[ Ignored Comment ]" instead of those dime store jewelry links under his name, which would probably crash the computer of anyone dumb enough to try to open one of them.

Jim, was this for me? Clearing cookies didn't help. It's possible that I have to log in when I switch devices. So I'm on my iPad and can get in without re-logging in. But if I go to my PC and log in there then the next time I use my iPad I have to log into the site again.

I fixed the width of the edit box. Can you let me know where the edit box isn't working? It might just be I haven't updated some of the other templates, which is why I'm doing some work consolidating embeds.

Currently you can edit comments until the thread is closed. EE now supports comment editing natively. They seem to have a bug in the timely code. Hunting down and fixing the bug is on my to-do list. I'd describe the time issue as slightly annoying and not hilarious, however. :)

What is the "del" key? Are you talking about one of the styling buttons about the comment box? If it was, tell me what the function was and I will add it back.

EDIT: I have found the bug in EE's code regarding the time limit for comment editing expiration. Comment editing now expires after 15 minutes.

Aside from restoring the bookmarks, which has already been addressed, it'd be nice to have the "strike" function restored above the comment box. It's a low priority, but sometimes it adds a bit of flair to those "FIFY" comments.

That would actually be great. (Even better if the link opened up a separate browser window.)

I'm forever having to take separate time to access a player's b-r page during a discussion.

Agree with this.

Aside from restoring the bookmarks, which has already been addressed, it'd be nice to have the "strike" function restored above the comment box. It's a low priority, but sometimes it adds a bit of flair to those "FIFY" comments.

For Andy, the strike function still works, if you really want it. I agree with preferring to have it on the top of the comment section instead of having to manually put it in. (I highlight what I want to strike choose italic bold then edit it to del.)

Sean, is there any way for you to make b-r show just select categories on demand (such as year and OPS+, or year and OPS+ and WAR)? Most of the time when I'm listing a player's year and OPS+ here I'll have to switch back and forth between the websites on my browser to enter the data, or print out the player's b-r page and enter the data in.

It would be easier if I could just select to only show year and OPS+, and then be able to just cut and paste that data into a BBTF window.

It would be easier if I could just select to only show year and OPS+, and then be able to just cut and paste that data into a BBTF window.

Go to the top of the table you want, click share, select X (x inside of an arrow will delete every category right of the x)on the categories you want to eliminate, then hit close(yellow box says sharing toolbox)

and of course if you click pre, then copy and paste it inside of code, it should come out right. Just don't make the table too big or else you'll piss everyone off. (should work without using pre just haven't tried it that way)

Sean, I have added your suggestion to my to-do list. I should get to it some time in the next day.

CoB, there is a 15 minute time limit. I've just moved to EE's built in code for the edits. I put in a feature request to add an error message when the timeframe is exceeded. Can you verify that the text gets updated if you try to edit your comment right away?

[70] of course if you click pre, then copy and paste it inside of code, it should come out right

When I click PRE and then copy and paste inside of code, the spacing comes out wrong in the preview and when posted. By wrong I mean multiple consecutive spaces are treated as one. Here, the player column label and the player names should be aligned right-justified, with varying numbers of leading spaces:

When I click PRE and then copy and paste inside of code, the spacing comes out wrong in the preview and when posted. By wrong I mean multiple consecutive spaces are treated as one. Here, the player column label and the player names should be aligned right-justified, with varying numbers of leading spaces:

What I don't get is why your copy and paste inside of code is not blue like my copy and paste inside of code?

My example above is where I selected pre on bb-ref, then highlighted the rows and columns I wanted, copied, then came here clicked code, then pasted in between the code.
The rest of the text in the comment is outside of code which is why it's not blue, not sure why your code is coming up with normal text color.

I have to say, and this is a relatively minor thing, that it's a bit strange to have "You are ignoring [XXX poster]" instead of, "[Ignored Comment]", in part because I end up thinking that certain posters keep posting that they're being ignored.

It's more than just a minor thing, because the new formatting makes the ignore message appear to be an actual post. Plus, it takes up more space than a real one-line post since there seems to be an extra set of p tags around the "You are ignoring X" text. The previous formatting worked well because the italicized text was easily recognizable as something one's eye can skip over as an ignored comment. Can the ignore tool return to using that previous format?

I'm amazed by the popularity of the "ignore" feature. I might understand it at sites with no adult supervision, but I've yet to run across anyone here whose opinion I never want to see (or whose opinion is unlikely to ever advance the discussion).

Why not just skip over posts you wish to ignore? It works fine for me.

You can; this just makes it easier.

I have very few people on ignore; I reserve it only for those who really do piss me off to the point where I cannot take reading them anymore. It's usually less about stupidity and more about pure dickishness.

You don't have any patience, Andy. I said they would be back in about a week. If you look above, you will find a "Bookmark" link. Although I want to gussy up the Ajax a little, that part is just about done. I still need to finish the bookmark Hot Topics, however. I want to add some additional features to the code I will be using for the new bookmark's Hot Topics so I can reuse it for other things, like a Hot Topics for the entries you commented on.

So any chance you could fix the login issue? I think it appears to be a problem with logging in on multiple machines as I now have to log in every morning on my work machine and then when I go home, I have to log in again at home. It's minor, but rather annoying.

You don't have any patience, Andy. I said they would be back in about a week. If you look above, you will find a "Bookmark" link. Although I want to gussy up the Ajax a little, that part is just about done. I still need to finish the bookmark Hot Topics, however. I want to add some additional features to the code I will be using for the new bookmark's Hot Topics so I can reuse it for other things, like a Hot Topics for the entries you commented on.

Yeah, now that I look at your e-mail from the 3rd you did mention "a week" towards the end of the message. In any case thanks for the information, which is all I was really asking for.

So any chance you could fix the login issue? I think it appears to be a problem with logging in on multiple machines as I now have to log in every morning on my work machine and then when I go home, I have to log in again at home. It's minor, but rather annoying.

This. I think it may be a multiple machine thing, too. I read BBTF on my iPhone, iPad, work machine, and home machine, alternatively, and it seems to happen when switching. But whatever the cause, it's very annoying.

Yeah, I posted in #40 and I still seem to be having the problem. I've stayed on one computer for the past 2 days and haven't had to log in. But when I switch to my iPad I have to log in there. Then when I come back to the PC, it makes me login there again.

This is weird...hot topics says this thread has 102 posts, but I can only see up to 99. Even if I click on the "Last" -- which theoretically would take me to the second page -- I still only see one page and 99 posts. (And the URL correctly has the /P100/ on the end, too.)

EDIT: hot topics now says 103, and lists my post as last...and I see 100 posts instead of 99.