What Is Happening To The jQuery Plugins Site?

For about a week, instead of hosting several hundred jQuery plugins and several thousand advertisements for laptop batteries, our plugins repository has been serving up a pretty pathetic message about spam and an allusion to some “new submission process.” This happened very suddenly, and we’re sorry to everyone who’s been inconvenienced. Please allow me a few minutes to explain what happened, where we’re headed, and how it impacts you. If you’re in a rush, here’s the short version.

The Backstory

Though the plugins site you’ve known and loved was a valuable tool when it was first set up, it gradually became something of a white elephant for the project. While powerful distribution tools like GitHub and npm have come to the fore, we’ve been stuck in an aging, CMS-oriented paradigm that frustrated developers and consumers of plugins alike. Many people moved onto alternative sources for finding and vetting plugins. Furthermore, the sites’s original implementors and maintainers had since moved on from active involvement within the jQuery project. While the team faced a steady stream of complaints about usability and a general lack of features, the site itself faced a veritable barrage of (several flavors of) spam. There was your standard, keyword-rich SEO garbage, but there was also something slightly more insidious: the batch posting of under-documented, demo-free plugins with links to paid download sites by third parties trawling for affiliate cash. While this wasn’t explcitly disallowed, it led to a terrible, confusing experience for users and gave the site the distinct sense that all was not on the up and up.

The Best Laid Plans…

Sensitive to all these problems, the team began drawing up requirements for a complete overhaul. We knew we wanted to get out of the business of accepting uploads, serving downloads, and generally legislating the plugin “release” process on our own servers. With our official projects already on GitHub, we knew we wanted to leverage the many wheels they’d already invented for distribution, versioning, and facilitating open, collaborative development. Another requirement was to create a standard schema for authors to clearly delineate dependencies, like which version(s) of jQuery a plugin supports, as well as other plugins and CSS assets it requires to be functional. We also wanted to introduce some quality control, with both passive mechanisms like requiring GitHub and a manifest file for distribution, as well as active ones like user ratings. Finally, we wanted a clean slate; with stricter requirements for submission, there could be no mass import of all the old plugins. We made a lot of progress preparing specifications, but hampered by a lack of resources and a number of other projects, we never were able to get too deep into the implementation phase. After all, the old site “worked!”

…Often Go Awry

As the glut of spam grew worse and multiple reports started showing up on the jQuery Core bug tracker, I wanted to at least take some steps to wipe a bit of the spam and egg off our face. With newly provisioned access to the administrative tools on the site, I teamed up with the Drupal Views Bulk Operations module and set out to identify and delete spammers and their posts. Within a day or two, nearly 90% of the alleged “content” was gone from the site. I continued to monitor the situation over the ensuing days, and deleted spam as it came in. Unfortunately, I likely cast too wide of a net, and threw out several perfectly good babies with the bathwater. Even more unfortunately, I didn’t back up the database before I began this process. At this point, you can probably see where this story is headed. Later in the week, while I was attempting to delete four spam items, I was left completely horrified when the results of the operation reported that the remaining 10% — every single plugin remaining in the database — had been purged. All that remained was a year-old backup. Of course, I realize that this is a dreadful outcome, and I take full responsibility for it. If it helps, I am very receptive to hate mail and tweets reprimanding me for being irresponsible, unprofessional, or just stupid.

At this juncture, we were left with two choices.

Keep on kicking the can down the road: Restore from the old backup, losing a lot of data anyway, and have authors go through the painstaking process of re-uploading their plugins, even though the site was slated to be deprecated entirely anyway.

Use this glaring mistake as the impetus to hunker down, cleave from the past, and finally implement the site we’ve talked about for so long.

We’ve chosen the latter.

Nuts And Bolts

Over the past few days, we’ve started converting our plans into action, building out an infrastructure that’s backed by GitHub. There are two requirements for listing a plugin on the new site:

A valid package.json file
We’ve followed the lead of CommonJS and npm and created a schema for specifying dependencies, delivery, and other metadata of jQuery plugins. While the format is largely similar to those other projects, we’ve had to make some minor tweaks to account for some plugin-specific details.

At least one versioned release
This means having tagged your release point(s) with a valid semantic version number (semver) string.

We’ve pared down the submission and maintenance process to a single, one-time step: adding a post-receive hook to your plugin’s GitHub repository. Assuming your plugin meets the guidelines, a page will be created on the plugins site to present your usage and download information. We’ll keep track of new releases as you push them.

In The Interim

We recognize that the old site was still serving as an regular resource for a lot of people, especially newer jQuery users, who simply valued the existence of a central browsing location, despite its flaws. Though many experienced users had moved on to other sites, or relied on relationships with trusted authors and word of mouth, these can take time to develop. Until we’re able to launch the new system, we’re happy to direct you to several other directories and people who can help pick up the slack:

If, in the process of searching these or any other directories, you are directed back to the current site at plugins.jquery.com, typically a quick search for the author’s name and the plugin name will yield an alternative site where it was hosted. In addition, DailyJS just did a useful roundup of alternatives that goes into more detail.

Next Steps For Plugin Authors

If you’re a plugin developer who wants to make sure your plugins will be ready to go on day one (or even beforehand), you’ll want to make sure your plugins are up on GitHub, and then you’ll want to get started on creating your package.json files and making sure your versions are appropriately tagged. If you aren’t already familiar with Git and GitHub, then this is probably a very good time for you to take the plunge and get started. If you prefer another SCM system, you can look into setting up a mirror to git from svn or hg, or other project hosting sites, like bitbucket or gitorious. If you don’t develop your plugins as open source, or you don’t use any source control at all, we will not be able to accomodate you at this time.

How Can I Help?

Though the site is still a work in progress, you can track progress and even set up your own local development right now at github.com/jquery/plugins.jquery.com/. We’re still working on getting a public staging environment together.

Though the site is essentially “powered by” GitHub, there are obviously a number of moving parts in play. The site will actually be served from inside of WordPress, which is populated with the contents of the indexed plugins using a node.js tool. There are still many kinks to be ironed out, so if that sounds like a stack you want to jump on, please join us over at the repo. We’ll be using GitHub Issues for all feature requests, bugs, and discussion.

While we’ve put a great deal of thought into putting together our package.json schema, it’s still a living document, and if you have questions or comments on why it is the way that it is, head on over to this issue.

Errata

In an ideal world, this certainly wouldn’t have happened exactly as it did. Sadly, it did. We hope you’ll accept our apologies for the many ways this transition might make your job harder, give you some extra work you weren’t expecting, or just plain leave a bad taste in your mouth. If you are able to forgive us, bear with us, and maybe even lend a hand, we’re confident we’ll be able to deliver the modern, useful plugins site the community deserves.

TL;DR

So that was a lot of words, here’s the takeaway.

We’d been planning on replacing the original, spam-ridden plugin site for quite some time

In the process of deleting on the spam, all the plugins were deleted and we didn’t have a recent backup

Instead of burning cycles keeping the old site on life support, we decided to make a clean break and kick development on the new site into high gear.

This is a great site and I’m very sorry such thing happened. That being said, not having backups of the database was very, very stupid (I hope that never happens to me, karma’s a bitch). Anyway, what’s done is done. Good luck, I’m looking forward to the launch of the new site.

I’m going to echo some of the recent comments – do you have a timeline for the new plugins site? I’m excited to see it, or at least a rough version of it. Given the holidays this time of year, no big rush, but just wondering about the schedule.

I once “accidentally” overridden clients old site with a new one without backup as a first demo. Boss chewed me out for a bit, but the client was happy with the new site and didn’t mind. Similar situation here as I understand it ;)

I’d go on and say it was a good turn whether accidental or not, but I’d make sense to see the new plugins site up and running.

The fact that you’re so open and honest about what happened says a lot. You could have so easily spun that and made it NOT your fault but you took it full brunt. You’re human, we do make mistakes. I love what you guys have done here. jQuery, the plugins, the UI components all of it is just killer and I thank you for it. Don’t fret too much about it, as the end result will be much better! I so understand how much it sucks to lose a database that is extremely important, I’ve been there myself. I bet if nothing else you ALWAYS backup you DB first! :)

I recently deleted a site ‘accidentally’ Being lazy and using C-Panel I selected the wrong site – should have been a temp site setup for testing…
At least there was a current backup database and files – only had to upload the files, restore the database, and replace a few images.

Good! The plugins search was horrible on the old site. I had to go through google to find anything relevant. Don’t hate on Drupal it is what you want it to be, but you have to use the correct tool for the job.

The old site wasn’t perfect but it did give developers a central point to show off their creations and for users to see what new features have been produced. The Battery ads were a pain but not as frustrating as the search feature BUT it was still the only way to sift through all these to find the gem you needed.

Any idea when something similar (the listing not the Battery Ads) will be back on-line again?

The plugins are the life-blood of jQuery presenting the latest developments in the product. They keep user interest at a high level and allow developers to implement often complex procedures much more easily.

While the new site may be wonderful, something is needed now so that us web-site developers can continue to provide our organisations with product. If jQuery is continue to be taken seriously in a professional world it can’t just say “No product for the time being” with no interim arrangements or target date for completion.

I’m sure you are all working hard and will come up with something that is much better than before – but – we do need something in the meantime (ie now) – please – please – please !

Does your blog have a contact page? I’m having trouble locating it but, I’d like to shoot you an email. I’ve got some ideas for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it expand over time.

@santo Im serious about jQuery, and I am not happy to keep on waiting for several months until the new website is ready. I understand that it was a mistake, and I really hope that no one will get fired, but it is being too long, dont you think so?

jQuery is such an important resource nowadays, that two months wainting for the plugins repository I think can hurt jQuery itself. Maybe more resources should be used in order to speed up the website development.

Im sure the new website will rock and will be much better, but I really hope to see it soon!! :-)

jQuery is an excellent product and is well supported by a dedicated group of individuals. Someone made a mistake – not nice but we have to live with it and move on.

The plugins are what makes this product great – If they didn’t exist it would be a less useful product. It is amazing to have a wealth of powerful, compact and tested appliactions. Except that at the moment we don’t have access to that set of applications. What’s more we don’t have a date for when we might have them…

If someone is working on this can we have a probable end-date, please?

It sounds as if you didn’t value the Plugins site that much Bruno! The rest of us did (even if there were spam items and out-of-date items one could with little trouble skip over them) and used it extensively. The attitude that `its not there, so what?` must be very demotivating for the developers, although not as obnoxious as the `sack them all` brigade!

My feelings are:
(1) It is are that it is an integral part of jQuery. Not having a plugin site lessens the value of jQuery itself.
(2) I’d like it back on-line ASAP
(3) But above all keep us informed of what’s happening and when a new one will be available.

While I do use the vast majority of features in jQuery, I’ve never really used plugins much. I understand some users’ frustration in the glacial pace of the new plugin site’s development. Personally I wish that jQuery UI 1.9 would finally be finished, and I do find it frustrating to see how slowly it’s progressing. I also think an occasional update on the status of the plugins site would be nice.

However, it is crucial to maintain a respectful, level-headed jQuery community. Some of these comments are very rude and unnecessary. jQuery has expanded to include jQuery, plugins, UI, and Mobile as central projects.

If you’re unhappy with the pace of jQuery releases, I’m sure they’d appreciate having more developers to contribute towards the various projects’ development. While having more developers does not always speed up a project, I think in this case, it might be helpful – and it’s certainly more constructive than bashing the jQuery team.