The number one question asked at the summit of me was “Can I join the plugin review team?” I told everyone “Follow make/plugins and I’ll post there by [last] Friday.”

Sorry about the delay, we had coordination issues which ironically is why the current answer is “No, we’re not adding anyone new to the review team.”

State of Things

The way the review of new submissions is sorted right now, it’s a single-thread system. There is a single queue that contains all submitted plugins, and it can only be viewed by one person at a time – or we run the risk for two people reviewing a plugin at once, which can end with one rejecting and one approving simultaneously. In order to avoid this, we are constantly asking each other which of us is currently in the queue. Even then, the system is archaic and has issues. So yes, it’s entirely a technical limitation and it’s one we ARE actively addressing. We’ve all talked (in one-offs) about what we want and need, and we have it spelled out. A lot of this is because we were intentionally waiting for the inevitable bbPress 2.x upgrade, but since that’s not happening any time soon, we’re going to have to make an interim plan.

What We Do

But there IS a future where we will want more people to help out in various roles and it’s with that in mind I want to talk to you all about what we actually do.

Review New Submissions

This means we download a submission, check it for any violations against the guidelines, test it on a sandbox of our own, and make sure there isn’t anything egregiously wrong. We also have to check for licensing and trademarks, which leads to fun things like the time I rejected the Official Facebook plugin because they used a gmail email address and a dropbox URL for the zip.

Right now, the check is 100% manual. We’re developing a Plugin Checker (like the Theme Checker) but it’s much harder since themes are pretty standard when you compare them to how crazy plugins get. We have, finally, boiled down to what we know we can auto-reject and what we need to warn/inform people about, so we’re making progress on that end.

One thing we don’t do is put our own feelings into a plugin review. If the code is good and there’s nothing ‘morally offensive’ about it, it comes in. That’s why we have a bajillion twitter plugins. Determining what is and is not offensive is hard, though. We don’t allow things we determine to be black-hat SEO (“This plugin will improve your SEO by 1000%!”) and we don’t allow things we feel would be detrimental to the community, but we do allow things we know will offend some people. It’s a fine line.

Handle Guideline Violations

Every single email you send to plugins AT wordpress.org saying ‘So and So’s plugin puts in powered by links!’ has to be verified. Usually this is easy, but once you report one user, we check all of their plugins. This can take a while and it gets worse when we get a submission like “Joe’s twitter plugin emails him when installed!” Sounds easy, right? Go on and figure out how many twitter plugins that might actually refer to. I reply to those a lot and ask “WHICH plugin? Please link to the repository page.”

Everything we do in the guideline violations has to be done here, but worse, we have to reproduce the bug and give suggestions/information about possibly fixing it. Why? Because not everyone actually understands why they have to sanitize, or why their plugin which we approved 4 years ago, calling wp-load.php directly, needs to remove that now. The guidelines and standards change over time, and while we don’t expect people to keep up with them 100%, when they do change, it’s a waste of time to argue with us that they changed… Which bring us to the number one thing we actually do.

Be patient with angry people

If you’re not good at handling support tickets or forum posts, I have news for you. You will not survive the plugin team. Getting sent the dread “Your plugin has been removed…” email is possibly the worst day for a plugin developer. It’s earned us a lot of anger from the community, from people who feel we single them out or that we specifically hate them. We don’t.

Just because you’re the most awesome person when it comes to reverse engineering security issues and solving them doesn’t mean you’re great at explaining to people why they can’t phone home or why something that was okay 4 years ago isn’t now, or even teaching them how to fix an issue even if it’s not actually our responsibility. And yes, people absolutely lose their minds to the plugin team fairly regularly. Buy me a coffee, I’ll tell you about the guy who tried to impersonate me by sending emails ‘from me’ telling other devs their plugins were removed, because I’d closed his.

The point here is that we really need people who either are great communicators from day one, or who are comfortable asking for help when they know someone’s gone off the rails and can’t be reasoned with by them. If you’re this guy, you’re not ready:

So … now what?

Well now we just want this post to get you all on the page we are. And we want you to understand that until we fix the technical issues, we can’t actually address the training people up to help out. I promise you, I’m just as riled up about not having more people on the team as you are, because right now if two of us go away for a week, we have a massive queue which is just depressing. Trust me, we’re all in agreement here. But since they won’t let me reboot the plugins directory, we’re going to have to take this seriously and careful, and I beg of you to be patient.

And that’s what we need most of all. Be patient. Stick around here. Be understanding. Don’t nag. Seriously, that never helps. We know who’s interested, and maybe we’ll come up with some quizzes and tests to see ‘Would you approve this plugin?’ and sort folks out even more. But it’s not today, and it’s not because we don’t want more people. It’s because more people would break a broken system worse.

Because not everyone actually understands why a plugin which we approved 4 years ago, calling wp-load.php directly, needs to remove that now.

The guidelines have evolved over time. We change and fix and evolve and we do not usually go back. So if no one takes the time to email plugins AT wordpress.org and tell us, we probably don’t know. Now that I do, I’ll go handle that one.

We’re also not being overly naggy about tags at the moment, though that may change.

A plugin of mine has a php file that may be loaded directly via an AJAX request. It loads wp-load.php to get to $wpdb and some other stuff. I noticed you mentioned that’s not ok anymore? What’s the alternative, I can’t seem to find a conclusive answer.

Including wp-config.php, wp-blog-header.php, wp-load.php, or pretty much any other WordPress core file that you have to call directly via an include is not a good idea and we cannot approve a plugin that does so unless it has a very good reason to load the file(s). It is prone to failure since not all WordPress installs have the exact same file structure.

Usually plugins will include wp-config.php or wp-load.php in order to gain access to core WordPress functions, but there are much better ways to do this. It’s best if you tie your processing functions (the ones that need but don’t have access to core functions) into an action hook, such as “init” or “admin_init”.

If you’re trying to use it because you need to access WordPress functions outside of WordPress, we’d actually much rather you didn’t do that at all. Your plugin should be inside WordPress, only accessible to people who are logged in and authorized, if it needs that kind of access. Your plugin’s pages should be called via the dashboard like all the other settings panels, and in that way, they’ll always have access to WordPress functions.

Sounds like maybe you’re fighting with the wrong tools. I’d imagine something like HelpScout would be beneficial here — a queue of issues, reviewers who can self-assign as they pick one off the queue, canned responses, notes, reassignment / unassignment. Even hooks for magic like auto-scanning zip files perhaps. I say HelpScout but basically anything similar. Surely better than bbpress.

Such things cost something, but what price your own time? Who pays might be an issue, I don’t know, but I’m sure sponsorship wouldn’t be hard to find. Even “reviewer pays” where the reviewers stump up the $15/month for the privilege of putting WordPress Plugin Review Team (if you see me running, try to keep up) on their T-shirts.

The real issue we’re facing is integration with the system, otherwise we’re right back to the same issue. Multiple people in the queue at once == disaster. Until we fix THAT, everything else is theory. We already have notes, canned replies, and actually a queue. We just have it single-threaded. We need to fix it

Also we will never ever EVER ever ever consider ‘paying’ for reviews. If you see me tweet about how I’ll review faster for Smarties or Kinder Eggs, that’s just me being a goof. No. No monetization of this. EVER. That’s just the antithesis of the repositories.

Sponsorship for anything review related starts coming across like bribery to a lot of people, even with the best intentions. “I give money/time, I should get precedence in reviews!” We don’t even review our own new plugins first!

Thanks for the really thorough explanation. I’ve nagged Pippin a few times about joining the plugin review team, but now that I know all this I’ll stop nagging and wait patiently I’m really keen to help out, so looking forward to when it becomes possible.

9 people voted. Results: Scott Reilly as first lead, Pippin Williamson as second lead. New team rep terms starts with the new year, so I’ll get in touch with you guys to make sure everyone is on the same page re expectations. Congratulations, and thanks for your willingness to serve!

Time to vote for team reps again! If you haven’t seen the spiel on one of the other team blogs about how team reps/voting/terms work, the longer explanation is after the jump. tl;dr version: time to elect reps for the first half of 2013. This past time it was Mark Riley and Scott Reilly, but since then Mark has stepped back from heavy involvement with plugins so you need at least one new rep.

Note: It can’t be folks who are already the team reps for other teams, and it should be folks who want to the responsibility (mostly posting weekly updates on team activity to the weekly updates blog). Since there are some newer members of this group it might be nice for one of them to level up and learn the ropes from Scott? Up to you guys. Anyone interested in being a plugin team rep should leave a comment saying as much so people know who they can/should vote for. Voting is open until December 15, and results will be posted here once voting closes.

Well, what’s more important is which side you’d rather represent. If you’re more interested in being a plugins team rep, someone else could step up in forums. For that matter Scott could totally switch to meta team rep with Otto, too, if he’d rather.

For those who don’t know, I’m currently the only active rep for the team.

I am more than happy to continue in the role. However, there is also now the first official voting for reps for the meta team. Most likely Otto would be the de facto lead for that, continuing in his acting lead team rep position. That team doesn’t have very many candidates for potential reps (basically Nacin and I asides from Otto). If all the criteria Jane set forth are followed, it stands to reason I should be the secondary rep for the meta team, requiring me to relinquish my current status as a rep for plugins.

Since Mika is most likely continuing on as a rep for the support team (at least for this term), Mark seems to have stepped back from involvement with the plugins team, and Nacin will likely continue as a team rep for core, that pretty much leaves Pippin as a prime team rep candidate.

I’m cool with continuing on as a plugins team rep or moving on to the meta team, depending on where interest from others lie. If I remain we can prep Pippin as a secondary rep and follow Jane’s desired rep transition procedure, though leaving Otto as the sole meta team rep. If I switch teams, both plugins reps will be new and one will be simultaneously joining the team and becoming a rep.

That makes the most sense, and is my preference as well, for the two of us to be the team reps. Then you can continue on in the senior role next term, to be joined by someone else (either a new member to the team or one of the others who may be coming off a team rep role for another team).