We’ve finally come to the end of Summer of Code, so it’s time for the final GSoC release in this slightly late update (hey, would it be a post by me if not?). This release is mainly a stability release, so there are only minor changes:

Add generic CPT class – Plugins are now encouraged to extendWP_JSON_CustomPostType and get free hooking for common actions. This
removes most of the boilerplate that you needed to write for new CPT-based
routes and endpoints (#380)

Use defined filter priorities for endpoint registration – It’s now easier to
inject your own endpoints at a defined point

Update the schema – Now includes documentation on the Media entity, plus more
(#264)

Add better taxonomy support – You can now query for taxonomies and terms directly. The routes here might seem strange (/posts/types/post/taxonomies/category for example), but the intention is to future-proof them as much as possible (#275)

The big feature for this final week is a bunch of documentation changes. I’ve created guides and documents on how to use the API, intended for both beginners and experts. I’d especially like to highlight the Getting Started guide, as well as the Extending the API guide for plugin developers. The documentation in the past has been less than fantastic, so I’ve made a concerted effort towards it this week. In addition, I’ve also fixed up all the remaining bugs reported via Trac.

Now that GSoC’s over, what’s next? The aim with the project is to now move it from a solo project to a team one, and to that end, I’ve been working on assembling a fantastic team to work on the project with, with aim to integrate the API into core in the future. 3.8 time is fast approaching, so we’ve elected to aim for 3.9 as a more realistic target, although the advantage of the Feature as a Plugin method of development is that we’re not locked down here.

We’re held two meetings so far as a team, and I’ll announce a proper office hours time next week, but I’m also looking to try something new with the organisation of the team. More to come on that in the next team update, but in the meantime, you can check out the internal team discussion site. Anyone looking to get involved in the team is welcome to join as always, but I’d ask that only those serious about working on the project join, as there are a fair few people committed already.

Thanks to everyone, especially my mentors and Jen, for making this project a joy to work on so far. Here’s hoping we can keep the momentum as we push forward with the project.

This week finally brings media into the fold. The process for uploading media is a little different to creating normal posts, so here’s how you do it.

First, upload the file via a POST request to /media. This can either be as a standard HTTP multipart body, with the name field set to file, or as a raw file in the body with the Content-Type header. (You can also optionally send a Content-MD5 header if you’d like the server to check the consistency of your file.) This will give you a 201 Created status, and point you to the new resource. You can now update that resource with the correct post data.

This multistep procedure is required to enable raw file uploads, and I’m not entirely pleased with it, but it’s the only way without requiring multipart requests. I’d love to have feedback on this system, as I think practical use will eventually reveal the correct method here.

So, it’s time to start winding up the Summer of Code portion of the project. There’s still one week left for the Summer of Code project, so you may still see a release next week, but most likely in the form of smaller updates, especially with documentation and testing. As I finish up, it’s time to look forward to the future of the project. The plan is to form a Feature as a Plugin team as we work towards core integration in future releases. People have already volunteered for the team back in the 3.8 scoping days, and I’ll be getting in contact with them shortly, but it’s not too late to nominate yourself for the team; let me know if you’re interested.

That was my feeling too, but multipart is harder for clients to use, so I’m conflicted. Interesting that even streaming from php://input still appears to use large amounts of memory; thanks for the thoughts.

It’s been a while since you’ve all heard from me, so I wanted to check in and assure you I am still alive. I’ve been plodding along behind the scenes with the JSON API and mostly getting design documents sorted.

The big feature I’m working on at the moment – media – has turned out to be tricker than I initially thought. While media is technically handled via custom post types, it’s a completely different kettle of fish to normal post types, so I’ve been working through the conceptual issues behind this to ensure that the API stays consistent both internally and externally. I really believe that it’s worth my time to sit down and consider these conceptual issues in depth rather than pumping out the feature as soon as possible. (The implementation of the media-related endpoints is working locally, but I don’t want to push this up while it’s still in flux.)

I still hold out hope to push through media, but will likely reduce the scope of the other tasks to compensate, due to the complexity of media and the time it has taken so far. I’d like to personally apologise for this, this is a result of improper scheduling for the second half of the project.

Personally, the past month or so has been pretty stressful as well, due to a number of other things going on in the background. Balancing this project with university work has become a big issue recently, so I’m working through this as best as I can. Ideally, my preferred option at this point would be to push this project out of the Summer of Code phase and into the open source team phase rather than continuing to work on the project in isolation.

Along those lines, revisions will be bumped from the Summer of Code phase completely. While they are part of core functionality, they’re a rather large task that is secondary in importance to media and also behind taxonomy handling. I’d love to see these in the plugin at some point, but that won’t be happening during the Summer of Code phase. What I would love is for someone to volunteer to develop this (in a separate plugin for now, thanks to GSoC’s restrictions) for integration back in as soon as possible, which would also help with validating the API’s usefulness.

So again, sorry and hopefully I’ll have something better to show you next week. Thanks for your patience.

This release brings the first version of the Javascript API, based on Backbone models and collections. To use it, simply declare `wp-api` as a dependency to your script, then start using the `wp.api` classes. Here’s a quick example of how to use it:

These are intended purely as building blocks for your own software. I had been looking at rewriting P2 partially to use these, however it appears that would require gutting P2 and basically starting from scratch, due to P2’s architecture. I’d love to see what you can do with this though, and bonus points if you can get a API-ified P2!

The coming week will introduce some specialised page handling as an example of how to enable custom post type support, plus the beginning of the media/attachment-related APIs. These will probably be a fair bit of work, so it’s possible only basic functionality will land next week.

This is really cool stuff Ryan, especially the Backbone wrappers. I’m going to check it out in detail as we’re working on http://geto2.com, which is the next generation of P2, and is already based on Backbone (although most of that was written well before this was available, so we’ve implemented it all ourselves).

This week is a little different to the previous ones, as it’s a primarily testing focussed week. I’m skipping the release for this week in favour of just an update post.

I noticed fairly late this week that the aforementioned client library wasn’t actually public and available to you guys, so I’ve now fixed that up. Given the lack of comments, I’d hazard a guess this also means no one tried using it. 🙂

Previous Week

Over the past week, I’ve been looking at a variety of testing-related items, including code coverage and schema validation. I’ve finally sorted the schema validation out with the help of a JSON Schema validation library, after much messing around. The unit tests now integrate this schema validation with the normal testing suite, which should ensure that the API is conformant to the specification as well as fully functional. I do suspect that the schema is slightly outdated and is missing a few items, so I’ll be ensuring the documentation and schema are consistent in the next week.

Another area of the testing that I’ve been looking at is working with code coverage. After exploring the inner workings of PHPUnit’s code coverage, I think I’ve worked out a solution using a test helper plugin. This gathers the statistics on the server, then serialises them to send them back to the client. This still needs the PHPUnit end to connect to, which should be simple once I work out how to override the PHP_CodeCoverage object. (Help here would be appreciated if anyone has familiarity with PHPUnit.)

I’ve also started work on the Backbone client, however that’s currently not in a state to release. The plan here is to create a generic set of Backbone-based classes that can be used as a library, with the proof-of-concept itself being a theme based on P2. I’ve started work on the Backbone side of things, however I’ve not worked on much here with the theme side. I suspect that P2’s Javascript will be discarded in large part, but hopefully I can avoid that as much as possible.

Next week, you can look forward to more testing updates and the Backbone client. I’ll also be pushing out a new version with some bug fixes (thanks to Mike Schinkel for reporting these) as well as the usual slate of updates.

Documentation

I’ve also discovered that my documentation thus far has been a little lacking, so here’s a bit of an intro to using the API.

After downloading and activating the plugin, your site will now capable of serving up data from its API. The API is accessible at /wp-json.php from your site (if you’re not sure where this is, copy your admin URL and replace wp-admin with wp-json.php), with the specific route added after this. For example, the index is available at /wp-json.php/ and a list of your posts is available at /wp-json.php/posts.

If you’d just like to take a look around, you can view this in your browser and navigate via the links in the JSON. I personally use JSON View for this in Firefox, which converts the URLs in the data into proper hyperlinks as well as making it easier to view the data. The API uses a concept called HATEOAS, which basically means that all the possible data is discoverable via these links.

Some endpoints will add extra data if you’re logged in with the correct permissions. You can log in via the normal WP login page if you’re just viewing this in your browser (using cookie-based authentication), but if you’re accessing the API programatically, the API also has built-in support for HTTP Basic authentication. (OAuth support is not planned for the core of the plugin, but my OAuth provider plugin might be a good start for anyone who wants to write this as a plugin.)

I’m always open to questions regarding how to use this, so let me know here or on Twitter if there’s anything I can do to help you out.

I replied on the ticket (note that notification settings aren’t copied across from core Trac); multisite is out-of-scope for the GSoC project, but there’s a multisite plugin you can activate that should maybe hopefully work. Integrating this is out-of-scope, since I don’t actually have a multisite install handy (so testing would be appreciated).

A note about HATEOAS. One of the primary benefits of HATEOAS is that it allows the server to change its URL space as needed however if an API publishes its URL structures to allow developers to construct URLs then the primary benefit of HATEOAS and the justification people use for the significantly increased complexity on the client is pretty much moot. Those developers coupling their apps to the APIs make it not viable to change the URL space in the future as too many existing apps would break.

That said, IMO the URL space for “well-known” APIs should be accessible by URL construction though me stating this drives the REST purists mad.

Additionally, if the API serves a generic content type such as "application/json" vs. a content like something like "application/wordpress+json" a client should not know how to handle the response representation as doing so violates RESTful constraints. If the latter content type were used then it would be RESTful to write a client that knows the structure which as a side note diminishes the need for a schema.

OTOH having a WordPress-specific content type could have real benefits because it would allow for the development and evolution of RESTful WordPress-specific clients that don’t violate “the rules”; it would allow WP clients to avoid the need to layer application logic on top of a generic content type. And it could potentially result is a slew of new clients that can integrate with the WordPress API. But then again maybe that’s something that can be considered before WordPress 4.0+.

Speaking of schemas this post discusses REST and description documents. It discusses WADL not JSON Schema but many of the concerns still apply.

Back to HATEOAS, if the URL structure is published for URL composition and a generic media type is used them the only real benefit you get from HATEOAS is inline documentation, which is a real benefit. But to say an API supports HATEOAS and also URL construction is analogous to owning a Ferrari without an engine; you get to admire it’s looks but you can’t use it for what it’s meant for, to be driven.

Finally, HATEOAS is not required, it is for stable architecture meant to last decades; URL construction if perfectly workable and probably 95+% of APIs on the open web use it, although it does annoy the REST purists. But URL construction makes decades long implementations much less likely. FWIW.

So, given that we’re aiming to have set routing, you might ask why HATEOAS is needed at all. The primary motivation for this is to ensure that we have at least some sort of workable solution for pluggability. It’s conceivable that a plugin could change all of the routes, or rework some of them, so adding this means that clients can be robust in terms of routing if they’d like.

A content type (likely application/vnd.wordpress.api.v1+json) is something that I’d like to use eventually, but I haven’t yet for various reasons. The main one of these is that even vnd. types should be registered, but the format is still in flux, so it’d be inappropriate to register it now.

As for the schema, the intention of that is to be specific about how exactly to parse the responses, without me having to write ABNF for every single field. I’ve written ABNF for the more structured ones, but the point is to be as explicit about the data as possible. The fact that I can use that for automated testing is a huge plus, but it’s primarily a documentation tool (which is also why it lives in the docs directory).

(Just to clarify on the registration of vnd. types: you technically don’t have to register these, but it should be at least discussed with IANA. In addition, this would require the consent of the core team, and they’re rather busy, so it’s not worth the pain.)

Same here, I got issues and nothing happened. On google Chrome, it tells there is a server error 500 when trying the URL /wp-json.php/posts/ and on Firefox it tells anything, just the white screen of death lol.

Has the user need to set permalinks “on” in order to use the API or not ? Will wp-json.php will be hidden later with permalinks ? Like reaching the API with this kind of url : /api/v1/posts/ or /api/posts/

Also, is there a way to place the posts IDs in the URL for retrieving a specific post on a GET request => /api/v1/posts/25 ?

Time for another release of the JSON REST API! We’re closing in on midsemester evaluation, so it’s time to get our test on and ensure that everything’s working well. Only small updates to the API itself this week, but also the first introduction of the comment API:

Add initial comment endpoints to get comments for a post, and get a single comment (#320)

Return a Post entity when updating a post, rather than wrapping it with useless text (#329)

Allow filtering the output as well as input. You can now use the json_dispatch_args filter for input as well as the json_serve_request filter for output to serve up alternative formats (e.g. MsgPack, XML (if you’re insane))

Include a profile link in the index, to indicate the JSON Schema that the API conforms to. In the future, this will be versioned.

The big part of this week is the introduction of the testing suite and reference client. These are now both available on GitHub and I’d love for you to run them and test out the API. There’s currently one known bug that causes one of the tests to fail, which I’ll be handling in 0.4 due to the large scope of the issue.

I’m planning to continue work on the test suite this week, due to the large scope of the testing suite. Ideally, I’d like to get up around the 90% mark for code coverage (which I’ve done previously on my Requests library), however I’m not sure as to how coverage can be measured in this case, so I’ll also be investigating this. The coming week will also involve creating a Backbone-based sample theme as a proof-of-concept of real world use. In addition, coming up I’ll be consulting with Max Cutler from the mobile team and ensuring that any possible issues are sorted out at this early stage.

As always, I appreciate all and any feedback on the project. This week and the next will be boring for you guys in terms of new features as we approach midsemester review, so I’d love you to get out there and break the API. If you can find problems with it, please file them and let me know so I can work them into the testing suite.

As you can see, it’s a bit of a quiet week in terms of new features, but with the exception of revisions, attachments, and comments, the post API is now considered feature complete. Implementers can now expect that there won’t be any major changes to the post API.

Since you’ve all been so great, I’ve thrown in a bonus for this week. I’ve created a small proof-of-concept of extending the API for a multisite installation, which is now available via GitHub. Keep in mind that this is just a proof-of-concept, so it doesn’t have too much implemented. (This is also not covered under the scope of the GSOC project, so pull requests are most welcome!)

In the next couple of weeks, development will be focused around creating tests and example implementations. While this is occurring, I plan to also implement commenting, which was not in the original scope, but is still fairly important.

Astute observers amongst you will notice that the main collection parts (#266, #267) are missing from this release; I’m slightly behind on the release process due to just finishing up final exams and illness. Don’t worry though, I preplanned for this by allowing myself a buffer week before the midsemester deadline.

Hi everybody! Some of you may know me from various patches or WP-related endeavours: I’m Ryan McCue, developer of the SimplePie library used in core, along with a HTTP library called Requests, and long-time core hacker. I’ve been working with WordPress for quite a while now, both on open source and professional work with clients. What you may not know is that I’m also studying Electrical Engineering and Maths at UQ in Australia, and I’m here to give you a heads up on my Summer of Code project over the coming months.

For those who missed the discussion on wp-hackers about my proposal, I’m working on a JSON-based REST API for core. I started on this with an initial proof-of-concept back at the end of last year, and I’m now working on expanding this out into a larger project. The code is being written as a plugin for testing, with the goal of having it integrated into core in some form post-GSOC.

(You’ll notice I’m currently about a week behind on my schedule, which I suspected may happen, as I’m in the midst of my final exams here. I’ve allocated an extra week just before the midsemester review for catching up if I don’t do so before then.)

As it is, the plugin is in a usable format, and you can grab it from either GitHub or Subversion. I’d also recommend checking out the GSOC Trac issues if you’d like to keep track of the status. I’d love to have your feedback (especially on the design documents) as I move forward.

Cheers, and I look forward to working with you all in the coming months!

Part of the design of the core of the JSON handler is the idea that global state is to be avoided where possible in the methods. Although it’s not part of the GSOC scope, it’s definitely designed so that you can add an endpoint handler and route everything yourself while still reusing the existing code. (I’d like to tackle it at some point, but out of scope for now)

Good luck working out the endpoint handling though. 😉

That’s correct regarding SVN, but the .org SVN is our “official” repository in terms of GSOC accountability (and is also linked to Trac).

The plan is that plugins will be able to add additional endpoints with all the same flexibility and features as the core API, so yes, as long as bbPress adds in those hooks for additional endpoints, this will be possible eventually.

Great work, Ryan. This will really empower JavaScript developers to develop applications on top of WordPress. No longer will they have to create WP Ajax handlers every time for anytime they may need to do: they can do it all in JavaScript.

It’s time for us to apply for GSoC 2013 if we want to participate. I can take care of the application, but will need everyone’s help in producing the project ideas and mentor sections.

“Why didn’t we get accepted last year?”

We didn’t get accepted last year because our Ideas page wasn’t as good as it had been in past years, and they wanted to make room for some new orgs, so we got cut. Our ideas page wasn’t great because we were planning to embed students in regular cycle feature teams, but we didn’t write up all the potential features this might have included. This year we need to write out all the possible features.

PROJECT IDEAS:

The more project ideas the better, so things that aren’t targeted for the next cycle but that we might want to do as a plugin to get some traction for inclusion in a later release would be great projects, too. Project ideas can include core features, plugins, themes, unit tests, etc. This includes work on .org site things like the plugin used to power WordCamp sites. The projects must be code, though, not design, documentation, etc. List your project ideas in a comment on this post. Please give it a title, and a short one-paragraph description. If there’s a ticket or relevant thread somewhere, link to it.

POTENTIAL MENTORS:

Want to volunteer to be a mentor? You need to be good enough with wp coding that you currently don’t need oversight to generally get it right (though obviously we all can get better with more eyes on code), and have enough time to be available to your student for feedback and code review (at least once per week, though more often is better). The time commitment can be anywhere from one hour to ten hours per week, depending on how far along your student’s project is, and what there is for you to review (an hour or two a week is typical). You get a GSoC tshirt at the end. Want to be a mentor? Leave a comment on this post with a short/one-paragraph little bio about yourself, what kinds of projects you are interested in/qualified to mentor, and links to your website, wordpress.org profile, and twitter account.

LEAD DEVELOPERS:

I know you are right smack dab in trying to get this release out, which is why I’ll take care of the application and associated materials. That said, I’m a bit out of the loop with core plans, so coming up with the potential projects list on my own would mean they might not mesh with the actual goals/plans of core team very well. I’ll be looking to you guys to give me a sense of priority/possible plans for the next dev cycle, and to tell me who is qualified to mentor a project from the final list of volunteers. Will ping you offline for this.

Not sure how to write up a project idea or your short mentor bio? Take a look at the ideas page from 2011 for a guide.

Note 1: This year BuddyPress will be applying as a standalone organization with WP as the vouching organization. I’ll be helping Paul Gibbs get their application going.

Note 2: I haven’t announced it yet since it just happened, but we’ve been accepted as a mentoring organization for the Gnome Outreach Program for Women for the summer session. Unlike GSoC, where Google foots the bill, we’ll have to raise the money to cover our interns for the Gnome program, so I’ll announce it along with a fundraising drive. The Gnome program is a little broader than GSoC, and does allow non-code projects, so things like design, ux, documentation, translation, community management, etc would all be possibilities there.

Jen, I appreciate all the work you (and others) are putting into this. I’m fairly certain I wouldn’t qualify as a mentor (at least this year) as my WordPress coding knowledge is just now starting to solidify. However, I would like to know what I can do to prepare for future involvement (if anything). I would also be interested in learning more about the Gnome Outreach Program.

Idea 1: Dashboard Widget API
We have a great widget API for sidebar widgets, but the admin dashboard widget API is far less reusable. Certain widgets rely on the same code (i.e. feed widgets) without a clean abstraction layer to prevent changes to one from breaking another. This is one of the biggest reasons #19239 hasn’t gone anywhere yet. Merging the widgets required some hackery and the new widget in that proposed patch isn’t abstracted enough to be reusable anyway.

The idea would be to create a basic object template for dashboard widgets that can be extended in the same way WP_Widget can be, then to revise existing dashboard widgets to use the new API.

Idea 2: Post by Email Plugin
Deprecating the post by email functionality has been on a list of proposals for quite a while, and despite a patch (#22942) that disables the feature (similar to how Links were recently disabled), momentum has stalled since no clear alternative exists.

The idea would be to create a new plugin that extends post-by-email functionality in a clean manner. Host the plugin in the .org repo. Then update the existing patch on the above ticket for 3.7.

Idea 3: UUID Abstraction
One of the most talked-about “vulnerabilities” in WordPress is the ability to slurp down user IDs and login names programatically since user IDs are auto-incrementing integers. This could be prevented by changing the data structure to use UUIDs instead.

Also, updating post data structures such that new posts populate the GUID field with a true UUID (rather than the current post permalink) would prevent a lot of developer confusion as to what the field is and how it’s used. (Regular post IDs should remain integers since they’re used in the default link structure.)

Other objects in the database can be rekeyed as UUIDs as well, but my proposal is to start with user IDs and post GUIDs since changes there would solve specific problems that exist today.

Just to clarify, I’m willing and capable of mentoring any of these projects. I’ve worked with data abstraction in the past, and I wrote the first patch approaches for the two linked tickets above. For anyone unfamiliar with me … I’m @ericmann on Twitter, WordPress.org, and just about everywhere else. I also speak at WordCamps, moderate the WP Stack Exchange, and geek out on WordPress projects for a living.

Work can begin from Scribu’s plugin +improvements ( Debian-style version comparisons: >=, >>, <=, <<, =), there is a proposed patch out there.

Second step: Theme *code* dependencies: just like parent-child relationships are declared within the style.css (child themes), themes that need certain plugins to work properly (not enhacements, but hard dependencies) can declare so in a comment at the beginning of functions.php
RATIONALE: just like plugins can declare "Depends" and "Provides" within their meta information in the plugin's header comment, themes can depend on certain functionality from plugins.

BENEFIT: ability to "mix and match" code and allow for more modular code. Individual, granular updates to functionality via plugins, where all other components benefit from the upgrade without intervention.

BASIC FUNCTIONALITY:

Plugin needs certain other(s) plugin(s) to be active before this one can be activated (or at the same time, think “bulk activate”) + feedback in the plugin screen (Depends: plugin1, plugin2) with colors.

(thisi is basically the functionality present in Scribu’s plugin right now)

Plugin needs certain other plugin and version (equal, newer, older) to work.

Theme needs certain plugin(s) (and particular versions, while at that) to be activated. Clearly show it at the theme selection screen (just like “theme tags”, maybe?)

When a dependency is deactivated/removed, all dependent items (plugins, themes) SHALL be deactivated in turn (for a theme, fallback to “default”)

I’m proposing this for a GSoC project since I guess it would need quite some more work over what we currently have, and It would need a close eye before it can go in core (where it belongs, IMHO)

I have been using WordPress for a long time, but, sadly, I’ve never contributed any code to it. The main reason for that is that WordPress’ codebase is not fully object – oriented. I know that someone will say “don’t fix what isn’t broken”, but I really think that rewriting the codebase would be extremely beneficial for everyone. It would make a much cleaner API for plugin developers and it would be easier to maintain the codebase. I’d like to work on this through my GSoC participation. For the beginning, we would have two API interfaces – the current one which will become deprecated after the rewrite has been done, and a new one which will allow all plugin developers to gradually switch to new API as the time progresses. Then, say, in WordPress 4 the deprecated API would disappear (or we can provide some sort of plugin for the deprecated API) and a new, OOP – way API will be in place. I know, rewriting the whole codebase can break many things, but no one said that the transition won’t hurt.

For those of you who don’t know I’m Justin Shreve. I’m a former GSoC student (2009, 2010) and co-mentored Summer 2011(with Aaron as lead mentor) the HTML Emails project. I also work on social projects over @ Automattic.com

I’d be interested in mentoring any general projects but specifically: API projects (XML-RPC, REST/JSON API in Core, etc), Projects around things like Post Formats, Multi-site improvements, and any social projects (if there happens to be any plugin proposals, etc).

I would love to be a mentor on one of the projects. I’m can work on Media (no backbone yet), mobile projects and on the API. Maybe a RESTfull API. But any other related project is fine.
I’m @markoheijnen on Twitter, WordPress.org, and just about everywhere else. My site is http://www.markoheijnen.com and I speak at WordCamps and WordPress meetups.