Tag Archives: rest-api

With WordPress 4.9, a bug has been fixed which would cause unexpected numeric results to be included in the parsed URL parameters for a REST API request. Prior to this change, calling WP_REST_Request::get_params() for a request like /wp/v2/users/(?P<id>[\d]+) with an ID of 10 would return array( 'id' => 10, 1 => '10' ), where the latter numeric key is unnecessary and a result of PCRE matching against a named subpattern (see preg_match() documentation). The fix ensures that the above request now only returns array( 'id' => 10 ) instead. This helps for example to verify that a request does not include more than a few specific parameters.

The WP REST API docs have always been using named URL parameters, using regular (numeric) matches was never recommended. With this bug fix in place, using named parameters is now effectively required, for example /my-namespace/my-endpoint/(?P<numeric_param>[\d]+) must be used instead of /my-namespace/my-endpoint/(\d+). For background discussion on these changes, see #40704.

If you’ve been following WordPress development this year, you may be wondering “what’s been happening with the REST API focus?” We’ve been a little under-the-radar for most of the year so far, so we thought publishing an update and roadmap might be a good idea.

For new contributors looking to get involved with the REST API focus or WordPress generally, there’s never been a better time, and we’d love to have your help on our projects. Read on to see what we’ve been doing, where we’re going, and how you can get involved.

Since 4.7

Since the REST API was merged into core in WordPress 4.7, development activity has unfortunately been light. The merge into core was a huge effort, and after shipping in 4.7 we saw a drop-off in contribution and overall momentum as many API contributors took a break to recover from the stress of the merge. These contributions have not returned to previous levels, and there’s a few factors behind this: the move to Trac, lack of a forward roadmap, and overall fatigue have hampered our ability to move forward quickly.

The core REST API focus goal is to utilise the REST API within the WordPress admin. Defining the scope of this project has involved auditing all admin-ajax calls, as well as the filters used inside these calls, and where they are used. In addition, we’ve been working on the low-level JavaScript utilities we need to offer conceptual compatibility: while we can deprecate and remove old PHP filters, we need to offer new JS-based filters to replace them.

The admin-ajax audit revealed that the majority of ajax requests can be grouped into four categories: Media, Themes, the Editor, and List Tables. The code for both Media and the Editor will gradually switch to the REST API moving forward with development around Gutenberg, and likewise endpoints for better managing Themes are expected to be incorporated into Customizer work.

Rather than simply refactor the existing code for the other actions in a piecemeal fashion, we’ve been working on prototyping bigger groups of related changes and features, starting with the New List Tables and Live Settings.

A significant portion of existing admin-ajax code is for handling list table actions. The existing JS code for these actions is particularly difficult to work with, and the existing list table actions user experience is frustratingly inconsistent (for example, deleting a comment happens inline, whereas deleting a post causes a page refresh). A reworking of the code has the potential to improve UX significantly. New List Tables allows us to explore ideas around how we can improve the content organisation and management experience in the admin. This is a prototyping plugin where we’ve been exploring backwards compatibility techniques, and thinking about how a theoretical new management interface would look.

In addition to these API team projects, work has continued on REST API-related pieces on other teams, notably the Customizer and Multisite teams, who are working on API endpoints in their respective components.

Renewing our focus

Moving forward with the REST API, there’s a few key items we’re going to be focussing on. These items will have their own dedicated subteams and development cycles, and will work in parallel. The two broad goals are to use the API in the admin, and to solve authentication for external applications.

API in the Admin

Getting the API used in the WordPress admin is our primary focus. While it is technically possible to directly switch from admin-ajax calls to REST API calls, this is essentially refactoring with no real user benefit. Instead, we want to focus on changes that can improve the user experience.

For the feature prototypes (New List Tables, and Live Settings), we’re engaging members of the Design team to lead these features from a UX perspective. So far, these prototypes have been primarily about proving out the features and ensuring it’s actually technically possible to migrate these features to use the REST API; with the initial success on the technical side, we need to switch focus to delivering compelling user experiences.

New List Tables will be working with the goal of prototyping improved content management, using the REST API. This includes unifying and standardising existing interactions, and improving the perceived performance. This is a React-based prototype, and uses the existing REST API endpoints.

Live Settings will be working with the goal of making settings changes seamless. We’ve seen huge strides forward with the Customizer for updating your site, and the backend settings deserve a better experience to match. Live Settings touches on similar areas as the Settings API Enhanced project spearheaded by the Accessibility team; we plan to continue working independently to avoid blocking each other, while keeping in touch about the respective projects.

Work on converting existing admin-ajax code to use the REST API will continue, however this won’t be a priority, as it generally doesn’t provide a strong benefit to end users. Most admin-ajax actions will naturally be deprecated as part of progress by other focuses, including Gutenberg and plans around the Media Library. We’ll continue working with other teams and focuses on their efforts here.

Authentication

External authentication is an unsolved problem, and one that’s crucial to API use outside of WordPress core itself, including the official WordPress apps.

There are two key problems to solve here: how do apps act on behalf of a user (authentication), and how do sites recognise valid apps (initial connection). We have existing solutions to both these problems (OAuth 1.0a and the Broker system respectively), however these are not the easiest solutions, and aren’t adequate for all use cases.

To date the most complete authentication solution maintained by the REST API team has been a plugin providing OAuth 1.0a authentication. Moving forward, we are switching authentication focus to OAuth 2. As Matt announced last year, we are going to begin shipping HTTPS-only features in WordPress: this allows us to switch to OAuth 2. Work started during the WordCamp Europe contributor day on a new official OAuth 2 provider plugin which is now under ongoing development.

Simplifying the initial connection is a much harder piece, and this is a long-term project. Eventually, this should be as simple as a “Connect to WordPress” button, requiring minimal effort for app developers and no effort for users. This is a complex problem to solve, and no similar software has to work on the same scale we have. In the meantime however, we’re going to investigate pre-configuring Core to recognize and permit authentication from certain default apps, including the official WordPress mobile applications. Whitelisting applications in core is a practical expedient but this solution is not sustainable in the long-term, and should be replaced with a better system as soon as feasible. Work on solving this issue will be in the mid-term, however, as we need to ensure we have solid basics first.

Help Us!

The toughest challenge facing the REST API team right now is resourcing. There are only a few people working on the API regularly, and we need help to build out our projects—which is hopefully where you come in. We need people of all skillsets to help on New List Tables, Live Settings, and OAuth 2. This includes regular contributors, JS developers, and designers. And all of this will need documentation, too: following a productive contributor day at WordCamp Europe we are making progress expanding and reorganizing the REST API developer handbook, and would gladly welcome any interested docs contributors.

Our plan is to release the first public beta of each of these projects within the next month, with regular releases from each project following that. New List Tables and Live Settings could be part of either a 4.9 or 5.0 release, while OAuth 2 will remain as a plugin until fully proven out, and would likely target a core release next year. This also requires coordination with the Mobile team, and finalising the approach to usage inside the apps.

If you’re interested in getting involved, we’d love to get your help. The API holds weekly meetings every Wednesday at 13:00 UTC (next meeting at Wednesday at 13:00 UTC), and we’re always happy to spend time helping people get started contributing.

Creating a useful sites/ endpoint for the REST API and making WP_Site_Query more useful have been frequent topics over the last few weeks in #core-multisite. Quite a bit of discussion has been centered around the idea of a global wp_blogmeta table, whether it’s a good fit for core, and (if so) how to approach its introduction. See #37923 and the previous make/core post discussing a sites endpoint for additional background.

The intent of this post is to step back a bit and clarify the issues at hand to help identify the right solution(s).

The initial site information problem

A request to a global wp-json/wp/v2/sites/ endpoint on a network should return a set of site objects, similar to how wp-json/wp/v2/posts/ returns a set of post objects. A request to wp-json/wp/v2/sites/4 would return a single site object.

Written today each site object, represented as WP_Site in PHP, would provide at least this data:

ID

domain

path

registered

last_updated

public

archived

mature

spam

deleted

lang_id

The above fields all mirror what is available in the global wp_blogs database table installed with multisite. These are useful on their own, but additional data is frequently required.

Example: One piece of the admin that would be great to power with the REST API is the My Sites menu that multisite users see in the toolbar. To build this view, the home URL, admin URL, and site name are all necessary. In PHP, this data is made available through magic properties on the WP_Site object. When home, siteurl, blogname, or post_count is requested, the site uses switch_to_blog() and get_option() to populate the data from the individual site’s options table. If 25 sites are on the list, this generates 25 context switches and accesses 25 different tables.

Migrate these properties into wp_blogs from wp_#_options as additional columns.

Migrate these properties into a global wp_blogmeta table.

Migrate these properties into another shared global space.

The querying sites problem

It’s currently possible to query for sites by the default fields listed above. This data is useful for querying, but it would also be nice to query by a site’s name, description, or other piece of data at a global level.

Example: In the list table that displays all the sites of a network, it’s possible to query by a site’s domain or path, but not by it’s actual name. In a large network, a user may find it difficult to find a site when many sites share similar domains or paths. The user may know the site’s title, but not the address itself.

There is no real workaround for this issue in core right now as each site’s name is stored individually in that site’s wp_#_options table and cannot be queried collectively.

Possible approaches:

Migrate these properties into wp_blogs() from wp_#_options as additional fields.

Migrate these properties into a global wp_blogmeta table.

Migrate these properties into another shared global space.

Feedback please

Please leave any thoughts you have on possible approaches to these 2 problems. It would be especially helpful to identify some use cases that may not yet be clear, or approaches that are not listed above. All feedback is welcome. 🙂

Improving the users endpoint was the main focus of this week’s office hours. The following is a recap of the decisions from that discussion. Please leave your feedback in the comments on this post. Related meeting notes are available from the January 10th office hours and the January 3rd office hours.

4.7.3

Some small changes to the users endpoint for multisite should be made in 4.7.3. The ticket for these changes is #39701.

Fail when a GET request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

Fail when a PUT request is made to site.com/wp-json/wp/v2/users/<id> and that user is not a member of the current site.

The expectation for the users endpoint in 4.7.3 is that only users from the current site can be listed or managed in any way via the REST API. Global access to users will not be available, even to global administrators (super admin).

4.8

The users endpoint will be improved significantly for the 4.8 release, ideally providing full compatibility with how users are currently managed in a multisite configuration. The ticket for this task is #39544.

Here are the expectations for the users endpoint in 4.8:

POST to site.com/wp/v2/users/ with an existing global user’s email address adds the existing global user to a site.

POST to site.com/wp/v2/users/ with complete new user data creates a new global user.

GET to site.com/wp/v2/users/ with a global parameter set to true lists all global users.

GET to site.com/wp/v2/users/ without a global parameter lists only the site’s users.

GET to site.com/wp/v2/users/<id> with a global parameter set to true lists data for a global user, even if they are not a member of the site.

GET to site.com/wp/v2/users/<id> without a global parameter lists data for a user only if they are a member of the site.

PUT to site.com/wp/v2/users/<id> with a global parameter set to true updates a global user and data for a user that is in the global context.

PUT to site.com/wp/v2/users/<id> without a global parameter updates a data for a site user that is in the site context. This is how role data can be passed to maintain a user’s relationship with the site.

DELETE to site.com/wp/v2/users/<id> with a global parameter set to true deletes the user completely.

DELETE to site.com/wp/v2/users/<id> without a global parameter removes the user from the site.

Note that while users exist in a global context, data attached to these users can exist in the global context (e.g. email address) or in a site context (e.g. site role). There may need to be a method for registering user meta in a way that specifies whether it should be treated as global or site data.

One of the objectives for multisite is to determine how sites can be managed with the REST API. This has been an agenda item for the last two weeks and quite a bit has been processed. This will continue to be a topic, so please stop in for #core-multisite office hours on Tuesday 17:00 UTC and please leave your feedback in the comments on this post.

The requirements for the /sites/ endpoint can be summed up with these assertions:

The /sites/ endpoint should provide a useful set of data for each site without requiring the use of switch_to_blog().

It should be possible to query /sites/ for something other than ID, domain, and path.

In its current state, any /sites/ endpoint is limited to the fields in the wp_blogs table. Data such as a site’s name and description are stored in each individual site’s wp_#_options table.

Given a list of 20 sites, switch_to_blog() will be used 20 times so that get_option() can be used to retrieve things like home, siteurl, blogname, and blogdescription. An example of how inefficient this is can be found in the generation of the My Sites menu. Caching has gotten better with the introduction of WP_Site and WP_Site_Query, but there is an opportunity to change how the information is stored.

In #37923, @johnjamesjacoby suggests the introduction of a wp_blogmeta table that provides access to some site information in a common table. After discussing this in the January 17 chat, @iamfriendly and @flixos90 agreed to each take a look at core’s default options stored in wp_options and determine which made sense in a shared wp_blogmeta table. Richard’s results can be found in a comment on the ticket and Felix’s in the Slack discussion.

After some discussion in the January 24 chat, that list can be pared down a bit more.

home

siteurl

blogname

blogdescription

admin_email

The creation of a new table, wp_blogmeta, and migration of data for each site from wp_#_options is something that needs feedback. Without this change, a /sites/ endpoint is still possible, but may be limited. With the change, a /sites/ endpoint is much more useful, but requires a careful migration process.

Following last week’s discussion in #core-multisite (read the recap) this week’s office hours agenda was to continue the chat about the multisite-related enhancements for the REST API users endpoint, focussing heavily on how to access the required functionality. Here is a wrap-up of the discussion.

Now that the way on how one should be able to modify user roles per site was clarified last week, this week the focus laid on where one should be able to perform those actions. The current state of the wp-json/wp/v2/users endpoint in multisite is:

The users overview accessible with a GET request to wp-json/wp/v2/users only lists users that are part of the current site.

When creating a user with a POST request to wp-json/wp/v2/users, that user is created and added to the current site. When providing the roles parameter, the passed roles are added to the user, otherwise the user will still be part of the site, but without any role. See #38526 for background.

It is possible to both read and edit any user from any site with a request to wp-json/wp/v2/users/<id>, regardless of whether the user is part of that site.

A DELETE request to wp-json/wp/v2/users/<id> results in an error. See #38962 for background.

After the discussion about how to be able to add a specific user to a site, update their site capabilities and remove them from a site, this week’s chat revolved around where these actions can be accessed, as they are for the most part network-specific actions not available to a site administrator. The approach that was agreed on is:

The users overview at wp-json/wp/v2/users should continue to only show users of that site by default, but a request like wp-json/wp/v2/users?global=true should show all users in the WordPress setup. This parameter must only be available to network administrators though, more specifically users with the manage_network_users capability. In the future a network parameter might also be introduced for support of multi networks, but at this point core does not support querying users per network. Accessing global users should be available from all sites in a setup instead of only from the main site. While this approach makes these endpoints duplicates of each other, it has several benefits like preventing the requirement for cross-domain requests, allowing easier API discovery and not requiring the main site of a setup to be exposed to REST API calls to a sub site.

Assigning an existing user to a site and removing a user from a site should generally be only available to network administrators, and the site administrators of the site that is being interacted with.

Similarly, editing a user that does not belong to the current site should only be possible for a network administrator. Currently this is available to site administrators as well which is probably wrong.

Deleting any user completely should only be available to a network administrator. A good way to handle the reassign parameter needs to be found though.

Before coming to the conclusion that dealing with multisite functionality at the existing users endpoint, the possibility of introducing a multisite-specific endpoint only available on the main site was discussed. However this was considered not practical due to the nature of how users work in WordPress. Having separate endpoints for other network-wide functionality might still be a possibility though as long as that component solely affects the network admin, so this idea is something to keep in mind for thought about further network functionality endpoints in the future.

Back to the users endpoint, one related question that came up is:

Should the sites a user belongs to be available at the wp-json/wp/v2/users endpoint or at a future wp-json/wp/v2/sites endpoint? If they were available in the wp-json/wp/v2/users endpoint, every user entity would have a new sites key available if the current user had sufficient permissions to see these. If they were available in the wp-json/wp/v2/sites endpoint, that endpoint could easily support this functionality through usage of a user parameter.

@jeremyfelt suggested to look at the “Add New User” screen in the site admin to have a good use-case for how to scaffold the multisite functionality of the API endpoint. This has helped during this week’s office-hours and can also be beneficial in the future. Eventually this screen should be revamped entirely, being powered by the REST API.

Regarding the enhancements of the users endpoint, a general ticket for this task was opened at #39544. This ticket is meant to be used for discussion on the topic, while separate smaller tickets should be opened for actually implementing the individual pieces. For now feedback is welcome on that ticket. The discussion on multisite improvements for the REST API will continue at Tuesday 17:00 UTC.

One of the objectives for multisite is sorting how users are managed with the REST API. This was one of the agenda items for last week’s #core-multisite office hours and generated some good discussion. Here’s a wrap-up of the ideas and thoughts from that discussion.

We already have functions like remove_user_from_blog() and add_user_to_blog() available to us.

Does “add” invite or literally add? This can probably be included as data in the PUT request.

What happens if an API client is built for single site and then that site gets switched to multisite?

Handling bulk actions on an endpoint would be nice. (e.g. Add a user to multiple sites) No endpoint has implemented batch handling yet though.

Initial tasks:

It should be possible to remove a user from a site with a PUT request to the wp-json/wp/v2/users/# endpoint.

It should be possible to delete a global user with a DELETE request to the wp-json/wp/v2/users/# endpoint once all sites have been disassociated.

New tickets will be created soon for these tasks. Please leave any initial feedback in the comments on this post covering the assumptions and conclusions made above. There will be another round of discussion during tomorrow’s #core-multisite office hours at Tuesday 17:00 UTC.

There are three main focuses this year: the REST API, the editor, and the customizer.

For the REST API we’re going to work on getting first party wp-admin usage of the new endpoints, and hopefully replace all of the core places where we still use admin-ajax.

The editor will endeavour to create a new page and post building experience that makes writing rich posts effortless, and has “blocks” to make it easy what today might take shortcodes, custom HTML, or “mystery meat” embed discovery.

The customizer will help out the editor at first, then shift to bring those fundamental building blocks into something that could allow customization “outside of the box” of post_content, including sidebars and possibly even an entire theme.

Each focus will have a tech lead, and a design lead, and I’ll be working closely with each to make sure we’re aligned and moving diligently in the right direction even though we don’t have the normal release hooks. These starting leads will be:

REST API: Ryan McCue and KAdam White.

Editor: Matias Ventura and Joen Asmussen.

Customizer: Weston Ruter and Mel Choyce.

Given there is no set timeline for releases that would normally set a term, these leads are free to bow out at any time they feel they can’t contribute fully and we’ll find a replacement.

You might be wondering what each lead is responsible for. The REST team gave some thought to this for their focus, and this is the list they came up with:

Tracking and measuring specific success metrics is a priority for the WordPress REST API Content Endpoints project. The aim is to be able to look back at the implementation of the feature project to see how it went after the fact, and to utilize that information to inform future decision making.

A submission form will be created to help track implementations, in addition to automated scripting and analysis. Feedback will be grouped based on which metric it applies to. Developers and implementers will be polled to determine usage, where automation is a challenge.

It is very important to note prior to analyzing individual goals, that these goals and the achievement of them does not automatically indicate success, or that not achieving them means a lack of success. For instance, one metric that’s very hard to capture in goals like those that follow is the impact of a singular implementation. One highly impactful implementation could indicate a great deal of success, but would not be fully captured by the following metrics. These goals are established to help identify success metrics that would offer clear insight on the project, but they are not all encompassing or definitive alone.

Metric #1: Plugin utilization

Per the “Popular” plugins page of the WordPress Plugin Directory, the following can be determined:

21 plugins have 1m+ active installs

126 plugins have 200,000+ active installs

232 plugins have 100,000+ active installs

Many of these are not good candidates for utilizing the REST API, but there are a number of plugins that are.

The following goals are proposed:

15% utilization of plugins with more than 1 million installs.

7% utilization of plugins with more than 200,000 installs and fewer than 1 million.

3% utilization of plugins with more than 100,000 installs and fewer than 200,000.

Additionally, it is proposed to seek 5-10 examples of non-directory or otherwise prominent plugins, that may not fit the above metrics, to showcase their utilization of the API, and how it has benefited them. This may include commercial plugins.

Timeline for analysis: three major releases after merge (5.0 release day).

Metric #2: Distributed theme utilization

Popular themes are a bit of a misnomer in the WordPress landscape. The default themes or themes that have been around for a very long time account for a significant percentage of all active themes. And feature development isn’t as common for existing themes as it is for existing plugins.

In addition to polling free theme makers, top theme shops (according to Alexa) should be polled to determine usage of the API for theming purposes.

Additionally, coordination can occur with the Theme Review Team to identify usage of the API in newly submitted themes, to determine adoption.

The API could be useful in themes themselves, as well as complimentary plugins, which are not likely to be picked up by the plugin analysis. However, it’s difficult to know exactly how WordPress themes may use the content endpoints in a distributed theme context. It could be part of efforts within the customizer, page building, custom widgets, etc.

A goal of 2% adoption of the API in new themes is proposed, determined within three major releases, and analyzed amongst distinct ecosystems:

New WordPress.org theme review submissions

Third party theme shops (A shop using the API in any capacity, likely new releases, would count as “yes”, rather than theme by theme)

Metric #3: Custom development implementations

The WordPress REST API is immediately useful for custom development projects, where the developer has control over the environment. The REST API in core provides a vote of confidence to outside developers that they are building with an API that has the full backward compatibility and stability expected of features in the core project.

It is proposed to identify 50+ projects in production or under development utilizing the Content Endpoints, within two major release cycles.

The likelihood is that there will be far more consumption-only applications, versus those that manipulate content. Within the same two release timeframe, the goal is to identify at least 12 custom projects that utilize the Content Endpoints beyond a read-only use case.

Utilization of the API for additional core features and feature projects will be tracked over the next three major release cycles. It is proposed to identify 6 core features or patches than can utilize the API, as well as 1 feature project, within three major releases.