This patch is the first big step towards an improved JavaScript development workflow for WordPress core and introduces the following major changes:

Introduces a required build step for WordPress develop. This means it will no longer be possible to run WordPress from the src directory, but I’ve taken multiple steps to make this transition acceptable and as smooth as possible, described below.

Fully backwards compatible reorganization of the JavaScript code in src. All JavaScript is placed under src/js and is organized in an improved directory structure that gives much more context and is future compatible. In production, everything is built back into its original location to ensure backwards compatibility.

This approach was taken for several reasons, almost all of which have to do with improving development workflows. This is broken down into the following concerns.

We need to maintain backwards compatibility while providing flexibility and future compatibility for modern JS development workflows.

We need to easily distinguish scripts that are enqueued from source modules that are bundled. We need to provide freedom in working with the source and organizing it in the best way with minimal to no impact on backwards compatibility.

We need to organize the enqueued scripts in a structure that provides more context about their scope, contents and use.

We need to have a better process for managing dependencies and vendor scripts.

We need to make it intuitive to make JavaScript modules and APIs globally available using Webpack.

We need to facilitate a scalable workflow for building and bundling code. (Allowing for a build step.)

We need to lay the groundwork to publish individual JS packages to NPM straight from the WordPress source, so the entire GPL community can benefit from them.

How?

To make this transition as smooth as possible for code contributors, we’ve taken a couple of initial steps, with more to follow.

Separate index.php in src informing developers with steps to take

We’ve added a special page that people running WordPress from source will start seeing:

As the screen says, npm install && grunt build will be enough to trigger a build. After that, you should have a smooth development experience using grunt watch. The wording in the notification can still be improved. I’ve created a separate ticket to improve the wording of this message.

Optimized, (much) faster grunt watch

We’ve optimized grunt watch rebuild times to be much faster, typically around 1 second for JavaScript and almost instant for PHP. Developing with grunt watch should be a comfortable experience. Future optimizations in this area should still be possible and could make the developer experience even smoother.

Documentation

I’ve written documentation drafts, which has been reviewed extensively by the Core JavaScript team, though more feedback is always greatly appreciated. There are 4 documents in total with the following topics:

JavaScript code organization and development practices

JavaScript modules and Webpack

Managing dependencies

Setting up your local development environment

These documents can be integrated into existing documentation or added to the handbook as separate entries once this patch takes effect.

What’s Next?

In working on this, we’ve come to the conclusion that we can do a much better job at creating a seamless development experience for WordPress core. Some complexity cannot be avoided. But if we can find ways to not force that complexity onto contributors, we should do so. This patch is a good first step in enabling more modern development workflows and practices, but it also invites future iteration. To that end I’ve opened two more tickets:

The plan is for this change to be committed to trunk, so iteration can continue. I do acknowledge that, while we work on further improvements and optimizations, trunk might become slightly less comfortable to work with. This is the point where we need everyone’s help to get it right. Any help to improve the developer experience for WordPress core is greatly welcomed. You can think of feedback in the form of comments or tickets, tooling proposals, better use of existing tooling, optimizations of the build, anything that helps make the development experience for WordPress core more seamless.

Hi Daniele, this change only affects the way the JavaScript source is managed, not what is included in WordPress itself. So it doesn’t touch how jQuery (or any other library) is being used in WordPress. That is a topic for a different ticket.

On your VVV question, it should be relatively simple to make VVV work with the new setup. Some fixes have already been done in preparation for this, see https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1447. I think documentation on how to run grunt watch inside VVV is something that could still be added/improved.

This is awesome! 💯
I’ve been thinking and working on something along this line but with nothing tangible. I want WordPress to have an easy to get started DevExperience. It’s not possible to use the latest JavaScript tooling without something like this.

I’m all in for this, big thumbs up! 👍

SUGGESTION: Maybe we can convert the WordPress’s JavaScript part into a big mono-repo that way it’ll be much easier to maintain all the new custom small and modular packages inside of the core without having to deal with a bunch of them.

We can use LernaJS for that, I make use of it in create-guten-block and it doesn’t get any better — though, works well with yarn workspaces.

We just recently started requiring a build process for our JS, and while it’s good and necessary, it has been a bit of a learning curve and configuration has been a bit time consuming (I’m pretty new to node JS etc).
In our case, one issue was that we were all on different versions of node.js and so all experienced slightly different behaviour. But once we all got on the same version things seem to be working ok.