The patch on that ticket breaks up the Backbone classes in media-models.js, media-views.js, media-audiovideo.js, and media-grid.js into modules and injects them via Browserify on build/watch into a built file. Let’s start at the beginning.

Brain overload

Files that are 1000s of lines long are hard to consume. We try to alleviate this by adding copious amounts of docs. Still, it’s a lot to look at. Ideally, we would break our files into smaller modules and then somehow join them together in a build process.

It is common practice to serve (sometimes very large) minified files for JS and CSS that have concatenated many smaller files together and uglify’d (minified/obfuscated) them. It is no longer common or best practice to develop with huge files. We can learn a lot from emerging front end development trends, especially those from the Node/NPM community. In some cases, we can even share code.

We’ll use Media as the main culprit, but this could apply to any “manifest” – a term I use to describe files that contain the entire public API for a feature. Something like media-views.js, it might be nice to bounce from view to view in the same file, provided you know exactly what you are looking at, what depends on what, etc.

I have found, it is completely overwhelming for almost everyone. It would be great if each discreet piece could be viewed in isolation with its dependencies clearly stated.

There are many ways to accomplish the splitting of large files. I want to focus on 2 of the most common.

Vocabulary

Backbone is one of a growing number of MV* frameworks for JavaScript. A large majority of the code related to media either belongs to a handful of Models or to the increasingly large library of Views and View Templates.

Views are the building blocks for the presentation of Media (you know, “the Media Modal” or 4.0’s “Media Grid”).

The main canvas on which these Views are stitched together are called Frames, which are themselves Views – tilting our use of Backbone more towards MVP, P standing for Presenter.

We have Controllers, which are called States, but they belong to a Frame (Presenter! also a View!), so anyways…. for now….

When we create new UIs, we are more than likely adding new Views/Templates, or updating existing Views.

If we wanted to move from one large file to many files that each contain a class, we would create Modules.

Grunt is a task runner. We use Grunt to build our src directory into our build directory.

Require

Require is a great tool for converting AMD modules into built files. Require leans on Dependency Injection in its syntax:

This syntax works great, unless you have way more dependencies. Refactoring code could unwind a module that has a lot of dependencies, but if you are just trying to convert legacy classes into a module, Require starts to get a little weird. Concrete example: Frames have a TON of dependencies.

Require becomes a Grunt task to make one big file by recursing the dependency tree in an initial manifest. Require, by default, loads JS asynchronously, which can cause race conditions with plugins or themes that expect code to be registered on $(document).ready() or window.onload.

Require works even if you don’t build via Grunt.

Browserify

Browserify is a tool that allows you to use Node-style modules and run them in a browser without changing from the Node syntax. Browserify requires a build for this to work.

Browserify scans the abstract syntax tree (AST) of your JS code to compile dependencies. Your modules themselves get wrapped in their own scope like so:

(function (require, module, exports) {
.....YOUR_MODULE.....
})

After spending a lot of time messing around with both: I think we should use Browserify.

Converting “Legacy” Code

The media JS code is some of the most “modern” code in WordPress, but it still clunkily lives in huge files. To convert the code into modules, we need to make a lot of individual files (one for each Backbone class).

We also need to make sure we maintain the existing wp.media namespaces for backwards compatibility. We don’t want any existing functionality to change, we just want to build the files differently.

Even though the code is defined differently, wrapped in a new scope, and looks different when “built”, we can still maintain our current API design: what is publicly accessible now will remain publicly accessible.

In the patch

Disclaimer: this patch is for experimentation only. It will go stale probably before this post is published. It works, but it is only a playground for now. If this moves forward, it will be a laborious Subversion process to create a bunch of new files.

I have added a folder to wp-includes/js, media, that contains the modules and the built manifests. My patch adjusts script-loader.php to use these new paths.

The build pipeline

If you are following along with that patch and want to see this in action, run in the project root:

npm install

Afterwards, run:

grunt watch

*.manifest.js files get built into *.js files when you change a file in media/*, provided you are running the grunt watch task. The watcher will automatically call browserify:media and uglify:media when those files change. This allows you to run your site from src or build, and you will still get Browserify’d files. SCRIPT_DEBUG will either run *.js or *.min.js, just like any other minified JS in core.

This is a proposal

I would like to hear feedback from the overall community and certainly from our fair share of JS-trained ninjas. A common reason to *not* do something like this is the barrier to entry for new developers. I would argue in this case that the code becomes MORE readable and understandable. I was shocked myself to see how much simpler it was to absorb one piece at a time once the code was laid out in modules.

I attempted several times to ‘load’ these enormous JS files to my brain in last few months. Every time with little to no success. As I am using RequireJS I would be more font of the later, but I absolutely agree with your foreseen troubles it would bring. So browserify + grunt it is, hurray!

I for one applaud this initiative. There may be a couple of extra hoops to jump through, but they become virtually irrelevant with a properly configured Gruntfile to handle watching and building modules into the project.

Big fan of Browserify, too. Browserify will handle AMD modules if you need them (using plugins like `deamdify`) and coexists with non-module dependencies much, much better than RequireJS. (Last time I got RequireJS and WP to work together, I had to mess with WP’s enqueue mechanism because the loader is rather sensitive to scripts that alter the DOM.)

Music to my ears, @wonderboymusic! Regarding the “barrier to entry” argument, in my experience the biggest barrier I’ve seen people hit while working with modules has been that in almost all circumstances, the complexity of setup and configuration required by a system like Require results in a front-loaded learning curve. Coming onto an existing project with a modular system is, by point of contrast, generally a pleasant experience because it is much more obvious where any particular piece of code lives.

I’d throw in my $0.02 that since we’d need a build process anyway to ship production files, we should optimize for ease of development: whichever system requires the least work when adding a file. Webpack’s a good build tool that we’ve been using which supports both AMD and Node-style modules, as well.

Like K. Adam I found that working with modules does increase the initial barrier a bit for developers who rarely use javascript, but certainly not by much. And once the required code to work with modules is understood it makes things easier due to the clear structure and focussed content of each module.

Currently we use AMD (require.js) and build our combined js file with Almond, but we’re strongly considering to switch to Browserify or Bower. AMD isn’t as widely supported by packages as we’d like, and Require.js appears to clash with our new general purpose build tool, Gulp.js.

Having the js broken up into modules will be great! Have you tested/considered your third option, the use of native es6 modules and then something like webpack with the 6to5 loader (es 6 to 5 transpiler)? es6 modules are finalized since last august – es6 was fully feature frozen then -and I’ve been working with 6to5 for a little bit and its rock solid. The main reason would be longer term viability. es6 modules are going to be here to stay guaranteed, plus they pretty well supersede common/amd modules…

Really looking forward to some sanity in this area. Concerns I typically have about barrier to entry are mostly irrelevant here – components such as media already require a high level of understanding and some amount of experience, which makes familiarity with build tools as a requirement of development much more likely. As I’m understanding the proposal, build/watch won’t be needed if you don’t touch any involved JS.

My concerns are the same as others, that this increases the barrier to entry. I think that as long as some documention get’s written for the core handbook that we can also reference and point to about how and why (with this post serving as a great foundation).

One thing we will want to consider, if a file like wp-includes/js/media/models.js is going to be a generated file, we should exclude it from jshint.

You have my +1 in this effort. The media-* files are just too much to wrap one’s head around effectively, and in general we are in dire need of organization and better documentation for the entire Media Manager API so that developers can write better integrations. I don’t believe there will be much increase in barrier-to-entry by splitting these files up. As Helen said, developers who dig into this are likely to already be familiar with the build toolchain, but may not need even it when using the files for reference.

Don’t think this is going to make huge difference. Generally whether you edit few places in a single file or have to open and edit several files doesn’t make a difference. Your IDE should handle both cases transparently However agree that it is better to have a nice file structure that somewhat matches the JS structure.

In my mind it is more important to improve/fix the “JS structure”, i.e. this:

The main canvas on which these Views are stitched together are called Frames, which are themselves Views – tilting our use of Backbone more towards MVP, P standing for Presenter.

We have Controllers, which are called States, but they belong to a Frame (Presenter! also a View!), so anyways…. for now….

I’m hoping we will get that opportunity when implementing the image flow changes.

+1 for a more logical file structure. Most of the files here will most probably only be changed by experienced / core dev’s so the barrier to entry is not a major concern. The entry barrier can easily be solved by supporting documentation.

Media Grid started as a standalone plugin by Shaun Andrews, which was a reimagining of the UI as an alternative to the traditional post list view in the Media Library. The argument was that images are the ubiquitous media type in most users’ libraries, so we should provide an interface to browse media visually.

I joined the project in late April, attempting to integrate existing Media modal code. This work was merged into the standalone plugin, and into trunk(see #24716) in early June. In the process, I created documentation for the Media code, which is the most comprehensive resource for untangling the Backbone wires in media.

Questions were raised about what problem the grid was solving, so in order to get a more hands-on understanding of user engagement with the Media Library, Jerry Bates performed user interviews. These confirmed our assumption that images are the pervasive media type, but also surfaced the fact that users identify media in different ways – some by the thumbnail, some by what post a media item is uploaded to, some by title.

After a good amount of UX/UI chatter in weekly meetings, we decided we could serve users better by making a few changes to the original implementation merged into trunk. We’ve landed on mock-ups for a quick pivot, which I’m working on implementing . I’ll be dropping diffs for y’all Javascript Jedis to peruse on #24716, feedback welcome and appreciated. I hope to have merge-ables by Monday morning, and then to progress to user testing.

Thanks for creating the “WordPress Media Javascript Guide” and posting a link to it above. What’s the best way to ask questions about the Guide’s content or raise issues? For example, I am unable to make the “media.events” example in section 9 work.

The latest major updates to Audio/Video before Beta were related to editing shortcodes in the medial modal. TinyMCE placeholders for audio and video have been added like so:

When you click the placeholder, the media modal will now be launched. And when you get there, there are some new ways to manage HTML5 playback without writing anything by hand.

Add Multiple Sources

MediaElement.js provides cross-browser support for many extensions by providing Flash and Silverlight files that will bridge the browser support gap when necessary. Ideally, as many browsers as possible would be using native HTML5 playback. To ensure this, you need to specify multiple versions of your files. Since 3.6, the audio and video shortcodes have supported this by allowing you specify multiple extensions as attributes: mp3="yolo.mp3" ogg="yolo.ogg" wma="yolo.wma", etc.

A quick and easy way to encode your files is to use FireFogg (works in Firefox only). Here are some instructions for audio and video files:

When you’ve added them, MediaElement knows to support them out of the box:

Boom. Subtitles while your video plays:

When you add your subtitles, if will show you a list of “track” elements you have added, you will still need to set your language manually – all will default to English. The idea is that you can add a track element per language. Tracks get stored as the body of your video shortcode.

Testing + Tweaks

Now comes the time for testing and refining, while fixing esoteric bugs. PLEASE HELP! Put your UI + UX hats on, if you wear that kind of hat.

I honestly hadn’t thought about other use cases until Scott’s comment. All of the code is in media-editor.js, media-models.js, and media-views.js which get loaded via `wp_enqueue_media()`. If those files + MediaElement are loaded, you can do whatever you want. The admin glue is the “wpgallery” TinyMCE plugin where we have shoved all of the shortcode handlers, playlists too. So, you need your own glue, but see my above comment, it’s pretty easy.

1. What kind of metadata and how is it formatted? is it just a hash of audio type and url?
2. Will the same events shown above ( ‘update replace add-source select-poster-image add-track’ ) work with this code?
3. (wait, didn’t I ask for 3?) what does detach do?

Thank you and sorry for all the questions. Digging through the code is tricky because I’m still new to backbone so I’m glad that people in the know such as yourself are will to share the knowledge. Can’t wait for documentation.

@Scott Taylor: Would it be possible that you update Codex pages with examples for developers such as `wp.media.audio.edit( ‘[ audio ]‘ ).open();` etc.? It would be beyond awesome to have this documented in detail, with properties, methods, exact parameters, return values, examples etc. I’m not that proficient in JavaScript and it’s hard to find information such as what the actual data (parameters) are for `wp.media()` or for `metadata: {DATA_GOES_HERE}`. If you hadn’t told us that we can do `wp.media.audio.edit( ‘shortcode-here’ ).open();` then few people would be able to get to know that from the source code.

Where is the relationship between video and subtitle file stored? I’ve played with the latest nightly and I can’t find anything in the posts, postmeta, or options tables that indicates which subtitle file goes with which video. I’d like to be able to query videos with subtitles vs. those without.

Is there any way to filter the box to add some fields: besides “Autoplay” and “Loop”? I would like to add the ability to select the video start volume and another field to define whether to use the video as background (according with a theme I’m working on). Is there any tutorial or resources? TIA, Manuel

Thumbnails for Audio and Video

We have been parsing the ID3 tags in audio and video files on upload since 3.6. ID3 tags, in many cases, contain the binary bits for an image related to the item (album cover, still, etc). On upload, if your theme and the relevant pseudo post type for attachment:audio or attachment:video registered support for thumbnails AND your current theme supported it, we would upload the image bits and attach to the media item as its featured image.

So basically it was a hidden feature, because it required you to jump through hoops to turn it on.

On top of that, if you switch themes, and the theme doesn’t support thumbnails for audio or video, the images will no longer appear alongside the media on the Edit Media page. Weird.

Playlists are best enjoyed with images, videos are best enjoyed with poster images. Soundcloud is doing some awesome things with images in their embeds – see a few on the homepage here: http://highforthis.com. Moral of the story: I think this support should be on by default. Alternately, we could add that code to every default theme’s functions.php, but then what if you switch themes…

Playlist UI

Translation: it needs to show up in a theme without much drama, inherit the style of the theme, respond the theme’s $content_width, all while allowing you to completely override this behavior. So, what I have is an ultra generic design controlled by settings in the media modal:

I have tested this in the last 5 default themes:

Twenty Fourteen

Twenty Thirteen

Twenty Twelve

Twenty Eleven

Twenty Ten

I would like to drop this code in soon, but I wanted to give an opportunity for feedback. All of this can easily be iterated upon once it goes in.

Documentation of 3.5 Media Code

This is ongoing – there has been a lot of code churn in the Backbone code, by myself and others, I’ll be picking this back up once that settles down.

I’ve been working on similar things recently (bulk import of audio files + multiple playlists among other funny things).

I came across a problem for the thumbnail (I use the ID3 tag too): detect duplicates. If you upload 10 audio files from the same album, you’ll have 10 identical image attachments too 😐

So far, the only way I have to avoid creating the same image attachment multiple times, is to store the raw data in an array until the bulk import ends, and check for duplicate for each audio file. So it’s pretty limited, but it works, as long as you import all the album files within the same import.

@Manny@Manuel Schmalstieg:
Indeed, each track could have its own artwork within the same album, that’s why I ran into this duplicate detection :/
It’s too bad but I think we can’t do anything for that 😦

Looking awesome indeed! Just one little thing: on a site that is run by a band, they should be able to switch off the “by NAME OF THE ARTIST” after each track title. This seems even more important than showing/hiding the track numbers. Especially if it’s an annoyingly long band name, all in uppercase.

I would love to see an easy way to get thumbnails for uploaded audio / video! Working on the CEUX project we have content blocks for media. Getting a poster image for fetched video (at least for youtube and vimeo) is easy, as the providers have an API that allows it, but getting this for self hosted media is another case. And media playlists is a great feature that we didn’t thought about yet. Maybe we’ll need a content block for that!

@sourceforge and @jk3us – you can use external files, but editing them in the modal won’t work as expected, because you can’t use the shortcode. The About page in core uses external files, sans shortcode:

MediaElement

Documentation of 3.5 Media code

This is underway, with several commits already. Slow-and-steady approach. Have started with annotations, gradually adding top-level comments to all class methods. I have been using pseudo-code to map out each file: https://gist.github.com/staylor/10115a0f455e16c6eafd.

Start with identifying every piece of the tree, then making sure all methods have each piece of the documentation.

Dashicons

These will replace the “crystal” icon set. For AV2, we are interested in replacing the icons that show up in the media list tables, and using the icons in the placeholders for Audio, Video, Playlist, and Video Playlist shortcodes in TinyMCE content.

Metadata Regeneration for audio/video

a button on the Edit Media page if your audio or video does not have a row in postmeta for _wp_attachment_metadata (! isset(), not ! empty())

On the fly in wp_prepare_attachment_for_js()

#1 requires the user to care

#2 IMO does not scale, as the media modal loads ALL of your media in one fell swoop upon being opened – let’s say you have 300 videos uploaded pre-3.6: there will be no metadata, so you might melt your filesystem by scanning and analyzing 300 files at once.

This is still in the exploratory phase – would love some UX thinking on this.

Playlist and Video Playlist shortcodes

The TinyMCE view code can probably be merged into the wpaudiovideo TinyMCE plugin once #26628 goes in, which has its own dependencies. As I mentioned, Playlists and Video Playlists probably need their own identifying icons.

I have created a basic UI for playlists that inherits styles from your existing theme. The HTML and CSS is minimal and generic. Here are some screenshots of it in action:

As a plugin author struggling to add functionality to the Media Manager Modal Window I’ve spent countless hours going over the WP code and the code of other plugins which go to conflicting extremes to extend the WP code. JSDoc comments are much needed and appreciated, but don’t provide much insight to the larger design concepts behind the major parts of the work, such as controller/StateMachine/State/states/workflow and Library/toolbar/menu/router/Region/Selection.

I’ve seen (very popular) plugins that copy source code from media-views.js, modify it and then replace the WP method altogether by assigning their function to the WP .prototype. There must be a better way…

Are there any efforts or plans to illuminate the thinking behind Media Manager and/or to make it possible for plugins to cooperate on extending it?

Thanks again for all your hard work on this difficult and important task!

I wanted to do a Make post on my wants for Audio / Video in 3.9 to solicit feedback and spark some discussion about what the community wants / needs / doesn’t want / doesn’t need. Adding audio / video in 3.6 was a great first step, but there are some things we can do to continue to modernize Media and give our huge user base even more ways to display and manage their content. We can also make some changes that help developers navigate the new world of MediaElement.js, Backbone, and Underscore.

First Things First: New Icons

There are some lingering icons in the admin that don’t look as pretty as their MP6ify’d brethren

Document the “new” Media code introduced in 3.5

In 3.5, we overhauled Media. @Koop produced some beautiful code, and a LOT of it. Raise your hand if you’ve ever dived in and tried to program against it? Raise you hand if you understand how all of it works? Me neither. As a community, we need to help each other learn what it is and what it does. Documentation will go a long way in getting us all on the same page. Do we have a documentation standard for JS? We need one. While this isn’t the easiest place to start, it is a necessary one. I would be happy to spend time on this, as I have spent many hours recently reading the code and learning how it works. The main files: media-editor.js, media-views.js, media-models.js

Generate audio/video metadata on-demand

Add “Playlist” and “Video Playlist” shortcodes

Adding inline players for audio and video was a great first step. How do I add music to my site? Just upload an MP3 and drop the URL on a line by itself. Done. Or use the audio shortcode. This works most of the time, but can be a little clunky if you want to share an album of your tunes. MediaElement doesn’t “support” playlists out of the box, but MediaElement is JavaScript, and with JavaScript and little UI elbow grease, we can EASILY support playlists.

My ticket already contains a patch, but is still considered a work in progress. I think the playlist shortcode should produce markup that does the following:

Works out of the box with any existing theme: the HTML should be semi-bulletproof. Many of the Player libraries make heavy use of DIVs instead of items that might be overridden easily with CSS: LIs and the like.

Gives the developer total control if they want to write their own implementation

Exposes enough data to the page so the themer/dev can make their own decision regarding display of album cover, track meta, captions, etc.

My current implementation drops data onto the page for each playlist inline. A wrapper div “.wp-playlist” will have a script tag in it with type=”application/json”. I do this so that if ‘wp-playlist.js’ is unenqueue’d, the developer still has the data necessary to write their own implementation. The data is reachable in JS like so:

var data = $.parseJSON( el.find('script').html() );

My current UI for playlist is a basic one, and uses Backbone Views to render the tracklist on load and update the “current” playing track view. There are 2 camps of people when it comes to “JS on the frontend” – one who doesn’t like it (others) and one who says “who cares” (me). One of the reasons I am posting this at the beginning is so we can flesh out issues like this.

Abstract Gallery logic into “Collection” logic that Galleries, Playlists, etc can use with minimal registration code

I have already done a first pass at this in the playlist shortcode patch. It goes like this: a “gallery” is really a “collection” of attachments of type “image.” A “playlist” is really a “collection” of attachments of type “audio.” So they should extend or be instances of a “collection”-type class. Currently, the Gallery code has to be dupe’d. By abstracting this code, Gallery, Playlist, Video Playlist, + any other “collection” of media type can be registered minimally.

Thoughts?

Definite thumbs up from the Accessibility team for adding captions support. In addition to that, I’d like to suggest making it possible to enable keyboard accessibility more easily than it is right now. MediaElement.js includes settings which enable this — one is enabled by default, which enables keyboard access to the controls, but without ‘alwaysShowControls’ enabled, it’s not possible for a keyboard dependent user to move focus onto the player, so they can’t take advantage of any of those controls.

I’d like to see a localization variable so that the MediaElement settings can be adjusted via wp_localize_script, but for keyboard accessibility it may be valuable to make this an option, either via shortcode or settings, so it’s possible for non-programmers to enable keyboard accessibility.

I’ve got a plug-in, Accessible Video Library that hacks in support for alwaysShowControls by deregistering the default wp-mediaelement script, but that’s not an ideal solution.

It would also be nice if the caption selector within MediaElementjs could be made keyboard accessible; though that’s probably something that should be handled as a patch to MediaElementjs itself.

The Add Media Panel, in addition to new documentation, is in desperate need of accessibility work. See #23560. And several other tickets; there are quite a few accessibility tickets on the Add Media Panel.

the JSDoc standard is pretty close to the phpdoc standard that we use for PHP and thus it makes the most sense in my eyes. There was a passing IRC conversation between members of the team that did the jshint work and the inline documentation that a future project could focus on improving the documentation of our JS. The media files seem like a great place to prototype this.

The inline docs will soon (ish) be present on developer.wordpress.org, which will help the visibility of them. Do you mean something more like a walkthrough on exactly how it works? I’m trying to think of where the best place for that would be. Definitely on the developer hub, but probably not part of the theme/plugin handbooks.

Not sure to be honest. I think there are limits to inline documentation. Inline is always so contextual to the code it surrounds; separate documentation can speak of overarching design principles and secondary information that doesn’t pertain to any code block in particular.

WP-API’s documentation for example is quite verbose, and covers a range of topics including the API’s philosophy, tutorials, schema details, etc. None of this could fit easily into inline documentation.

Full disclosure: I’ve only used the media library cursorily, and am not even sure what secondary documentation for it might look like, so I may be off-base here.

We provide custom Theme WordPress websites to our customers and the primary challenge we face, regarding media, is that customers frequently are embedding video from sources beside the big guys (YouTube, Vimeo, etc…) so they have HTML or something. But that requires them to have access to the HTML (insert blood curdling scream here). What would be a potential solution is a simple widget on the right of Pages/Posts that allows them to paste whatever code they have, then see a rendered version without reloading the page, then the ability to drag that rendered video sample into the Visual Editor.

As much as I like shortcode, customers still don’t get it for some reason. Anything they hear the word “code” they freak.

the shortcodes get inserted automatically from the Media Library, don’t have to be written by hand – they are just the easiest way to save the most minimal data necessary in the post content without hardcoding markup or URLs

Sounds like a pretty wide scope of things. I’m primarily interested in extending the Media Editor, and have prepared a feature-as-plugin to this effect as a proof-of-concept. As I missed the deadline for 3.9, I was planning on waiting until the next round to bring this forward, but if work is being done in media-editor.js and friends, perhaps this is a good time to look at the challenges that the current architecture of the Media Editor presents, and my proposed solutions to it.

Okay, well, when I get a few hours, I’ll throw up a post and share the code. I’ve completely modularized the “editor groups” – which are basically the Media Editor’s version of Meta Boxes. It is currently **possible** to extend the media editor without core modifications, but it’s really ugly and involves duplicating some core files. This is one of those cases where it would really benefit from some well-placed hooks.

Half of what’s important in WP is making sure that it is easy to create great stuff and publish it on the web.

The other half is people finding that great stuff. Right now other platforms do a better job of sharing and redistributing great stuff. Better or more sophisticated oembed would probably mean more WP posts that look “right” when someone shares them on another site or tries to link to them.

That sounds really interesting to me – but I’m not sure what you are thinking about there.

As the author of Media Library Assistant (https://wordpress.org/plugins/media-library-assistant/), I have devoted quite a bit of time and effort to enhance the Media experience in WordPress. One of the frequent comments I get in my support forum is “this should be in core”, and I would be delighted to see any relevant parts of my work get to a wider audience.

Two of your proposals are of particular interest. First, “Document the “new” Media code introduced in 3.5″. I have had several requests to enhance the Media Manager Modal Window, and I have devoted a frustrating amount of time to understanding the code behind it. One of the best aspects of WordPress is its provision for actions, filters and other extension mechanisms; these are nowhere to be found in the Media Manager code. Consider these support requests I’ve received and responded to:

I would welcome an opportunity not just to understand, but to improve the Media Manager code.

Second, you propose to “Abstract Gallery logic into “Collection” logic that Galleries, Playlists, etc. can use with minimal registration code”. This is a great idea, and should be extended to all of the items that can be stored in the “Media” Library, not just image, audio and video items. There are many sites using WordPress to manage large collections of other assets such as PDF documents. Let’s give them some attention as well.

I look forward to following your progress and finding a way to contribute something to it. Thank you!

With this move away from galleries to collections, (an idea which I really like btw), the gallery code should be rewritten to use walkers. This would make it much much easier to override the markup that galleries output.

Post Formats are a big feature in WordPress 3.6. What you may not know is: there is now native support for Audio and Video in core! There has been great support for embeds by way of WP_Embed and oEmbed providers for a while, but, if you wanted to play an MP3 from your Media Library, you had to install a plugin. Supporting audio and video in core gives bands, podcasters, vloggers, et al the ability to easily and beautifully expresses themselves through sounds and moving pictures without using an external service.

How does this work?

At the core of the experience is the fantastic library, MediaElement.js. MediaElement is the facade layer that gives us maximum file support and cross-browser compatibility. While some libraries require a Flash-only solution to make your media work cross-environment, MediaElement lets you use HTML5 audio / video tags in every browser, and, only when necessary, will use a Flash or Silverlight plugin in the background to make incompatible media work. Translation, things like this: <audio> tag works in old IE, Windows Media files work in Chrome.

MediaElement uses the same HTML markup, regardless of playback implementation, and you can use CSS to skin the players.

Shortcodes

MediaElement’s great, but we don’t want to be locked in to one external library forever. Instead of using MediaElement-specific markup everywhere, we expose audio and video markup through shortcodes: [audio] and [video].

For the following scenarios:

I have an old post that has a video in the Media Library attached to it, and I want to use the new shortcode: [video]

I have the URL for a video, from the Media Library or external, that I want to play:[video src="video-source.mp4"]

I have a source URL and fallbacks for other HTML5-supported filetypes:[video width="600" height="480" mp4="source.mp4" ogv="source.ogv" webm="source.webm"]

Same goes for audio:

I have an old post that has an audio file in the Media Library attached to it, and I want to use the new shortcode: [audio]

I have the URL for an MP3, from the Media Library or external, that I want to play: [audio src="audio-source.mp3"]

I have a source URL and fallbacks for other HTML5-supported filetypes:[audio mp3="source.mp3" ogg="source.ogg" wav="source.wav"]

Shortcodes focus on the “what” and abstract the “how.” If you want to use a library that is not MediaElement, you can! Just look at what to filter: here

Embeds

There are also new embed handlers for audio and video. Using them is easy as dropping a media link on a line by itself in the editor:

http://my.mp3s.com/cool/songs/coolest.mp3
I like this song because it is really cool!

Works for both audio and video with URLs matching the allowed (and filterable) list of extensions (see: here and here)

Admin

Using the new post formats UI, it is even easier to get directly at the audio and video in your Media Library. When selecting, the media modal opens to your library, filtered by media type.

Metadata

In previous versions of WP, you could upload audio and video, but we were not generating metadata like we do for images. In 3.6, using the getID3 library, we are able to extract data from audio and video like cover art, song length, artist, album, song title, genre, codec, etc. It’s pretty great. We will soon be exposing more of this data in the admin as well, along with inline previews on the Edit Media page:

Themers

Themers can get in on the action, too, using structured-post-formats in their theme (Twenty Thirteen is a great place to look). The admin gives users flexibility when associating media with a post. the_post_format_audio() and the_post_format_video() will automagically retrieve and output your media in the front end.

I just spent a month (not continuously) trying to figure out why MediaElements.js conflicted with Soliloquy (Flex based Slider) when both appeared on the same page on mobile. Only on mobile, everything worked fine everywhere else. I finally gave up, ditched ME.js for Video.js.

I’m now about to test that site on 3.6 just out of curiosity as to what happens.

I too really dislike this using shortcodes and my bigger concern is what this does to other plugins that use the shortcode already.

Always seemed to me WP ought to follow best naming practices and use [wp_gallery], [wp_video], etc…

That was easy to test… still conflicting somehow. I’ve let Thomas know with urls to dev/staging/live servers showing it all. It’s really bizzare that it only happens on mobile browsers (iOS chrome and safari) anbd throws no errors. Either works fine on it’s own, and we’ve recreated it on vanila WP running 2010.

Yeah, this is cool. And I miss some important points to have this as a useable feature for real blogger’s life: mejs is out of the box not resonsible and this allone makes the joy half at the first glance. Yes, there is a dev’s tip for videos out there, that, if you set the width to “100%” it will work. And it does, indeed! Maybe this width issue of mejs videos should go into core?

Responsive mejs audio seems to be more complicated. A simple width attribute setting does not work. At this time the width of the audio bar overlaps the standard width of 480 even in modern smartphones.

Regarding video: the poster attribute of the shortcode is rather important, since it leeds to a screenshot like above, showing this nice preview picture for the video – but it’s not as easy to implement as it looks like. If you take an image from the media library with its predefinded sizes, it is to small or you’ll have an overlapping picture. For me setting the CSS class “mejs-container” to “overflow: hidden;” seems to resolve the issue as a quick hack.

I think, the feature of having core supported video and audio is great, and it should be delivered in a way, that avoids frustration of users. The poster feature for videos is essential I think, the contra responsive issues should disappear as well.

I see a lot of potential for the post formats. I see many problems though.

If it is in WP core, it should be capable of both themes and plugins to utilize the functionality. At present the formats are hard coded and there’s no way for themes/plugins to add additional formats. Worse yet, if a theme only implements the audio/ format, it appears to process all 6. (line 203 of wp-admin/functions/post.php). There are no action hooks / filters either.

The post formats still display even though older themes do not call the function add_theme_support( ‘post-formats’, …). Plus if a theme only specifies 1-2 formats, only those specified formats should be available when editing posts. It does not appear to let you add custom formats either, which would be the bee’s knees.

Who ever is managing (supervisor or committee chair) of the post formats features could contact me, that be great. My email is cio [at] rawvoice dot com.

Have been playing around with it while developing a couple of themes for 3.6. It’s lovely, and easy to style. Have been using MediaElements,js before, and when i found out that it would be included in the Core, i was thrilled. Good move!

I have to say, I’m really disappointed that there’s no mechanism for people to add captions for videos or provide text transcripts with audio files. come on, people! We need to be encouraging people to do this kind of stuff but unless WordPress provides the methods, it just won’t happen.

The “first run” of a simple image editor has been in WordPress 2.9-rare for about two weeks. It lets the user crop, flip and rotate an image. It also has an option to scale the original (largest) image and to restore the originally uploaded version. There is a multi-step undo and redo support. Changes can be applied to all image sizes or only the thumbnail. The initial code is by Stephan Reiter author of the Scissors plugin.

Currently when saving an edited image all previous intermediate image sizes are kept and the meta data for them (filename, width, height) is moved to another postmeta field. This is necessary to avoid breaking any posts or pages where the image was inserted. The editing workflow (multi-undo, preview of each step) makes it unnecessary to edit an image more than once or twice, however if an image is edited 6-7 times the backup files would be 25-30.

There are couple of ways to avoid keeping unnecessary images, we could either track when an image was used in a post or a page, or we could have “standard” names for the intermediate image sizes. Tracking image use can be somehow problematic considering all the different ways content can be published in WordPress.

Renaming the resized images from [imagename]-150×100 to [imagename]-thumbnail, etc. seems the more viable option. This would also bring instant image updates in already published posts, however may also have some compatibility problems with plugins that expect the image size to be part of the filename.

The URL, width, and height would all be dynamic and change if you changed the thumbnail size, etc. This is opposite of now where even if the filename didn’t contain the dimensions, you’re still left with hard coded width/height parameters in the image tag.

Since both uploads would have a unique id in the DB, it doesn’t matter if they have the same filename. The correct URL would be output dynamically, based on wherever it was stored (as referenced by the DB)

Think this came up when we were doing the captions shortcodes a year ago and was rejected. Perhaps we can look at the pro & con again. Since all movies and other embeds will be handled by shortcodes it may make sense to extend that to include images too.

However I see a possible problem with that. The img tag doesn’t need any processing on display but a shortcode would need quite a lot as it seems we will have to do get_post() for the attachment. Also our shortcode parsing code/API doesn’t seem optimized enough to handle large quantity of shortcodes.

It is true that it can’t handle really large posts with lots of shortcodes. However, that’s definitely only affecting a very small percentage of people (if we’re really focusing on the 80-20 principle, it’s less than 20%). Do we really think that enough people will be writing stuff this long (and putting enough shortcodes in it) that we should worry about it? (That’s the post where I first ran into the issue…I tried to explain that it was just too long, but apparently it’s quite effective as it is).

I use similar code in two plugins of mine, and since you can’t rely on an attachment from being tied to a single post, I check for a post meta to make sure there are attachments (and which ones) before tossing a query.

We also have to not forget about those who use TinyMCE and expect to see the image as is. A shortcode would be there as text.

Perhaps a TinyMCE plugin could AJAX fetch the image (based on ID) and replace the shortcode with the image during editing, then put it back to the shortcode on save. That way it’d still stay a WYSIWYG.

Filesize (aka fat) is not directly related to performance. The image editor for example has literally zero (no joke on the literally) impact on the front page of your site for example as it’s never loaded except in the admin area.

It’s all about the 80-20 rule. I think more than 80% of the people will use this on a regular basis. I probably won’t, but most of my clients will. I’m all for keeping code as lean as possible, but I’m also all for adding this support in. Let’s trim in other areas. Just my 2 cents.

Another solution for the multiple image backup files would be to let the user decide. We can include a checkbox whether to delete the previously edited image (together with all sub-sizes). Since the suffix that is appended to the image filename contains a timestamp, we could do something like “This image was last edited 1 hour ago. If it wasn’t used in a Post since then you can remove the backup files by selecting this checkbox”.

On the other hand even the cheapest hosting accounts seem to come with a lot of storage so deleting backups would be unnecessary. Also a plugin can add a simple GUI to search the db and return a list of unused backup images that can be deleted. Not sure if something like this would be good for core.

I think the key to this problem is the following: “This is necessary to avoid breaking any posts or pages where the image was inserted.”

Why avoid breaking those posts/pages? If somebody edits an image, then change the image itself. That’s what they will probably expect to happen.

Having to have them go and reinsert the changed image into the page/post is confusing and unnecessary. We don’t need to save the original. The user uploaded the image, presumably they have the original. An in-place editor should simply overwrite the existing files with the new ones, not try to preserve them.

True, users that know what they’re doing may expect to break old posts when editing an image. However the majority will most likely complain that “the image editor breaks all old posts!”.

Also we cannot just replace the originally used image with the edited one as images are inserted with “width” and “height” attribs that in most cases will distort the edited image (as Viper007Bond pointed above).