Media Deduper

Description

Media Deduper will find and eliminate duplicate images and attachments from your WordPress media library. After installing, you’ll have a new “Manage Duplicates” option in your Media section.

Before Media Deduper can identify duplicate assets, it will build an index of all the files in your media library, which can take some time. Once that’s done, however, Media Deduper automatically adds new uploads to its index, so you shouldn’t have to generate the index again.

A “Manage Duplicates” page listing all of your duplicate media files. The list makes it easy to see and delete duplicate files: delete one and its twin will disappear from the list because it’s then no longer a duplicate. Easy! By default, the list is sorted by file size, so you can focus on deleting the files that will free up the most space.

A scan of media files as they’re uploaded via the admin to prevent a duplicate from being added to your Media Library. Prevents new duplicates from being introduced, automagically!

Media Deduper comes with a “Smart Delete” option that prevents a post’s Featured Image from being deleted, even if that image is found to be a duplicate elsewhere on the site.

If a post has a featured image that’s a duplicate file, Smart Delete will re-assign that post’s image to an already-in-use copy of the image before deleting the duplicate so that the post’s appearance is unaffected. This feature only tracks Featured Images, and not images used in galleries, post bodies, shortcodes, meta fields, or anywhere else.

Looking for more features? Media Deduper Pro includes features for image fields from several popular plugins as well.

Note that duplicate identification is based on the data of the files themselves, not any titles, captions or other metadata you may have provided in the WordPress admin.

Media Deduper can differentiate between 1.) media items that are duplicates because the media files they link to have the same data and 2.) those that actually point to the same data file, which can happen with a plugin like WP Job Manager or Duplicate Post.

As with any plugin that can perform destructive operations on your database and/or files, using Media Deduper can result in permanent data loss if you’re not careful. Back up your data before you try out Media Deduper! Please! We really don’t want you to destroy your stuff!

Need more support? Media Deduper Pro includes dedicated support from Cornershop Creative.

FAQ

How are duplicates computed?

Media Deduper looks at the original file uploaded to each attachment post and computes a unique hash (using md5) for that file. Those hashes are stored as postmeta information. Once a file’s hash is computed it can be compared to other files’ hashes to see if their data is an exact match.

Why does the list of duplicates include all the copies of a duplicated file and not just the extra ones?

Because there’s no way of knowing which of the duplicates is the “real” or “best” one based on your preferred metadata, etc.

Should I just select all duplicates and bulk delete permanently?

NO! Because the list includes every copy of your duplicates, you’ll likely always want to save one version, so using Delete Permanently to delete all of them would be very, very bad. Don’t do that. You’ve been warned.

Instead, we recommend using the Smart Delete action (which is also found in the Bulk Actions menu). Smart Delete will delete the selected items one by one, and refuse to delete an item if it has no remaining duplicates. For example, if you have three copies of an image, and you select all three and choose Smart Delete, two copies will be deleted and the third will be skipped.

At this time, Media Deduper only identifies and blocks duplicate media files manually uploaded via the admin dashboard — it does not block duplicates that are imported via WP-CLI or the WordPress Importer plugin.

Does this work with any plugins?

The free version does not include plugin support. Media Deduper Pro integrates with a number of popular plugins, including WooCommerce and Yoast SEO.

How can I contribute?

The git repository should be publicly available at https://bitbucket.org/cornershopcreative/plugin_media-deduper. Feel free to fork, edit, make pull requests, etc.

This is, in my opinion, a great, powerful plugin that will help you manage media libraries, especially if you have lots of items.
Being it very powerful, it is easy to make mistakes if you do not pay attention but if you manage it correctly it is an ally that will save you a lot of time.

Very good and time-saving plugin. I wish there was support for wp-cli so that I could remove duplicates via command line.
Suggestion: Make a file type filter, too. I just wanted to smart delete photos, not anything else, because smart delete doesn't work on them and they all get deleted. A file type filter on the selection age would be appreciable.

Thanks a lot for such a plugin. Please, keep up the good work.
Could you please add custom multilingual warning message for backend and frontend to inform that "the file you are trying to upload was uploaded before".

Interested in development?

Changelog

1.5.2

Fix an issue that could cause a performance hit or MySQL bottleneck on sites with large numbers of posts during initial activation, or when upgrading from 1.4.2 or earlier to 1.5.0 or 1.5.1

1.5.1

Fix missing files in initial release of version 1.5.0, which could cause fatal errors

1.5.0

Media Deduper now indexes attachments asynchronously, meaning you don’t have to keep the browser open while it indexes your media library.

1.4.2

Media Deduper now fully supports the media trash feature (a.k.a. the MEDIA_TRASH constant; see this post for more information). As of Media Deduper 1.4.2, if the media trash feature is enabled:

The Manage Duplicates screen will list (and allow the user to delete or Smart Delete) both trashed and non-trashed attachment posts. Previously, only non-trashed attachments were listed on this screen, which caused confusing behavior if, for instance, a site’s media library contained two copies of an image, but one was in the Trash.

If a user attempts to upload a media file that already exists in the media library, but the existing copy of the file is in the Trash, the user will no longer be prevented from uploading the new copy.

Bugfix: The “Attach” or “Detach” links in the list table on the Manage Duplicates screen are now working again, after they stopped working in a previous version of the plugin.

1.4.1

Tweaks to messages about Pro: display on fewer pages, don’t display to users who don’t have permission to install new plugins, change wording slightly for clarity.

1.4.0

“Delete Preserving Featured” has been renamed to “Smart Delete”.

Smart Delete logic has been changed so it will never delete the last copy of a media item. Previously, Delete Preserving Featured would only save the last copy of a file if it or one of its duplicates was used as a featured image somewhere on the site.

By default, the list of duplicates is now sorted by size (largest first), and secondarily by date (newest first).

Media Deduper will now warn users before executing a Smart Delete operation, so you have a chance to cancel if you change your mind.

We’ve added information about the newly released Media Deduper Pro.

If your site is running PHP 5.2 or earlier, Media Deduper will now deactivate itself instead of causing errors.

Bugfix: The second bulk action menu, at the bottom of the list of duplicates, was not behaving correctly under some circumstances. This has been fixed.

Bugfix: Prior versions of Media Deduper could potentially have affected how other plugins’ screen options were saved. This issue would only have affected plugins that did unusual things (e.g. data validation logic) on the set-screen-option hook, and we haven’t run into this issue in the wild, but now it’s fixed anyway.

1.3.2

Bugfix: Fixing an issue that caused bulk actions (Delete Permanently, Delete Preserving Featured) to not work correctly on some versions of WordPress.

When a media file is changed (using the built-in WP image editor or the Enable Media Replace plugin, for instance), the plugin will now re-check whether the updated file is a duplicate of another file in the media library.

Minor code cleanup.

1.3.1

Bugfix: Fixing an issue that caused attachments to be listed as “duplicates” even after all identical attachments had been deleted.

1.3

Refactoring PHP to take advantage of WP 4.7’s new hooks for handling bulk actions.

Minor improvements to ‘success’ messages displayed after a bulk action has been performed on a set of duplicate attachments.

When uninstalled, the plugin will now delete the ‘mdd_size’ meta field that it adds to attachments. Previously it would only delete the ‘mdd_hash’ field.

Minor code cleanup.

Bugfix: Fixing an issue that caused the columns on the Manage Duplicates screen (File, Author, etc.) to not actually be sortable.