if you want a plugin that can have a default setup when you create a new sub-site then give this a shot WPMUDEV New Blog Template do not let the name fool you it does what your somewhat looking for minus the Duplicate button but that would be no longer necessary.

Late to the party here but wanted to mention for the benefit of future searchers that we had the same need and built a plugin for cloning/duplicating with all settings intact just like the OP describes.

It is called the NS Cloner, and it doesn't add a "Duplicate" action link as described here, but it makes the process almost that easy by creating an easy-to-use network admin page where you can select a source site and then create a perfect duplicate complete with all content, plugin settings, etc, and it tops the WPMUDev New Blog Template plugin by being free (their are some more advanced features you can add with add-ons but the very powerful core is forever free), adding more power/flexibility in cloning settings, and avoiding hangups with updates since it's a wordpress.org plugin.