To overwrite that output copy the default.php file from plugins/content/pagenavigation/tmpl to templates/[your_template_name]/html/plg_content_pagenavigation/, then you can edit that file and change the output above.