Is W3C Replicating the WordPress Pingback System?

This post was contributed by guest author James Richman. James writes about marketing, digital design, entrepreneurship, and technology. He has gained most of his experience from running a variety of his own businesses for more than a decade.

On January 12, 2016, the W3C (the World Wide Web Consortium) released its first public working draft of Webmentions, but the announcement didn’t feel new. In fact, Webmentions have been around since IndieWebCamp created them in 2013, and top WordPress developers are already using a Webmentions plugin to utilize the tool.

Yet, for those who use WordPress, Webmentions seemed like a retread of WordPress’ Pingback system from the early 2000s, which featured a similar concept.

The Pingback system was invented in 2002 by Stuart Langridge, and in essence, it allowed pieces that were published on different WordPress sites to talk to each other. This is perhaps best explained by walking through an example scenario:

Website A posts a new entry on its blog.

Website B responds to that blog post with its own post and links to the post on Website A.

The Pingback system then notifies Website A that Website B wrote about and linked to its blog post.

Website A then verifies the content and link on Website B. If it is not spam, Website A will post a comment on the original blog post linking to Website B’s post.

It’s important to note a few things about the Pingback system. First, it is exclusively for WordPress sites, and both sites have to enable Pingback for it to work. Second, the whole Pingback system is automated, streamlining the process of trackbacks, which is the manual equivalent of the automated Pingback.

Yet despite its perks, Pingback system usage declined after the automatic system was taken advantage of by spammers.

The issue of spamming and abuse of such a communication channel has long been the problem with this type of communication channel. Prior to Pingbacks, WordPress used a Trackback system that provided the same type of communication.

The only difference between the Trackback system and Pingbacks is that Trackbacks had to be inputted manually. Spamming problems were just as prevalent with trackbacks. In fact the WPTavern site shut down in 2010 because of trackback spam.

So what makes Webmentions different than the Pingbacks and Trackbacks that came before? Well, not all that much. Turns out, Webmentions do the exact same thing as the Pingback system; they just do it better.

The biggest difference between the two is the code they’re composed of. Pingback uses XMLRPC, an outdated approach that encodes data with XML and then transports that data with HTTP. The Pingback system is bulky and slow. Webmentions, on the other hand, uses HTTP and x-www-form-urlencoded content, a much more widely accepted format in today’s world. The result is that Webmentions is much faster and much easier to integrate.

As Pingback’s creator Langridge points out on his blog, “XMLRPC is considerably less popular than it was, and is really heavyweight for this sort of thing. We’ve learned since then that HTTP can actually do all this stuff for us way more simply.” If you haven’t guessed, the Pingback founder has converted to Webmentions too.

Webmention’s growing popularity is due to the tool’s ease of use and the fact that it blocks spam effectively with the Vouch protocol. But Webmentions also look better aesthetically in the comments section. Pingback comments look robotic and aren’t exactly readable; a Pingback comment contains the title of the post that sent the Pingback and an ellipsed summary that doesn’t make much sense.

Webmentions look and feel like human comments with the help of the Semantic Linkbacks plugin. This plugin parses the Webmention linkback and translates it into a full sentence (e.g. ‘Sarah mentioned this post in her article x on site y’) and can even include the author’s profile picture.

Webmentions are quickly replacing the Pingback system because of the tool’s convenience and better implementation to reach a similar goal, and this has been happening prior to W3C’s public endorsement. So what does W3C’s support of IndieWebCamp’s creation mean?

Ultimately, W3C’s announcement will likely help cement Webmention’s place on the internet. And so W3C’s recent push can be and should be considered as an effort to standardize the wild web.

The internet was created to communicate and share information, yet individual pieces of content are unable to communicate with each other as easily as users can. The Pingback system was a worthy, but flawed, attempt to change that, and now Pingback’s legacy survives through the broader support and growing distribution of Webmention’s network. If Webmentions become more popular, maybe one day in the future, they will connect the strands of the web together, so that the web will have earned its namesake in truth.

Like this:

Related

19 Comments

Sorry, it is hard to take a post seriously when it claims that pingback is unique to wordpress and that handling XML-RPC is problematic in any serious way.

After glancing at the draft it seems like the webmention people haven’t learned anything from the failures of pingbacks and comments, if the whole point is to use JSON over XML, then you can just ignore that spec as it addresses the least important issue with pingbacks.

I can’t count how many sites I visit daily for years … tons of them.
I never find at least one nice or helpful example of pingback, trackback or whatever is that “irc technology called.”
I don’t know why this still exist on the web, especially in WP core. Probably some more experienced user know more about that thing. If I see it somewhere, for me it looks always like spam.

Also don’t understand why WordPress with 26% of all web, is just follower of any technology or cool thing, and never act as the leader (at least in some areas.)
– follow FB, Google, crappy hostings, follow anybody who hosts at least “two sites on higher version than first alpha” ;)
Even if WP is so cool and kind useful tool, it does not mean, it can’t be much better.

I agree with mark k. Saying “WordPress’ pingback system” the way this article does is really bizarre.

OTOH, does anyone ever see a pingback anymore *except* on a WordPress system?

Why would the W3C want to recreate pingbacks? Originally this was intended to fulfill the role that mentions and retweets play on something like Twitter, only distributed and not tied to a single platform.

And it pretty much failed spectacularly. It became spam central, and then WordPress helpfully repeatedly implemented XML-RPC in a way that introduced vulnerabilities.

Why would W3C waste time implementing a new version of a failed web technology?

W3C is just a governing body, I don’t think that using the phrase “waste time” properly represent how it works. If I got it right there is some “social interaction” group of the W3C for which this kind of thing probably falls in it mandate, so not really a time waste, but just a dead end idea.

Unfortunately, webmentions don’t solve any of the actual problems with the existing pingback system, it just replicates it with different methods, and adds a few nice-bits like how to display things. It’s still totally susceptible to spam (if you’re clever about it), and the DDOS problem still exists inherent in its basic design.

Though that page lists ways to mitigate the problem, all those same exact ways can apply equally well to pingbacks. They don’t much work there either, and the usual case of simply turning-them-off is likely to be the underlying result for webmentions as well.

So, I have to agree with the above comments. Replicating what you already have with a fresh coat of paint doesn’t really fix anything to speak of, and is kind of a waste of time.

Otto, while I have a great respect for you, I have to respectfully disagree on the matter. You are right that those same ways can be applied to pingbacks.

Webmentions is billed as an updated and simpler reimplementation of pingback. As someone who has opened(and tried to close) several pingback related WordPress trac tickets and intends to continue doing so, the pingback implementation in WordPress doesn’t have much going to encourage using it. I would like to change that.

The reason why the above comments are wrong in my opinion is that the argument is that a technology failed because people never did anything with it. Look at the Indiewebcamp community who are doing something with it.

The issue with pingbacks is no one ever made them worth using. You can make the same argument that email is totally susceptible to SPAM, but the information coming through was important enough to work on mitigation techniques. The display elements, which don’t depend on the protocol, are what would start to make this worth using.

And that same Semantic Linkbacks plugin the author linked to(by the way, its in the WordPress.org repository), also works on Pingbacks.

Don’t get me wrong. I’m not saying it is bad. I’m saying that, from what I can see, It is not actually any better.

If it solves actual problems, great! Please point those out to me. As of right now, at this moment, it solves nothing real and is mainly a concept. That doesn’t make it bad and I’m happy that people are thinking about the issues involved. But it is not a panacea, it is not a “fix”, and it doesn’t actually solve any real-world problems.

The whole “vouch” system is crap. It’s a concept with no real application and no real concept of how to implement it on a large scale. It is a valid idea, to be certain, but actually creating that in a real model would be insanely expensive for the senders and receivers. It doesn’t scale, in other words, which basically means that it does not work. This is not criticism of the idea, which is a valid idea, but they are really relying on centralization, which is anathema to the concept in the first place.

All I’m really saying is that other than using the magic of “hey, no XML”, there is nothing new here. This is the same stuff I’ve seen others consider for 8 years now. Nothing new under the sun. We deal with code, not magic. This ain’t tricky to figure out.

Maybe what fails is the idea that just because someone had linked to me I should care about it? It feels like someone trying to copy FB and twitter to the wide web, but in both I pre-select who I care about and probably will not be notified about others, while on the web people that mention me are more likely to be people I don’t know at all and therefor there will not be any explicit or implicit web of trust between us.

The vouch solution can work in a limited enviroment but just do not scale to the whole internet.

David, sadly, I think the very concept of trackback/pingback/linkback schemes itself is a flawed one because it assumes no-bad-actors in the notion.

Basically, the underlying premise is that I want people to be able to tell my site “hey, I linked to you, please link back to me”. The motivations here are many, but given that “links” are equivalent to a “currency” for SEO spam and such, then if you assume bad-actors, then you have to build a trust model. And nobody has built a decentralized trust mechanism that actually scales, short of crazy things like the Bitcoin blockchain. Which seems a bit far to go for linkbacks, I think. :)

Solve that bad-actors and trust problem and you’ve got something truly useful. But until then, everything else is either to centralize the problem (the Akismet approach) or to essentially ignore it in favor of cosmetic changes only. I agree that it is a problem in need of solution. I don’t know a solution, and after reading all of this new stuff, I still don’t see one. Webmentions is a minor enhancement on pingbacks, using newer and simpler technologies. It’s not earth-shattering. Props to them for looking at the problem, but I’m not surprised that they have not found the real hard answers yet. Maybe they will eventually.

Otto, I still think that until we try making something worth protecting, worrying about bad actors is premature.

If moving forward and iterating is not working toward a solution, then what is? As JFK said, “We choose to go to the Moon in this decade and do the other things, not because they are easy, but because they are hard”

The Pingback processing code is definitely in need of some love. And the changes committed to 4.5 allow plugins to access the raw HTML and enhance the output. So why not move forward with that?

David, the bad-actors problem is the only one that actually needs to be solved. There is nothing wrong with pingbacks in their current form other than that. Whether you use JSON, XML, or smoke signals is really irrelevant to the very concept itself. Those are mere implementation details, they are not the problem.

Re-implementing something in new ways doesn’t fix problems if you don’t actually, you know, *fix* those problems you found when using the old ways. Just creating something old using new methods doesn’t actually change anything of substance.

During my morning walk I actually found a solution to the fundamental problem of authenticating the pingback author, and it is to treat pingbacks exactly like comment and at to them an email address field.

This will remove the “unintended spam” from content scrappers, and can be managed under the current set of rule comments are handled now. If someone already have commented on the site his pingbacks will be accepted and displayed, otherwise to the moderator queue it goes.

This reduces the problem of spam to be the same as comment spam, which at least helps you reduce the effort fighting it, and after all isn’t a pingback just an automatic way to send a comment saying “I wrote about it something, you might be interested to read”?

First, it is exclusively for WordPress sites, and both sites have to enable Pingback for it to work.

It’s already been pointed out, but just wanted to reiterate that Pingback is an open spec that WordPress adopted, but it existed before WordPress and in other CMSes as well, we’re probably the last one standing though.

The only difference between the Trackback system and Pingbacks is that Trackbacks had to be inputted manually.

This isn’t really true. Trackbacks could be automatically sent to any linked site, to me the key differences were:

1. Trackbacks sent what content they wanted to post, rather than the receiving site choosing what it wanted to show.
2. They, by design, did not require a reciprocal link, this is so people could do things like a Technorati-like topic directory with them.
3. Discovery was a much more involved process, and involved RDF on the page. Pingback was discovered purely through HTTP headers which was a bit more efficient.

The RDF was because, at the time, it was common in Movable Type (who invented Trackbacks) for the permalink of a post entry to be an anchor on a weekly or monthly archive, not have its own page, anything in the URL after # wasn’t easily available to a server-side language, and many of the pages were static HTML anyway.

The no reciprocal linking and being able to send whatever content you wanted (rather than it being extracted from the page) did make it a bit easier to write spambots.

Webmention’s growing popularity is due to the tool’s ease of use and the fact that it blocks spam effectively with the Vouch protocol.

I think Vouch is a cool idea, and similar to how a predecessor to Akismet tried to work. If Webmentions doesn’t get much spam now, I suspect it’s largely because it isn’t widely adopted enough for spammers to pay much attention to it. I don’t know if Vouch will hold up to the techniques and machinations of modern-day spammers, who now often try to use multi-step processes of intermediate URLs on social media or other places with lots of user content, legit domains that turn spammy later, and leaving innocuous comments to gain “trust” and then spamming later.

But Webmentions also look better aesthetically in the comments section. Pingback comments look robotic and aren’t exactly readable; a Pingback comment contains the title of the post that sent the Pingback and an ellipsed summary that doesn’t make much sense.

This is entirely up to us! I think improving how pingbacks display could be a really interesting project, and we could completely change it in a new version of WordPress since the entire comment is created by the receiving site, not the sending one. We could support h-cards! I would be very supportive of a feature plugin or patch that iterated on this. Another rich area to work on is that we discard pingbacks sent to places like the home page, but actually that could be a nice replacement to the now-gone APIs that Google Blog Search or Technorati supported to let you know who was linking to your site.
Otto mentions the DDOS reflection problem, which is a real one and difficult to solve in a distributed way. The forwarded header we added to core and Akismet gives more information, but ultimately it’s global throttles in centralized services like Akismet and WordPress.com that mostly neutered the effectiveness of that attack.

More broadly, there seems to be a trend of doing something that was done a decade ago, but taking out XML. It’s Webmentions and Pingback, and we’re also doing it ourselves with AtomPub API and WP-API. For adoption it’s important to think about what makes things 10x better, not just 10% better. Protocols can have product-market fit just like products, and I think what we’re doing with the API has the potential to be really compelling, but distribution does not guarantee adoption. For Webmentions, I really appreciate the optimism and practical get-it-done attitude of the entire IndieWeb community.

I’d volunteer to be part of a project to improve pingbacks displays and/or to add webmention support(which is not mutually exclusive). I’ve proposed it in the past on WordPress Trac. The only thing that stops me from going full feature plugin is that I’d like the involvement of people much better at programming than I.