Posted
by
ScuttleMonkey
on Wednesday May 31, 2006 @01:02PM
from the still-watching-through-the-window dept.

John Battelle is reporting on his blog that word has leaked about a possible new API from Google that would allow sites to distribute AdSense earnings to individual members based on submissions or participation. From the article: "To toss a bit of cold water here, however, I've never seen UGC sites as the least bit driven by money. They are driven by pride, the desire to be first, reputation, whuffie. But dollars? That often screws it all up. I guess we'll get to see soon enough..." Relatedly many users are calling the 'nofollow' tag "Google's embarrassing mistake". Justin Mason is just one of many to take a look at the current status of nofollow and what may still be in store for that particular tool.

Then I added an amazon affiliate link at the end of it. I earned $5 yesterday because somebody clicked on the link, then browsed to other items and purchased something. See, the link doesn't even necessarily need to be a related product, but obviously better if it is (otherwise you don't get modded up).

Before Google cranks out another money making extension it should concentrate more on preventing click fraud AND provide tools for websites using AdSense to protect themselves. I know now several sites now that have been kicked out from AdSense - because of Click Fraud - but Google offers no tools, no insights, no answers and no support for those kicked out. Ever tried to talk someone from Google's 'customer service'? No eMail addresses, hardly any responses, mostly ignorance.

Google like to run anything on autopilot and pure technology - no human contact and no problems please. So this will be another Google technology I will ignore, because I can't stand the company and it's current attitude behind it. 'Don't be evil' should be renamed into 'simply ignore everyone'.

I completely agree one of my sites got its adwords shut down because of a minority of people that were making fraudulent clicks, their purpose for doing this? To get my adwords account shutdown. Before these clicks started I had almost $200 in my account from valid clicks they started I got shutdown.When I attempted to contact google about this they said they will not provide information that could give away their IP methods. They give you the chance to defend against being shut but give you no informati

They give you the chance to defend against being shut but give you no information on what you are defending against. It is kinda like going to court to defend against charges being brought against you without knowing what the charges are.

You do understand that this is the "Law of the Land" here in the USA now...

Along those lines and speaking of fraud, I was researching a web promotion group, which promises high page results (and seems to deliver too) and I found the most peculiar thing. Take a look at this [google.ie].
If you look at the results there, there are hundreds of links from sites like 6246.u2mme6.info, but if you look at the code for these sites, they aren't as such links. The pages don't load in a browser because of an iframe to a non existent site, but if you download the pages via httrack or one of those, you see the a page of bizarre code. An example of that is here (you'll have to look at the page source to see it, I don't know how to show it in slashdot)...

Has anyone got any idea what is going on here, or how that might affect their page ranking? Should I report them to google?

Without looking at the sites in question, it sounds like they're detecting the Google spider user-agent and feeding it false information --- which is strictly against the rules. I suppose you should report them, but I don't know if it'll do any good.

When browsing through the lower rent neighbourhoods of the internet, it's sometimes interesting to set your browser's user-agent to rep

Why does google use their own user agent? It seems obvious that this is going to result in people abusing it in this way. Why wouldn't they just use the user agent string of a popular browser? If sites don't want to be searched at all, they can use robots.txt. I don't see the legimiate use for being able to give different data to the spider than to everyone else.

Spiders don't just download spiders.txt and then proceed to download the entire server. They are designed to spread their downloading out over time so as not to overwhelm servers. Besides, they could just have one spider downloading robots.txt and another downloading everything else.

I have a hard time following this "they should do x before doing y". As a rather large company, Google has enough to people to work on both x and y at the same time. So this sentiment that "if they're doing x, they must not be doing y" is stupid.And I don't want to hear, "oh but I don't see a solution, so they must not be working on y". Bullshit. The issue of ClickFraud is not an easy one to solve. And if you think you can solve it... Then open your own Ad company and put google's ad division out of bu

I once got $50 from single post. BUT... it was close to Christmas and my list included affiliate links to amazon books and it got moderated +5 Interesting. I haven't been able to do that since... but I haven't really tried either. It was a fluke.

It misjudged the root problem -- page rank isn't the only thing spammers are looking for. This is the main point of Justin Mason's post, if I remember correctly. (I read it at home a few hours ago, before it showed up on./, and of course now I can't pull it up here.)

It relied on near-universal implementation. If even 50% of blogs, wikis, etc. used nofollow, it would still be worth the spammers' effort to blast comment spams to the entire net.

It got applied incorrectly, as a blanket label on all links from non-admins.

What nofollow could have been useful for is a simple "I don't endorse this link" statement so that you can link to sites you dislike without adding to their fame. But applying it to all user-supplied links in blog comments, slashdot threads, wiki pages, etc. diluted its meaning, and as a result, diluted its usefulness.

What nofollow could have been useful for is a simple "I don't endorse this link" statement so that you can link to sites you dislike without adding to their fame. But applying it to all user-supplied links in blog comments, slashdot threads, wiki pages, etc. diluted its meaning, and as a result, diluted its usefulness.

From the comments of TFA:

pudge said,May 31, 2006 @ 4:54 pm

I implmented nofollow for Slashdot. And I did it not primarily to reduce comment spam which our moderation system and other tools handle pretty well already, as Slashdot gets very littler comment spam but to reduce the effects of comment spam on search engines. If you post with a comment bonus (which you can get with high karma), you get no nofollow attribute, because we figure, chances are, your links will be useful to the search engines.

I didn't know that about the SlashCode, but it makes sense.

The type of person who blasts multiple sites with automated software isn't likely to spend time building up karma on (multiple) Slashdot accounts for his SpamBot to burn.

Google "darkmeridian" and basically any slashdot topic for the last year or so that I have posted in and that thread will pop up. The result of Slashdot's high PageRank, the relative uniqueness of my screenname, as well as my pathetic devotion to Slashdot is really scary.

I was going to update my nofollow story [homelandstupidity.us] from a year ago, but it seems nothing's changed -- except that blog spam has dramatically increased. Which anybody could have predicted -- and most people who thought about it for more than a second actually did.

Nofollow was a hare-brained idea from the start, cooked up over a couple of apparently drunken [msdn.com] (or perhaps stoned) nights between developers at Google, Yahoo! and MSN.

While nofollow failed to stop comment spam, that doesn't mean that it failed. On the contrary, it worked quite well. It's only bloggers who still have a comment spam problem that believe it failed. Contrary to the begged question, nofollow was never intended to stop comment spam. It was only intended to stop comment spam from affecting page rank. This is an important distinction.

Comment spam that affects page rank is Google's problem. Comment spam that doesn't have anything to do with page rank is not Google's problem. Google provided a mechanism to bloggers to eliminate a nuisance caused by Google's page rank algorithm. At that point, comment spam is no longer motivated by Google's page rank algorithm.

This blog entry is nothing but sour grapes that Google didn't solve their problem for them.

This blog entry is nothing but sour grapes that Google didn't solve their problem for them.

I agree completely. After reading TFA (and TFA's TFA), I got the impression that the original authors were naively thinking, "Nofollow isn't helping me personally, and if it has no benefit to me, why would Google do it?"

It relied on near-universal implementation. If even 50% of blogs, wikis, etc. used nofollow, it would still be worth the spammers' effort to blast comment spams to the entire net.

I don't believe this is the case. Comment spammers have a tendency to write scripts to bulk submit comments to particular locations across multiple hosts like/submit-comment.php that correspond to popular weblog software. You can't just "blast comment spams to the entire net", you need to target particular implementations.

Because, like you pointed out, pagerank is not the sole motivation for all spammers. I was quite clear in saying that there was no point for pagerank purposes.

I agree that at least some comment spammers are motivated by things other than pagerank, but I do believe that at least some of them are just after pagerank, and I believe nofollow moves them on to lower-hanging fruit (i.e. other websites that hav

As I posted elsewhere, spammers don't need to target specific scripts, since they have tools that just spider the net for <form>s and POST them. They may target certain popular scripts specifically anyway, but it's no requirement.

I don't believe this is the case. Comment spammers have a tendency to write scripts to bulk submit comments to particular locations across multiple hosts like/submit-comment.php that correspond to popular weblog software.

Incorrect. I've written various message board scripts, and spam bots hit those even though they are nowhere near popular enough to show up on spammers' radars. There are plenty of generic comment spam scripts out there, that will just look for suitable s and POST them.

I think people are being overly harsh on Google with the NoFollow business. I doubt Google ever though that NoFollow would cure the comment spam problem, I certainly didn't see any claims to that affect.

What NoFollow was designed to do, I suspect, was ensure that Google's Pagerank algorithm wasn't exacerbating the problem. Page rank may have have only been a marginal driver of comment spam, but Google was attempting to ensure that that particular margin was removed.

What nofollow could have been useful for is a simple "I don't endorse this link" statement so that you can link to sites you dislike without adding to their fame. But applying it to all user-supplied links in blog comments, slashdot threads, wiki pages, etc. diluted its meaning, and as a result, diluted its usefulness.

But that's exactly what nofollow is for. While you may only want to "nofollow" links for sites you hate, many sites use it for all external links they don't specifically endorse.

1) It reduces the impact of comment spam, forum spam, and wiki spam on the search engines that every web user relies on to get their work and play done.

As a side effect, very wide implementation *could*, hypothetically, one day lead to link spammers giving up on at least some of their spamming in the long run. Cool if it happens, but *not* required to reap the benefits.

Universal implementation is not required; every little bit helps. It's just part of being a responsible web site operator, like avoiding open relay configurations is part of being a responsible mail server operator. Closing open relays doesn't prevent all spam either, but it helps reduce the number of avenues it can creep through and thus helps reduce the impact.

Open comment systems, forums, and wikis are like open mail relays. If you must run one, being responsible about the impact you know it will have on the web ecosystem seems like a very good idea to me. Nofollow is a useful and important part of that impact mitigation.

Does it solve every problem everywhere at once? No. Does it help to do particular things in the real world here and now? Yes.

. . . being responsible about the impact you know it will have on the web ecosystem seems like a very good idea to me.

Being responsible about the 'web ecosystem' means working to prevent it from becoming a monoculture. Part of the monoculture is Google itself. There shouldn't be 'one big' search engine that everyone uses. There should be multiple search engines. I recently started using a different one for some of my searches and have found the lack of 'spamming' links refreshing.

Being responsible about the 'web ecosystem' means working to prevent it from becoming a monoculture. Part of the monoculture is Google itself. There shouldn't be 'one big' search engine that everyone uses. There should be multiple search engines. I recently started using a different one for some of my searches and have found the lack of 'spamming' links refreshing.

THe web ecosystem should evolve on its own. If millions of people flock to google, so be it. If they decide to use another search engine...that

Part of 'evolving on it's own' is for people to make informed choices on what search tool to use. Part of making informed choices is for people to advocate one search tool or another regularly as part of the discussion.

Millions of people flocked to AOL. Thanfully, it now looks like that wasn't a permanent comittment on their parts.

The whole point of the NoFollow Attribute wasn't necessarily to immediately decrease blog-spam -- it was to reduce it's detriment to Google and other indexes.

In this sense, it has probably succeeded. Sure a reduction in spam would have been nice, but this is still a nice first step. People always say spam is primarily an economic problem, so removing incentives is a good way to snuff it out in the long run.

I think that is a decision for the managers of blogs and forums to make. I have no financial interest in WOXY.com, and I think it is a little different than selling VAL/UM and PRoZ^ ck. In addition, they offer free streams and free forums.

I read the linked article on Mirrordot [mirrordot.org] and I have to say it's stultifying. Google added the "nofollow" tag to indicate that its crawler shouldn't follow the links on a given page. Some ignorant people apparently assumed that this would eliminate comment spam on blogs and other commentable media. When it became evident that it won't, it somehow became the fault of Google - their "embarrassing mistake." Spammers will continue to spam forever, there's no reason for them to stop. And this statement [tweney.com] is even more stupifying:

Worse, nofollow has another, more pernicious effect, which is that it reduces the value of legitimate comments. Here's how:

Why should I bother entering a comment on your blog, after all? Well, I might comment because you're my friend. But I might also want some tiny little reward for participating in a discussion, contributing to the content on your site, and generally enhancing the value of the conversational Web. That reward? PageRank, baby. But if your blog uses the nofollow tag, you've just eliminated that tiny little bit of reciprocity. Thanks, but no thanks. I'd rather just comment on my own blog. And maybe, if you're lucky, I'll link back to you.

So people only post on web forums for "greedy" reasons? If that's your motivation then I'm glad you won't be posting anywhere. If your comment isn't worth making just for the point you're trying to convey then I'd rather you keep it to yourself than just post to boost your own PageRank... your post is only marginally better than the spam you're complaining about anyway.

Well, actually, the Google post just says it will prevent spammers from benefiting by spamming blogs, it doesn't say that it will stop them from doing it. And just because Google doesn't follow the link doesn't mean there aren't other spiders (or humans) who won't follow it, so there's still benefits for spammers, just maybe not within Google. But people who thought this simple change would curb spammers must have been delusional, if a billion spam posts result in 20 penis pill sales then it's still worth

And furthermore - it's not as though they're disallowing signatures or personal links entirely - just the ability to affect pagerank with them. But why should that concern someone ? I have a link in my Slashdot signature - if they put nofollow in the URL, it wouldn't matter to me, because I'm not trying to coerce web crawlers into giving me a better ranking - I'm trying to get PEOPLE, who may potentially find the sites interesting, to go to them and try them out!

it seems to me that more and more people are devoted to the bottom-line these days, you can see that in the flood of "ends justifies the means" stories [boingboing.net] floating around. they just reinforce my feeling that people are awfully concerned with "what's in it for them" at all times, and now people can carry that over to posting online...they'll do it

Apparently Slashdot does put in a nofollow tag if you post with karma bonus!? Is this true? If I put a link to my site Desktop Linux At Home [desktoplinuxathome.com], does it not get a nofollow tag? I will post this and find out.

PageRank itself is the problem. It worked in 98 before everyone knew about it, now that they know the tricks, every search brings up forums and spam instead of the most relevant site. AdSense made the problem worse by letting spam sites turn an easy profit. Surely with all those PHD's there Google can come up with a more modern solution. Otherwise...wheres the next Google? Clusty.com ?

Surely with all those PHD's there Google can come up with a more modern solution. Otherwise...wheres the next Google? Clusty.com ?

I was thinking that myself. However, I came to realize that Clusty really only even WORKS because they leech off the search results of Google to begin with. If google goes down, Clusty will be tied to them as well.

Well, it may be more that people just don't understand pagerank. Pagerank has a *very small* overall impact on search results. If competition is tight, then this difference might be important, but in many cases it simply is not. What is important is what's always been important, content and code.The mystery surrounding pagerank is a big part of why people overestimate it. It's hard to say exactly how much of an impact it has, and exactly how it is calculated. How google determines if a site is "linking

Yes, and it's still not a word.* Even if one were to allow it as an adverb, as the sites you link to suggest - and I cannot begin to think how one would use it - one cannot use it here. It comes from the same school of people who think "irregardless" is an acceptable word. In the latter case, both the Oxford American and MW have entries, but neither consider it particularly acceptable (it contains two negatives, after all), and both recommend "regardless" instead.

If you're going to play the pedant, you have to expect pedantic responses. It IS a word! I think an American dictionary (like the Merriam-Webster that grandparent linked to) would be a more reliable (and more "proper") source on American usage than a British dictionary, like the Oxford. My own personal MW also has "relatedly" listed. It's a word! (At least, hyar in Amurricuh.)Now, if you want to argue that it shouldn't be a word, I might be sympathetic. If you want to claim that it's an awkward word,

OK, it's a word. But it's not a word. It's a vile creation, the product of a deficient intellect, and its existence is prolonged by induhviduals of similarly deficient intellects. Your phrasing may be different, but I am sure we agree.

But just a point on my choice of reference - I used the Oxford American Dictionary, whatever that is. Given that it lists all my spellings as "chiefly British", and, in my experience, omits certain British turns of phrase, does that make it American enough?